Don’t forget the 2014 U.S. House Legislative Data and Transparency Conference this week.
I’m now hard at work on our second generation web-based XML editor. In my blog last week, I talked about the need for and complexities of change tracking in a legislative editor. In this blog, I want to describe more of the overall motivation.
A couple years ago, we built an HTML5-based legislative editor for Akoma Ntoso. We learned a lot from the effort and had some success with a couple customers whose needs matched the capabilities of the editor. The editor was built to use and exploit, to the fullest extent, many of the new APIs added to modern browsers to support HTML5. We found that, by focusing on HTML5, a lot of the complexities of dealing with browser quirks and incompatibilities were a thing of the past – allowing us to focus on building the editing functions.
The editor worked by transforming the XML document into a close representation of the XML, expressed as HTML5 tags. Using HTML5 features such as the @contenteditable attribute along with modern CSS, the browser DOM, selection ranges, drag and drop, and a WebDAV repository API, we were able to implement a fairly sophisticated web-based legislative editor.
But, not everything went smoothly. The first problem involved the complexity of mapping all the intricacies of XML into an HTML5 representation, and then maintaining that representation in the browser. Part of the difficulty stems from the fact the HTML5 is not specifically an XML dialect – and browsers tend to do HTML5 things that aren’t always XML friendly. The HTML5 DOM is deliberately rather loose and forgiving (it’s a big part of why HTML was successful in the first place) while XML demands a very precise and rigid DOM.
The second problem we faced was scalability. While the HTML5 representation wasn’t all that heavyweight, the bigger problem was the transformation cost going back and forth between HTML5 and XML. We sometimes deal with very large legislation and laws. In our bigger cases, the cost of transformation was simply unreasonable.
So what is the solution? Well, early last year we started experimenting with using a browser to render XML documents with a CSS directly – without any transform into HTML. Most modern browsers now do this very well. For the most part, we were able to achieve an acceptable rendition in the browser without any transformation.
There were a few drawbacks to this approach. For one, links were dead – they didn’t inherently do anything. Likewise, implementing something like the HTML @style attribute didn’t just naturally work. Before we could entertain the notion of a pure XML-based editor built within the XML infrastructure in the browser, we had to find a solution that would allow us to enrich the XML sufficiently to allow it to behave like an HTML page.
Another problem arose in that our prior web-based editor relied upon the @contenteditable feature of HTML. That is an HTML feature rather than a browser feature. Using XML as our base environment, we no longer had access to this facility. This wasn’t a total loss as our need for a rich change tracking environment required us to find a better approach that @contenteditable offered anyway.
With solutions to the major problems behind us, we started to take a look at the other goals for the editor:
- Track Changes – This was the subject of my blog last week. For us, track changes is crucial in any editor targeted at legislation – and it must work at both the structural and textual level equally well. We use the feature for two things – redlining changes as is common in the U.S. and the automatic generation of amendment documents (amendments in context). Differencing can get you part way there – but it excludes the ability to adequately craft the changes in a way that deal with political sensitivities. Track Changes is a very complex feature which must be built into the very core of the editor – tacking it on later will be very difficult, if not impossible.
- Scalability – Scalability is very important to our applications. We need to support very large documents. Even when we deal with document fragments, we need to allow those fragments to be very large. Our approach is to create editing islands within a large document loaded into the browser. This amounts to only building the editing superstructure around the parts of the document being edited rather than the whole document. It’s like building the scaffolding around only the floors being worked on in a skyscraper rather than trying to envelope the entire building in scaffolding.
- Configurability – We need to support a number of different models – not just Akoma Ntoso. To achieve this, an XML-based configuration file is used to define the behaviors for any XML model. Elements can be defined as read-only, templates can be defined (or derived), and even the track changes behavior can be configured for individual elements. The sophistication being defined within the configuration files is to allow us to model all the variants of legislative models we have encountered without the need for extensive programming-level customization.
- Browser Support – We’re pushing the envelope when it comes to browser support. Our current focus is on Google’s Chrome browser. Support for all the browsers aside from Internet Explorer should be relatively easy. Our experience has shown that the browsers are now quite similar. Internet Explorer is the one exception – in this particular area. Years ago, IE was the best browser when it came to XML support. While IE had many other compatibility issues, particularly with CSS, it led the way in supporting XML. However, while Microsoft has made tremendous strides moving forward to match the other browsers and modern standards, they’ve neglected XML. Their circa 1999 legacy capabilities for XML do no match modern standards and are quite deficient. Hopefully, this is something that will soon be rectified.
It’s not all smooth sailing. I have been finding a number of surprising issues with Google Chrome. For instance, whitespace management is a bit fudged at times. Chrome thinks nothing of adding the occasional non-breaking space to maintain whitespace when editing the DOM. What’s worse – it will inexplicably convert this into a text node that reads ” ” after a while. This is a character entity that is not defined in XML. I have to work hard to constantly reverse this odd behavior.
All in all, I’m excited by this new approach to building a web-based XML editor. It’s a substantial increase in sophistication over our prior web-based XML editor. This editor will be far more robust, scalable, and configurable in comparison to our prior editor and other editors we have worked on. While we still have a way to go in our development, we’ve found solutions to all the risky issues. It’s a future-looking approach – support can only get better. It doesn’t rely on compatibility modes or any other remnants of prior eras in web technology. This approach is really working out quite nicely for us.
4 thoughts on “Building a browser-based XML Editor”
Didn’t Arbortext have a browser hosted XML editor a few years ago? I don’t know if pre-PTC Arbortext discontinued the product or if PTC did when they bought them up, but it does not seem to be around any more.
I was updating a GoogleDocs spreadsheet yesterday and I am always amazed at the control they have of the GUI in the browser. With you XML editor design I am surprised that I don’t see more about a Model-View-Controller architecture. Maybe MVC is so baked in that it is not worth mentioning. But when you talk about “the complexity of mapping all the intricacies of XML into an HTML5 representation” it sounds like you need another layer of abstraction.
I enjoy reading about all your activities. Living the dream.
I don’t know if Arbortext did, but XMetaL still do – it’s something called XMAX. However, it’s just an ActiveX (remember that?) plugin version of the XMetaL desktop application.
The thing about GoogleDocs that is so amazing is the extent they go to in order to control the GUI. They essentially use the browser as a canvas and then implement everything on top themselves. The end result allows them to mimic desktop applications very nicely in the browser, but the underlying information model is incredibly intricate and tortured. Occasionally you will see a quirk in their canvas – and the cursor with separate itself from the edit point you’re typing into. This is one of hints that betrays their implementation. This is all quite different from the Writely word processor that they initially bought. It took a far more conventional approach.
My new approach is less MVC than it was with the HTML5 implementation I used before. In the last effort, I managed my own XML DOM and then rendered it into an HTML5 view. The biggest frustration was dealing with all the HTML, but not quite XML, aspects that would constantly creep into the picture. I could take the GoogleDocs approach, and build a nice abstraction, but I would need a Google-sized army to help me. In the new effort, I’m relying much more on the fact that modern browsers know how to deal with XML very nicely – once you kick them into an XML mode. I leave the basics to the browser and simply wrap my editing scaffolding around whatever part of the document needs to be edited. The underlying XML DOM is provided by the browser – including namespace and XPath support. Rendering is simply managed using CSS (with a few tricks of my own).
Pingback: Imagining Government Data in the 21st Century « Legix.info
Interesting work. ContentEditable is just full of these little surprises like the Chrome quirk you mention. Any developer who attempts to create something kind of serious using cE ends up doing a lot of work to keep some control over the DOM (and never succeeds 100%).
There is currently a discussion going on W3C’s public-webapps mailing list on a possible new alternative implementation of cE that would give much more control to the developer. See this message (and the 100 or so followups):
Given your background I thought you might be interested in hearing that people are talking about making things better, and possibly contributing an additional developer’s perspective to the conversation.