The Sun is Rising on Akoma Ntoso — and LegisPro too!

Two great news piece of news this week! First, the documentation for Akoma Ntoso has now been officially released by OASIS. Second, we’re announcing the latest version of  our LegisPro drafting platform for Akoma Ntoso, codenamed “Sunrise”.

After several years of hard work, we’ve made a giant step towards our goal of setting an international XML standard for legal documents. You can find the documents at the OASIS LegalDocML website. A special thanks to Monica Palmirani and Fabio Vitali at the University of Bologna for their leadership in this endeavour.

legispro250Later this week, Xcential will be announcing and showing the latest version of “Sunrise” version of LegisPro, at both NALIT in Annapolis, Maryland and at the LEX Summer School in Ravenna, Italy. This new version represents a long-planned change to Xcential’s business model. While we have a thriving enterprise business, we’re now focusing on also providing more affordable solutions for smaller governments.

Part of our plan is to foster an open community of providers around the Akoma Ntoso standard for legislative XML. With Akoma Ntoso now in place as a standard, we’re looking for ways to provide open interfaces such that cooperative tools and technologies can be developed. One of my goals at this years summer school in Ravenna is to begin outlining the open APIs that will enable this vision.

LegisPro

The new edition of LegisPro will be all about providing the very best options:

  1.  It will provide a word processing like drafting capability your drafters demand — along with the real capabilities you need:
    • We’re not talking about merely providing a way to style a word processing document to look like legislation.
    • We’re talking about providing easy ways to define the constructs you need for your legislative traditions, such as–
      • a configurable hierarchy,
      • configurable tagging of important information,
      • configurable numbering rules,
      • configurable metadata,
      • oh, and configurable styles too.
    • We’re talking about truly understanding your amending traditions and providing the mechanisms to support them, such as–
      • configurable track changes, because we understand that a word processor’s track changes are not enough,
      • as-published page and line markers, because we understand your real need for page and line numbers and that a word processor’s page and line numbering is not that,
      • robust typography, because we know there’s a quite a difference between the casual correspondence a word processor is geared for and the precision demanded in documents that represent laws and regulations.
  2. It will be as capable as we can make it — for real-world use rather than just a good demo:
    • We’re not talking about trying to sell you a cobbled together suite of tools we built for other customers.
    • We’re talking about working with specialists in all the sub-fields of legal informatics to provide best-of-breed options that work with our tools.
    • We’re talking about making as many options available to you as we know there is no one-size-fits-all answer in this field.
    • We’re talking about an extensible architecture that will support on-board plug-ins as well as server-side web-services.
    • We’re talking about providing a platform of choices rather than a box of pieces.
  3. It will be as affordable as we can possibly make it:
    • We’re talking about developing technologies that have been designed to be easily configured to meet a wide variety of needs.
    • We’re talking about using a carefully chosen set of technologies to minimize both your upfront cost and downstream support challenges.
    • We’re talking about providing a range of purchasing options to meet your budgetary constraints as best we can.
    • We’re talking about finding a business model that allows us to remain profitable — and spreads the costs of developing the complex technologies required by this field as widely and fairly as possible.
  4. It is as future-proof as we can possibly make it:
    • We’re not talking about trying to sell you on a proprietary office suite.
    • We’re talking about using a carefully curated set of technologies that have been selected as they represent the future of application development — not the past — including:
      • GitHub’s Electron which allows us to provide both a desktop and a web-based option, (This is the same technology used by Slack, WordPress, Microsoft’s Visual Studio Code, and hundreds of other modern applications.)
      • Node.js which allows us to unify client-side and server-side application development with “isomorphic JavaScript,”
      • JavaScript 6 (ECMAScript 2015) which allows us to provide a truly modern, unified, and object-oriented programming environment,
      • Angular and other application frameworks that allow us to focus on the pieces and not how they will work together,
      • CSS3 and LESS that allows us to provide state-of-the-art styling technologies for the presentation of XML documents,
      • the entire XML technology stack that is critical for enabling an information-centric rather than document-centric system as is appropriate for the 21st century,
      • and, of course, using the Akoma Ntoso schema for legislative XML to provide the best model for sharing data, information, tools, and other technologies. It’s truly a platform to build an industry on.
  5. It is as open as we can possibly make it:
    • We’re not talking about merely using an API published by a vendor attempting to create a perception of openness by publishing an API with “open” in the name.
    • We’re talking about building on a full suite of open source tools and technologies coming from vendors such as Google, GitHub, and even Microsoft.
    • We’re talking about using non-proprietary protocols such as HTTP and WebDAV.
    • We’re talking about providing an open API to our tools that will also work with tools of other vendors that support Akoma Ntoso.
    • And, while we must continue to be a profitable product vendor, we will still provide the option of open access to our GitHub repositories to our customers and partners. (We’ll even accept pull requests)

Our goal is to be the very best vendor in the legislative and regulatory space, providing modern software that helps make government more efficient, more transparent and more responsive. We want to provide you with options that are affordable, capable, and planned for the future. We want to do whatever we can to allay your fears of vendor lock-in by supporting open standards, open APIs, and open technologies. We want to foster an Akoma Ntoso-based industry of cooperative tools and technologies as we know that doing so will be in the best interests of everyone — customers, product vendors, service providers, and the people who support them. As someone once told me many years ago, if you focus on making the pie as large as you can, the crumbs left on the knife will be plenty enough for you.

Either come by our table at NALIT in Annapolis or join us for the Akoma Ntoso Developer’s Conference in Ravenna at the conclusion of the LEX Summer School to learn more. If neither of these options will work for you, you can always learn more at Xcential.com or by sending email to info@xcential.com.

The Sun is Rising on Akoma Ntoso — and LegisPro too!

Escaping a Technology Eddy

Do you need to escape a technology eddy? In fluid dynamics, an eddy is the swirling of a fluid that causes a reverse current against a downstream flow. It often forms behind a major obstacle. The swirling motion of an eddy creates resistance to forward motion by creating a backward force. Eddies are also seen in air and electromagnetic systems.

I see a similar phenomena in my work that I’m going to coin a technology eddy. A technology eddy forms in organisations that are risk adverse, have restricted budgets, or simply are more focused on software maintenance of a major system rather than on software development. Large enterprises, in particular, often find their IT organisations trapped in a technology eddy. Rather than going with the flow of technological change, the organisation drifts into a comfortable period where change is more restricted to the older technologies they are most familiar with.

TechnologyEddy

As time goes by, an organisation trapped in a technology eddy adds to the problem by building more and more systems within the eddy — making it ever more difficult to escape the eddy when the need arises.

I sometimes buy my clothing at Macy’s. It’s no secret that Macy’s, like Sears, is currently struggling against the onslaught of technological change. Recently, when paying for an item, I noticed that their point-of-sale systems still run on Windows 7 (or was that Windows Vista). Last week, on the way to the airport, I realised I had forgotten to pack a tie. So, I stopped in to Macy’s only to find that they had just experienced a 10 minute power outage. Their ancient system, what looked to be an old Visual Basic Active Directory app, was struggling to reboot. I ended up going to another store — for all the other stores in the mall were up and running quite quickly. The mall’s 10 minute power outage cost Macy’s an hour’s worth of sales because of old technology. The technology eddy Macy’s is trapped in is not only costing them sales in the short term, it’s killing them in the grand scheme of things. But I digress…

I come across organisations trapped in technology eddies all the time. IT organisations in government are particularly susceptible to this phenomena. In fact, even Xcential got trapped in a technology eddy. With a small handful of customers and a focus on maintenance over development for a few years, we had become too comfortable with the technologies that we knew and the way in which we built software.

It was shocking to me when I came to realise just how out-of-date we had become. Not only were we unaware of the latest technologies, we were unaware of modern concepts in software development, modern tools, and even modern programming styles. We had become complacent, assuming that technology from the dawn of the Millennium was still relevant.

I hear a lot of excuses for staying in a technology eddy. “It works”, “all our systems are built on this technology”, “it’s what we know how to build”, “newer technologies are too risky”, and so on. But there is a downside. All technologies rise up, have a surprisingly brief heyday, and then slowly fade away. Choosing to continue within a technology eddy using increasingly dated technology ensures that sooner or later, an operating system change or a hardware failure of an irreplaceable part will create an urgent crisis to replace a not-all-that-old system with something more modern. At that point, escaping the eddy will be of paramount importance and you’ll have to paddle at double speed just to catch up. This struggle becomes the time when the price for earlier risk mitigation will be paid — for now the risks will compound.

So how do you avoid the traps of a technology eddy? For me, the need to escape our eddy became most apparent as we got exposed to people, technologies, and ideas that were beyond the comfortable zone in which our company existed. Hearing new ideas from developers beyond our sphere of influence and being exposed to requirements from new customers made us quickly realize that we had become quite old-fashioned in our ways. To stay relevant you must get out and learn — constantly. Go to events that challenge your thinking rather than reinforce it.

Today we are once more a state-of-the-art company. We’ve adopted modern development techniques, upgraded our tools, upgraded our technologies, and upgraded our coding skills. These changes allow us to compete worldwide and build software for multiple customers in a fully distributed way that spans companies, continents, and time zones.

I hope we’ll remember this lesson and focus more on continuous improvement rather than having to endure a crash course of change every few years.

 

Escaping a Technology Eddy

Data Transparency Breakfast, LEX US Summer School 2015, First International Akoma Ntoso Conference, and LegisPro Edit reveal.

Last week was a very good week for my company, Xcential.

We started the week hosting a breakfast put on by the Data Transparency Coalition at the Booz Allen Hamilton facility in Washington D.C.. The topic was Transforming Law and Regulation. Unfortunately, an issue at home kept me away but I was able to make a brief pre-recorded presentation and my moderating role was played by Mark Stodder, our company President. Thank you, Mark!

Next up was the first U.S. edition of the LEX Summer School from Italy. I have attended this summer school every year since 2010 in Italy and it’s great to see the same opportunity for an open dialog amongst the legal informatics community finally come to the U.S. Monica Palmirani (@MonicaPalmirani), Fabio Vitali, and Luca Cervone (@lucacervone) put on the event from the University of Bologna. The teachers also included Jim Mangiafico  (@mangiafico) (the LoC data challenge winner), Veronique Parisse (@VeroParisse) from the European Union, Andrew Weber (@atweber) from the Library of Congress, Kirsten Gullickson (@GullicksonK) from the Office of the Clerk at the U.S. House of Representatives, and myself from Xcential. I flew in for an abbreviated visit covering the last two days of the Summer School where I covered how the U.S. Code is modeled in Akoma Ntoso and gave the students an opportunity to try out our new bill drafting editor — LegisProedit.

After the Summer School concluded, it was followed by the first International Akoma Ntoso Conference on Saturday, where I spoke about the architecture of our new editor as well as how the USLM schema is a derivative of the Akoma Ntoso schema. We had good turnout, from around the world, and a number of interesting speakers.

This week is NCSL in Seattle where we will be discussing our new editor with potential customers and partners. Mark Stodder from Xcential will be in attendance.

In a month, I’ll be in Ravenna once more for the European LEX Summer School — where I’ll be able to show even more progress towards the goal of a full product line of Akoma Ntoso tools. It’s interesting times for me.

The editor is coming along nicely and we’re beginning to firm up our QuickStarter beta plans. I’ve already received a number of requests and will be getting in touch with everyone as soon as we’re ready to roll out the program. If you would like to participate as a beta tester — or if you would just like more information, please contact us at info@xcential.com.

I’m really excited about how far we’ve come. Akoma Ntoso is on the verge of being certified as an official OASIS standard, our Akoma Ntoso products are coming into place, and interest around the world is growing. I can’t wait to see where we will be this time next year.

Data Transparency Breakfast, LEX US Summer School 2015, First International Akoma Ntoso Conference, and LegisPro Edit reveal.

Coming soon!!! A new web-based editor for Akoma Ntoso

I’ve been working hard for a long time — building an all new web-based editor for Akoma Ntoso. We will be showing it for the first time at the upcoming Akoma Ntoso LEX Summer School in Washington D.C.

Unlike our earlier AKN/Editor, this editor is a pure XML editor designed from the ground up using the XML capabilities that modern browsers possess. This editor is much more robust, more precise,  and is very scalable.

NewEditor

Basic Features

  1. Configurable XML models — including Akoma Ntoso and USLM
  2. Edit full documents or portions of large documents
  3. Flexible selection and editing regardless of XML structure
  4. Built-in redlining (change tracking) supporting textual AND structural changes
  5. Browse document sources with drag-and-drop.
  6. Full undo & redo
  7. Customizable attribute editor
  8. Search and replace
  9. Modular architecture to allow for extensive customization

Underlying Technology

  1. XML-based editing component
    • DOM 4 support
    • XPath Support
    • CSS Styling
    • Sophisticated event model
  2. HTTP-based resolver architecture for retrieving documents
    • Interpret citations
    • Deference URLs
    • WebDAV adaptors to document repositories
    • Query repositories with XQuery or databases with SQL
  3. AngularJS-based User Interface using HTML5
    • Component modules for easy customization
  4. XML repository for storing documents
    • Integrate any XML repository
    • Built-in support for eXist-db
  5. Validation & Publishing
    • XML Schema validator
    • XSL-FO publishing

We’ll reveal a lot more at the LEX Summer School later this month! If you’re interested in our QuickStart beta program, drop me a note at grant.vergottini@xcential.com.

Coming soon!!! A new web-based editor for Akoma Ntoso

Upcoming U.S. and European events related to Akoma Ntoso

In my last blog post I covered the public review of the new proposed Akoma Ntoso (LegalDocML) standard for legal documents. Please keep the comments coming. In order to comment, please send email to legaldocml-comment@lists.oasis-open.org. If you wish to subscribe to this mailing list, please follow the instructions at https://www.oasis-open.org/committees/comments/index.php?wg_abbrev=legaldocml

In addition, there are three upcoming events related to Akoma Ntoso which you may wish to participate in: (this list coming from Monica Palmirani, the chair of the OASIS LegalDocML technical committee)

1. Akoma Ntoso Summer School, 27-31 July, 2015, George Mason University, Fairfax, Virginia (USA): http://aknschool.cirsfid.unibo.it
Registration fee: http://aknschool.cirsfid.unibo.it/logistics/registrations-and-fees/
Application Form: http://aknschool.cirsfid.unibo.it/wp-content/uploads/2015/05/ApplicationForm.pdf
Brochure:
http://aknschool.cirsfid.unibo.it/wp-content/uploads/2015/05/brochure_2015_US_DEF.pdf
Deadline: end of June, 2015.

2. IANC2015 (First International Akoma Ntoso Conference): August 1st, 2015, George Mason University, Fairfax, Virginia (USA)
Brochure: http://aknschool.cirsfid.unibo.it/wp-content/uploads/2015/05/AKN-CONFERENCE1.pdf
Call for contributions:
http://www.akomantoso.org/akoma-ntoso-conference/call-for-contributions/
Deadline: June 19th, 2015.

3. Summer School LEX2015, 7-15 Sept. 2015, Ravenna, Italy: http://summerschoollex.cirsfid.unibo.it
Registration fee: http://summerschoollex.cirsfid.unibo.it/?page_id=66
Application Form: http://summerschoollex.cirsfid.unibo.it/wp-content/uploads/2010/04/ApplicationForm2.pdf
Brochure:
http://summerschoollex.cirsfid.unibo.it/wp-content/uploads/2015/05/brochure_2015_LEX1.pdf
Deadline: July, 15th, 2015.

I have been participating in the European LEX Summer school every year since 2010 and find it to be both inspirational and very valuable. If you’re interested in understanding where the legal informatics field is headed, I encourage you to find a way to attend any of these events. I will be speaking/teaching at all three events.

Upcoming U.S. and European events related to Akoma Ntoso

Akoma Ntoso (LegalDocML) is now available for public review

It’s been many years in the making, but the standardised version of Akoma Ntoso is now finally in public review. You can find the official announcement here. The public review started May 7th and will end on June 5th — which is quite a short time for something so complex.

I would like to encourage everything to take part in this review process, as short as it is. It’s important that we get good coverage from around the world to ensure that any use cases we missed get due consideration. Instructions for how to comment can be found here.

Akoma Ntoso is a complex standard and it has many parts. If you’re new to Akoma Ntoso, it will probably be quite overwhelmed. To try and cut through that complexity, I’m going to try and give a bit of an overview of what the documentation covers, and what to be looking for.

There are four primary documents

  1. Akoma Ntoso Version 1.0 Part 1: XML Vocabulary — This document is the best place to start. It’s an overview of Akoma Ntoso and describes what all the pieces are and how they fit together.
  2. Akoma Ntoso Version 1.0 Part 2: Specifications — This is the reference material. When you want to know something specific about an Akoma Ntoso XML element or attribute, this is the document to go to. In contains very detailed information derived from the schema itself. Also included with this is the XML schema (or DTD if you’re still inclined to use DTDs). and a good set of examples from around the world.
  3. Akoma Ntoso Naming Convention Version 1.0. This document describes two very interrelated and important aspects of the proposed standard — how identifitiers are assigned to elements and how IRI-based (or URI-based) references are formed. There is a lot of complexity in this topic and it was the subject to numerous meetings and an interesting debate at the Coco Loco restaurant in Ravenna, Italy, one evening while being eaten by mosquitoes.
  4. Akoma Ntoso Media Type Version 1.0 — This fourth document describes a proposed new media type that will be used when transmitting Akoma Ntoso documents.

This is a lot of information to read and digest in a very short amount of time. In my opinion, the best way to try and evaluate Akoma Ntoso’s applicability to your jurisdiction is as follows:

  • First, look at the basic set of tags used to define the document hierarchy. Is this set of tags adequate. Keep in mind that the terminology might not always perfectly align with your terminology. We had to find a neutral terminology that would allow us to define a super-set of the concepts found throughout the world.
  • If you do find that specific elements you need are missing, consider whether or not that concept is perhaps specific to your jurisdiction. If that is the case, take a look at the basic Akoma Ntoso building blocks that are provided. While we tried to provide a comprehensive set of elements and attributes, there are many situations which are simply too esoteric to justify the additional tag bloat in the basic standard. Can the building blocks be used to model those concepts?
  • Take a look at the identifiers and the referencing specification. These parts are intended to work together to allow you to identifier and access any provision in an Akoma Ntoso document. Are all your possible needs met with this? Implicit in this design is a resolver architecture — a component that parses IRI references (think of them as URLs) and maps to specific provisions. Is this approach workable?
  • Take a look at the basic metadata requirements. Akoma Ntoso has a sophisticated metadata methodology behind it and this involves quite a bit of indirection at times. Understand what the basic metadata needs are and how you would model your jurisdictions metadata using this.
  • Finally, if you have time, take a look at the more advanced aspects of Akoma Ntoso. Consider how information related to the documents lifecycle and workflow might be modeled within the metadata. Consider your change management needs and whether or not the change management capabilities of Akoma Ntoso could be adapted to fit. If you work with complex composite documents, take a look at the mechanisms Akoma Ntoso provides to assemble composite documents.

Yes, there is a lot to digest in just a few weeks. Please provide whatever feedback you can.

We’re also now in the planning stages for a US LEX Summer School. If you’ve followed my blog over the years, you’ll know that I am a huge fan of the LEX Summer School in Ravenna, Italy — I’ve been every year for the past five years. This year, Kirsten Gullikson and I convinced Monica and Fabio to bring the Summer School to Washington D.C. as well. The summer school will be held the last week of July 2015 at George Mason University. The class size will be limited to just 30, so be sure to register early once registration opens. If you want to hear me rattle on at length about this subject, this is the place to go — I’ll be one of the teachers. The Summer School will conclude with a one day Akoma Ntoso Conference on the Saturday. We’ll be looking for papers. I’ll send out a blog with additional information as soon as it’s finalized.

You may have noticed that I’ve been blogging a lot less lately. Well, that’s because I’ve been heads down for quite some time. We’ll soon be in a position to announce our first full Akoma Ntoso product. It’s an all new web-based XML editor that builds on our experiences with the HTML5 based AKN/Editor (LegisPro Web) that we built before.

This editor is composed of four main parts.

  1. First, there is a full XML editing component that works with pure XML — allowing it to be quite scalable and very XML precise. It implements complex track changes capabilities along with full redo/undo. I’m quite thrilled how it has turned out. I’ve battled for years with XMetaL’s limitations and this was my opportunity to properly engineer a modern XML editor.
  2. Second, there is a sophisticated resolver technology which acts as the middleware, implementing the URI scheme I mentioned earlier — and interfacing with local and remote document resources. All local document resources are managed within an eXist-db repository.
  3. Third, there is the Akoma Ntoso model. The XML editing component is quite schema/model independent. This allows it to be used with a wide variety of structured documents. The Akoma Ntoso model adapts the editor for use with Akoma Ntoso documents.
  4. And finally, there is a very componentised application which ties all the pieces together. This application is written as an AngularJS-based single page application (SPA). In an upcoming blog I’ll detail the trials and tribulations of learning AngularJS. While learning AngularJS has left me thinking I’m quite stupid at times, the goal has been to build an application that can easily be extended to fit a wide variety of structured editing needs. It’s important that all the pieces be defined as modules that can either be swapped out for bespoke implementations or complemented with additional capabilities.

Our current aim is to have the beta version of this new editor available in time for the Summer School and Akoma Ntoso conference — so I’ll be very heads down through most of the summer.

Akoma Ntoso (LegalDocML) is now available for public review

Achieving Five Star Open Data

A couple weeks ago, I was in Ravenna, Italy at the LEX Summer School and follow-on Developer’s Workshop. There, the topic of a semantic web came up a lot. Despite cooling in the popular press in recent years, I’m still a big believer in the idea. The problem with the semantic web is that few people actually get it. At this point, it’s such an abstract idea that people invariably jump to the closest analog available today and mistake it for that.

Tim Berners-Lee (@timberners_lee), the inventor of the web and a big proponent of linked data, has suggested a five star deployment scheme for achieving open data — and what ultimately will be a semantic web. His chart can be thought of as a roadmap for how to get there.

Take a look at today’s Data.gov website. Everybody knows the problem with it — it’s a pretty wrapper around a dumping ground of open data. There are thousands and thousands of data sets available on a wide range of interesting topics. But, there is no unifying data model behind all these data dumps. Sometimes you’re directed to another pretty website that, while well-intentioned, hides the real information behind the decorations. Sometimes you can get a simple text file. If you’re lucky, you might even find the information in some more structured format such as a spreadsheet or XML file. Without any unifying model and with much of the data intended as downloads rather than as an information service, this is really still Tim’s first star of open data — even though some of the data is provided as spreadsheets or open data formats. It’s a good start, but there’s an awful long way to go.

So let’s imagine that a better solution is desired, providing information services, but keeping it all modest by using off-the-shelf technology that everyone is familiar with. Imagine that someone with the authority to do so, takes the initiative to mandate that henceforth, all government data will be produced as Excel spreadsheets. Every memo, report, regulation, piece of legislation, form that citizens fill out, and even the U.S. Code will be kept in Excel spreadsheets. Yes, you need to suspend disbelief to imagine this — the complications that would result would be incredibly tough to solve. But, imagine that all those hurdles were magically overcome.

What would it mean if all government information was stored as spreadsheets? What would be possible if all that information was available throughout the government in predictable and permanent locations? Let’s call the system that would result the Government Information Storehouse – a giant information repository for information regularized as Excel spreadsheets. (BTW, this would be the future of government publishing once paper and PDFs have become relics of the past.)

How would this information look? Think about a piece of legislation, for instance. Each section of the bill might be modeled as a single row in the spreadsheet. Every provision in that section would be it’s own spreadsheet cell (ignoring hierarchical considerations, etc.) Citations would turn into cell references or cell range references. Amending formulas, such as “Section 1234 of Title 10 is amended by…” could be expressed as a literal formula — a spreadsheet formula. It would refer to the specific cell in the appropriate U.S. Code Title and contain programmatic instructions for how to perform the amendment. In short, lots of once complex operations could be automated very efficiently and very precisely. Having the power to turn all government information into a giant spreadsheet has a certain appeal — even if it requires quite a stretch of the imagination.

Now imagine what it would mean if selected parts of this information were available to the public as these spreadsheets – in a regularized and permanent way — say Data.gov 2.0 or perhaps, more accurately, as Info.gov. Think of all the spreadsheet applications that would be built to tease out knowledge from the information that the government is providing through their information portal. Having the ability to programmatically monitor the government without having to resort to complex measures to extract the information would truly enable transparency.

At this point, while the linkages and information services give us some of the attributes of Tim’s four and five star open data solutions, but our focus on spreadsheet technology has left us with a less than desirable two star system. Besides, we all know that having the government publish everything as Excel spreadsheets is absurd. Not everything fits conveniently into a spreadsheet table to say nothing of the scalability problems that would result. I wouldn’t even want to try putting Title 42 of the U.S. Code into an Excel spreadsheet. So how do we really go about achieving this sort of open data and the efficiencies it enables — both inside and outside of government?

In order to realize true four and five star solutions, we need to quickly move on to fulfilling all the parts of Tim’s five star chart. In his chart, a three star solution replaces Excel spreadsheets with an open data format such as a comma separated file. I don’t actually care for this ordering because it sacrifices much to achieve the goal of having neutral file formats — so lets move on to full four and five star solutions. To get there, we need to become proficient in the open standards that exist and we must strive to create ones where they’re missing. That’s why we work so hard on the OASIS efforts to develop Akoma Ntoso and citations into standards for legal documents. And when we start producing real information services, we must ensure that the linkages in the information (those links and formulas I wrote about earlier), exist to the best extent possible. It shouldn’t be up to the consumer to figure out how a provision in a bill relates to a line item in some budget somewhere else — that linkage should be established from the get-go.

We’re working on a number of core pieces of technology to enable this vision and get to full five star open data. We integrating XML repositories and SQL databases into our architectures to give us the information storehouse I mentioned earlier. We’re building resolver technology that allows us to create and manage permanent linkages. These linkages can be as simple as citation references or as complex as instructions to extract from or make modifications to other information sources. Think of our resolver technology as akin to the engine in Excel than handles cell or range references, arithmetic formulas, and database lookups. And finally, we’re building editors that will resemble word processors in usage, but will allow complex sets of information to be authored and later modified. These editors will have many of the sophisticated capabilities such as track changes that you might see in a modern word processor, but underneath you will find a complex structured model rather than the ad hoc data structures of a word processor.

Building truly open data is going to be a challenging but exciting journey. The solutions that are in place today are a very primitive first step. Many new standards and technologies still need to be developed. But, we’re well on our way.

Achieving Five Star Open Data