Imagine All 50 States Legislation in a Common Format

Last week I expressed dissapointment over NCSL’s opposition to the DATA Act (H.R. 2146). Their reasoning is that the burden this might create on the state’s systems will not be affordable. Contrast this with the topic of the international workshop held in Brussels last week – “Identifying benefits deriving from the adoption of XML-based chains for drafting legislation“. The push toward more transparent government need not be unaffordable.

With that in mind, stop for a while and imagine having the text from all 50 states legislation publishing in a common XML format. Seem like an impossibly difficult and expensive undertaking doesn’t it? With all the requirements gathering, getting systems to cooperate, and getting buy-in throughout the country, this could be another super-expensive project that in the end would fail. What would such a project cost? Millions and millions?

Well, remember again Henry Ford’s quote “If you think you can do a thing or think you can’t do a thing, you’re right”. Would you believe that a system to gather and publish all 50 states has recently been developed, in months rather than years, and on a shoe-string budget? That system is It’s a 50 state bill tracking service. Check it out! We, at Xcential, helped them to do this herculian task by providing a simple and neutral XML and the software to do much of the processing. The press release is here. The format is SLIM, the same format the underlies my prototype. It’s a simple, easy-to-adopt XML format built on our past decade’s experience in legislative systems. Karen Sahuka at BillTrack50 recently gave a presentation on her product at the Non-profit Technology Conference in San Francisco.

SLIM is not as ambitious as Akoma Ntoso. If you take a gander at my site, you will see that it’s very easy to go from SLIM to Akoma Ntoso. In fact, going between any two formats is not all that difficult with modern transformation technology. It’s how we built the publishing system for the State of California as well. My point is that with the right attitude, a little innovation, and the right tools, achieving the modern requirements for accountability and transparency need not be out of reach.

Imagine All 50 States Legislation in a Common Format

5 thoughts on “Imagine All 50 States Legislation in a Common Format

  1. Gordy says:

    i’m sorry if i seem critical, but I just don’t see how this is accomplishing anything. can you point to something useful that is done by transforming text into a common format?

    BillTrack50 was developed I believe based on scraping efforts of 50OpenStates and other projects, the bill text in one format doesn’t seem vital to that, more the metadata.

    Can you explain why this isn’t just a waste of time and effort?

    1. grantcv1 says:

      Having a commn format allows you to achieve a number of things. My primary objectibe behind a common format is that is makes it possible for common tools or building blocks to be produced – in essence turning legal informatics from an academic pursuit and a collection of consulting firms into a real industry of interoperable tools. Today, each jurisdiction must resort to spending millions and millions of dollars in risky projects to craft full custom solutions for their internal drafting systems. And when they publish their data, they do so in some custom format that likely is not documented. That means everyone then has to write translators to consume these outputs. So if you’re in the business of tracking or publishing legislation, you’ve got a major undertaking to access the state’s data and you’ve always got to worry about the format changing in any number of ways. At the next level, by having a common format, the tools that are built to analyze legislation can be additive rather than repititive. People that track legislation are interested in tracking changes to proposed legislation, tracking references and interdependencies between legislation, identifying conflicting legislation, identifying common language in legislation which might shed light on the legislation’s origin, and on and on. There are numerous reasons why people want to be able to interpret and analyze legislation in ways that go well beyond merely reading it. In many cases, people want access to this information in a very timely fashion and on a scale which is just not possible or affordable with manual labor. That is why a common format is invaluable.

      To answer your question about BillTrack50, no it isn’t built on the scraping efforts of the Open States project. It relies on several data sources, not including Open States data, and does a lot of its own scraping. They integrated this data and then provide tools on top of the integrated data set.

    2. It was quite a lot of time and efforts, and chewed through a chunk of cash, it’s true (Grant’s shoestring and mine differ by a 0 I think 🙂 ). I do not, however, feel it was wasted.

      By standardizing the bill text we are able to:

      1) easily make all bills look the same on the screen, making the experience for the casual user far more friendly,
      2) diff the bills more easily allowing people to see quickly what has changed between any two versions of a bill
      3) apply standardized statistical analysis techniques to calculate key words, adding unique and valuable meta data to the bill
      4) allow for standardized output of the bills to html, xml (SLIM), word, pdf etc with only one converter per format instead of 50 for better sharing of the data outside our system
      5) add bill prognosis and other statistically based features that rely on standardized data as input (this work is in process)
      6) start working towards having the states adopt our standard to simplify life for those who follow

      Basically having one common format to start from allows for much easier further processing, both by the human brain and by machine, allowing significant additional value to be created.

      I hope that helps clarify why I was willing to spend my own personal funds, and efforts, on this project.

    3. What Grant said. A common format is very important for third-party tool development. I am a contributor to the Zotero reference manager project, with a particular interest in adapting the tool for legal research and writing. You can get an idea of what statutory support will look like from this rough screencast:

      The one-click download illustrated in the screencast depends on screen scraping. With a common format (with or without properly embedded metadata), one scraper will serve all jurisdictions. That’s a big win.

  2. Hi Grant,

    Interesting work you guys are doing, we’ll be sure to keep up with it.

    I do want to clarify one thing- BillTrack50 does use some Open States data, we aren’t their only source of course, but we’ve been in regular contact with them and they rely on our data dumps,etc. to as they put it “augment” their data. (which we’re happy to see of course, that’s why it is all out there)

    -James Turk
    Open States Lead

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s