To go Open Source or Not?

It is my dream to establish a legal informatics industry. Today, legal informatics is conducted either as an internal function or by consulting firms that specialize in long multi-year projects to build custom solutions. The few commercial products that exist are in the form of proprietory products or web services. Compared to many other forms of informatics, legal informatics has evolved very slowly. Part of the reason for this, of course, is the specialized nature of this field. This is particularly the case with legislative information where each legislature or parliament has long established traditions that are difficult to change.

As with every other informatics field, an industry will be established eventually. The costs of custom built software are simply economicially impractical in many cases, demanding a re-think about how solutions are created. For me, a key part of establishing that industry is the creation of standards. Whether there are official standards or de facto standards, standards will spur on the creation of an industry by creating a common model upon which to build. I have seen this happen in other industries that I have participated in and I don’t see why legal informatics should be any different. Yes, legal informatics is tardy in this regard, but that slowness should not discourage us from making it happen now.

So the question is quite simple. Can an open source solution form the basis of a de facto standard for legal informatics? And if so, what does that solution need to consist of? That is the question we have been wrestling with. There are two sides to this argument. While we might want to promote the establishment of an industry, at the same time we need to provide an economic incentive that will encourage businesses to participate. Could providing too much of an open source solution merely enable existing players to be more efficient, yet continue to work in relative isolation? It seems that a better outcome would be to promote the creation of interoperable products that can be mixed or matched to solve the multitude of needs in this field.

To this end, we’re trying a two pronged approach. First, we are fully supporting the establishment of official standards through bodies such as OASIS. Secondly, understanding that the official standard route is going to be a slow and perhaps arduous process, we’re pushing for the establishment of de facto standards. To this second goal, we have open-sourced our own SLIM model for legislation. It’s a very simple XML model based on 10 years experience building these types of models. While it isn’t a be-all and end-all solution, it is consistent with the current XML thinking and is quite easy to adopt. I have spent a fair amount of time wrestling with how to release it as an open source package recently. There are two questions I have:

  1. Which model? There are so many to choose from: Creative Commons, GNU, BSD, etc. Which model is permissive enough without discouraging commercial entities from adopting it.
  2. What aspects should be open source? I think it is quite clear that any and all XML information models should be open. That is in the spirit of XML, is consistent with the public domain nature of legislative information, and will allow the data to be accurately interpreted long after any particular software application of company has run its course. But would providing foundational software packages that are also open source further encourage the adoption of the model? And if that is the case, what foundation software would be beneficial?

At this point we have answered the first question and the first part of the second. We have chosen to release the SLIM XML schema as open source by using a Creative Commons Attribution-ShareAlike license. The rest of the second question remains open. What else we should provide with an open source license? Certainly it cannot be our full software suite. We are a commercial business and we need to make a living. But parts of our packages could be released to promote the adoption of SLIM as a de facto standard of sorts. What do you think?


9 thoughts on “To go Open Source or Not?

  1. I have strong views on this and (full disclosure) also a personal stake in the outcome. I agree that the creation of legislative data standards is essential to create better access to legal information.

    I also think that the standards should be open source, with strong guarantees about the future of that openness. We already have a number of situations in law where private publishers claim copyright on some part of the legal process. Bluebook claims the citation format, Westlaw and Lexis the page numbers of case reports, Adobe’s pdf is pervasive, etc. These and other examples have made law less open and hindered creation of a legal information industry.

    As you point out, it is very unlikely that governments or standards-bodies will agree on a useable legislative standard anytime soon. So I think the best approach is a privately created, open sourced standard. I happen to like the simplicity of SLIM and would like to see it spread far beyond California.


  2. grantcv1 says:

    I’ve been wondering how the existing proprietory legal information industry will react to the ascension of a standard for legal documents. Will they welcome it, will they ignore it, or will they attempt to prevent it? You can see examples where this scenario has played out in other industries. The question often seems to come down to whether a standard is an opportunity or a threat. It has always seemed to me that standards are inevitable. Some companies choose to embrace standards and rebuild their value-add atop a standards-based infrastructure. Other companies try to hold onto their existing business models for as long as they can, eventually falling victim to the march of progress.

    As Ari points out, we see hints that our field is inclined to try and prolong the status quo by hindering openness. I’ve personally worked for companies that have taken both tried to adapt or maintain the status quo. It will be interesting to see how this field will evolve over the next few years.


    • I think this turns on whether the delivery of legal information is perceived as a product or a service. In the US, in particular, there is a very strong sense in and around the market that these resources are a product; the leading vendors bundled a range of research support tools with the source text they supply, and it is the bundle, not the core text archive on its own, that attracts customers.

      A move toward standards-based distribution of the raw materials for legal research would both encourage and feed upon a tertiary market for research support products and services, which is being stifled under current arrangements. I don’t think there is any doubt that such a market would up the pace of innovation, and produce sufficient profits to sustain underlying standards. The question is whether the stakeholders in the law-as-product model can keep the cost and risk of entry high enough to prevent a more efficient market from taking hold.


  3. legalinformatics says:

    Mr. Vergottini: Thanks for this very interesting post. Would you be willing to say more about what you mean by the phrase “legal informatics industry”? There is a lot of evidence that such an industry has existed for several decades. It has a trade organization called ILTA . It serves law firms, coporate legal departments, government agencies, and citizens. It includes for-profit organizations, such as LexisNexis, Thompson Reuters, Bloomberg, Wolters Kluwer, Propylon, Justia, , Lexum , Fastcase, , POPVOX, , etc., It also includes non-profits like the legal information institutes , Leibniz Center for Law, , Indian Kanoon , Open Congress , Open Government , Richmond Sunlight, , OregonLaws, , etc. If organizations such as these don’t belong to a “legal informatics industry,” what industry do they belong to? And how does the legal informatics industry you envision differ from the existing industry?


    • grantcv1 says:

      Most certainly. First of all, I want to make a distinction between legal publishers and what I consider to be legal informatics. The legal publishers model has evolved from publishing paper to providing that paper online along with a proprietary suite of analysis tools. I am looking at the problem from a different viewpoint – the viewpoint of automation. Today’s automation market in legal informations is quite a bit dispersed. There are some case management tools that I know of, some voting systems, some bill tracking tools, etc., but they all operate in relative isolation. They serve specific point needs rather than working together to provide solutions. Standards for data interchange are irregular or non-existant.

      My background is in Electronic Design Automation. Thirty years ago, EDA tools were largely custom built solutions paid for by outsized IT budgets at the very largest of companies. The few vendors that existed operated in isolation requiring large integration budgets to adapt the products. After the past 10 years of working with legislatures and parliaments around the world, the situation is much the same as in my original field, just time shifted back 30 years. The largest legislatures have large budgets and either build all their tools in-house or they contract with consultants (like myself) to build custom solutions at great cost. The end result are very expensive solutions that take many years to develop and deploy, and then quickly face obsolescence as technology marches forward. With today’s budget realities, these increasingly expensive solutions are becoming impractical and unaffordable.

      Today in the EDA world there is a thriving industry of tool vendors that provide a wide range of relatively plug-and-play tools, based on open standards, that are affordable by any electronics company. I would like to see the same thing evolve in this field. While I realize that every legal entity has their own special ways of doing things that makes a turnkey solution impossible, I heard all these same reasons presented in the EDA industry way back when as well. But what was impossible then quickly became possible when the benefits of adapting overcame the cost of change.

      I believe that the same things must happen for legal informatics. There really should be a market for commercial tools that serve the legislative side of the legal informatics industry. These tools should be easy to adopt. They should have long-term upgrade paths so they won’t become obsolete as technology moves forward. They must be affordable.


  4. legalinformatics says:

    Sorry to post twice. Another comment: the role of open source in legal informatics has been discussed for many years in the legal information institutes community; see, e.g., Daniel Poulin et al. (2007). Free access to law and open source software, ; and the many papers listed here: . That discussion seems to address many aspects of the questions you pose.


  5. legalinformatics says:

    Sorry to post separately. Another important group of participants in today’s legal informatics industry is government agencies, such as the UK National Archives, which developed ; the US House of Representatives ; GPO ; and the Brazilian national Senate . Many of these agencies have created open standards, and have developed innovative technologies. Note how the URI model, developed by Jeni Tennison and John Sheridan, has influenced more recent innovative legal informatics systems, such as the MetaLex Document Server , , and The State Decoded, .


  6. I have a few observations from someone working for a legal publisher (disclaimer: this is only my personal view and opinion not my employer). When trying to answer whether economic incentives are necessary to encourage businesses to support the adoption of an open-source standards for legal data there are some things I would consider:

    (1) In general, legal publishers are evolving into workflow software companies and most no longer get the majority of their revenue from selling primary source information (laws, regs, cases, etc.). The views of a publisher of information from those of a software company are very different.

    (2) Many economic incentives already exist for businesses in this market to support standards. Legal publishers have conversion costs that they must pass on to customers solely because of the lack of a standards. Standards would shift some or all of the cost to taxpayers generally.

    (3) Even if businesses support the use of a standard by government entities because it would lower there own conversion and collection costs, businesses still have many incentives to retain there own proprietary standards. Internal standards would likely be needed to differentiate themselves from there competitors (e.g., additional metadata, features, or other enhancements). In addition, there are business reasons to prevent interoperability with competitor products. The assumption that supporting a standard also means adopting it internally or lead to greater interoperability between commercial products is, I think, flawed.

    (4) The detail level (e.g., a basic set of metadata) and scope (i.e., what legal data is covered) of a potential standard is extremely important in assessing whether a business would support it or feel threatened by it.


  7. Pingback: To go Open Source or Not? | Law Practice Strategy

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s