This is the briefing document I've provided for prospective lead legislative sponsors (see questions and comments below):
Along with a number of other open government advocates, I've launched a campaign to put a definition of "open data online" into California and San Francisco law. The issue is that often when documents and data are published online, they cannot be accessed or used in a meaningful fashion because they cannot be searched, indexed by Google, or combined in a meaningful way with other documents for analysis. I want to tackle this not by mandating that certain documents and data be published online, but simply by creating a reference standard so that when new mandates pass, or new documents are published online as a matter of course under existing law or regular business, they are in accessible formats.
This has the benefits of making things easier for people who use screen readers, for developer who want to use public data to build applications, for transparency advocates, and is simply good policy. Publishing data in formats that can't be searched, compared to other documents or reused in a meaningful way is as useless as keeping it tucked away in an obscured file cabinets. Publishing in accessible formats online is as simply as education employees in how to properly save and store documents for online publication using the same software they already have on their computers. In an ironic demonstration of the current problem, San Francisco's current open data law was published by the Board of Supervisors as an unsearchable PDF.
- Javier Muniz, CTO and co-founder, Granicus (based in SoMa and one of the greatest open gov tech company success stories in the U.S.)
- Steve Ressler, founder, GovLoop
- Rep. Jason Murphey, Chairman of the House Goverment Modernization Committee, Oklahoma
- Scott Primeau, OpenColorado
- Luke Frewell, founder and publisher, GovFresh
- and many more who can be viewed online - http://www.wiredtoshare.com/structured_open_data_campaign
Comments meant for official consideration should be directed to Alicia Lewis, alicia.lewis [at] sen.ca.gov
Open data in San Francisco, the state of California, and throughout much of the U.S. and the world remains hobbled by a lack of legal definition. San Francisco's own open data law, for example, is posted online by the Board of Supervisors as a non-searchable PDF. On December 10-11, at the winter CityCampSF Hackathon, Gov 2.0 advocates will publicly launch an advocacy campaign to institute an open data standard in San Francisco municipal and California state law. The primary goal of this advocacy will be to achieve a clear and reasonable definition of open data for all materials required by law to be published online.
Please join us in endorsing this advocacy campaign, and encourage your friends and legislators to sign on as well.
For another definition of open data online that we will consider, see the CityCamp model Open Government directive, which describes open data as being published online in an "open format that can be retrieved, downloaded, indexed, sorted, searched, and reused by commonly used Web search applications and commonly used software."
This legislation should also encompass the goals of increased transparency in responses to SF Sunshine Ordinance requests and California Public Records Act requests - documents released in an electronic format after implementation of this ordinance would have to follow its standards of accessibility.
Machine-readability: Data should be published in structured formats easily processed by machines/software.
Malamud is not a lawyer, but he's met plenty - allies and adversaries - in his time as the nation's "rogue archivist." If you want open government, Malamud's your go-to guy. Intense and lightly sweating, at 9 a.m. he was decorating tables with postcards highlighting one of Law.gov's foundational elements, a state-by-state national inventory of legal materials; after the event, he broke down the space himself. Soon he'll be in Chicago and DC before returning home to the Bay Area and wrapping up a project report. He exudes a revolutionary zeal and the steady confidence of a veteran of many open government and privacy skirmishes.
Wednesday's series of panelists balanced open data dreams with hard truths about privacy in the globalized infoweb. Bob Berring, a UC Berkeley law professor, summed up the core issue: Carl is working on a 10 year old's question: Government has laws. We have to obey those laws. Where are they?
Twitter in-house counsel Alexander Macgillivray talked about the difficulty for legal staff's at small companies to afford basic research because of high Westlaw and Lexis fees - fees that units of government pay as well for access to legal documents.
Malamud believes that the law is one area that the disintermediating promise of the Internet has barely touched, and he brought in friend O'Reilly for a lunchtime discussion with California Secretary of State Debra Bowen. "What are we missing as a society because we are denied access to what is essentially the open source of our democracy?" O'Reilly asked.
A recurring theme was the problem of authentication of legal materials online, and the implied authority of the two major vendors. Erika Wayne, a Stanford law librarian, asked if anyone had seen an "informational only" disclaimer - common on web legal materials - on a physical book.
Chris Hoofnagle, a privacy researcher and UC Berkeley law professor also had a stark warning about the need to protect individual privacy as advocates seek to put more government information online. He argued that believers in "Big Brother" powers for the government - "I'm serious" - will use the language of the transparency movement to accomplish their goal of a surveillance society.
Despite the serious mission and very real challenges, the promising theme of open data, Law 2.0 mashups and lowered barriers to legal knowledge was not lost. Said Macgillivray, imagine a statue with its own Twitter account, tweeting its revisions.