Back in November, I proposed that California adopt an open data standard for official documents. Scores of you signed on to support such legislation, and Sen. Leland Yee, D-SF, shortly thereafter proposed a step in the right direction - SB 1002, which makes machine readability a criteria for open records in the California. On Thursday, the bill passed out of the Senate, 34-0.
This bill nearly went down and it is to the credit of many of you in Adriel Nation that it didn't. Just last week it was basically dead in the Appropriations Committee, before a flurry of calls, emails and faxes to two key senators on the committee helped resurrect the bill.
Here is Leland's statement on the passage of the legislation out of the Senate.
And here is an article on the bill from California Forward.
I want to thank all of you for your efforts promoting this important principle - that records posted online should be in the most accessible possible formats. There is a long way to go, but this is a great step in the right direction for efficient, transparent and technologically adaptive government. Special thanks to Javier Muniz of Granicus, whose support for open government and thoughts on how to advance it helped spur my efforts on this issue, and to David Cruise, whose tireless advocacy helped to get SB 1002 this far.
Get started now with a free NationBuilder trial account.
When I ran for office in 2009, one of the most significant hurdles I faced was getting voter data for my campaign. It took forever, and when I did get it, the data was in a database format that might as well have been in ancient Greek.
This morning, as part of the nationwide SOPA/PIPA protests, I called my Congressman Jerry McNerney, a Democrat for whom Pro Publica had no public statements on record regarding the bills. One of his constituent services representatives confirmed my address and took my statement of opposition - SOPA would hurt Internet businesses and the free exchange of ideas. I asked for a public statement against SOPA from the Congressman. Four hours later I got this letter by email, which has also been reported in the local press:
"Thank you for contacting me to share your opposition to H.R. 3261, the Stop Online Piracy Act (SOPA),and other similar proposals. I appreciate learning your views, and I value your input.
"I oppose H.R. 3261 as the bill stands as well as similar legislation introduced in the Senate. The intent of the bill is to help law enforcement agencies pursue websites that violate intellectual property laws and sell counterfeit goods, which is a serious and costly problem. However, H.R. 3261 is too broadly written, and I am deeply concerned that the bill infringes on individuals' civil liberties and innovation by small companies. Freedom of speech and the open exchange of information on the internet are directly impeded by this legislation.
"Our economic power has always been based on innovation and the free flow of ideas. The thinkers and creators in our country have always showed remarkable ingenuity, and stopping the stream of information on the internet would put the United States at an international disadvantage. We must remain committed to American advancement that comes from the freedoms we hold most dear. I will oppose legislation that undermines our nation's freedoms and finest traditions."
Call, meet, get to know your reps. It matters.
Here's what Liberal Art has to say about NationBuilder vs. Salsa+Drupal: "NationBuilder's tool suite is competitive with competing platforms, but its tools are fully integrated into the CMS. This means less setup and easier management of your messaging and action campaigns. What's more, social sharing functions are baked into every page, tool, and action taken by your supporters."
NationBuilder is a nonpartisan community organizing system with wide adoption in the U.S., UK and Canada.Â It brings together social media, customizable websites with content and community action pages, and email and text blasting.
NationBuilder is an alternative for Blue State Digital for campaigns and non-profits that want to run their own actions and emails without need for costly integrations and consulting contracts.
"The Cost of Freedom" - Another Powerful Democracy Project Brought to You by Folks from the Internet
What do folks do at civic hackathons? Most of us are trying to make democracy better by giving of the skills we have. Faye Anderson just brought me this great example from a Random Hacks of Kindness event earlier this month (be sure to check out the call to action and then get in touch with Faye if you can help):
"The Cost of Freedom Project is a citizen-led initiative that is developing a location-based web app to provide voters with information on how to apply for a voter ID. A prototype for the Cost of Freedom App was developed at the Random Hacks of Kindness hackathon at Drexel University.
In 2008, only two states required voters to show a government-issued photo ID in order to vote. Since then, six states have changed their voting laws to require a photo ID. Civil rights groups are challenging photo ID laws at the Justice Department and in the courts. But with the election less than 11 months away, voters need help right now.
There are millions of voters without an official photo ID who cannot wait until October to tune into the election. They must act now to ensure they have the documents they have to produce to establish their identity.
The Cost of Freedom Project needs help gathering data for the eight states that require an official photo ID – Georgia, Indiana, Kansas, Mississippi, South Carolina, Tennessee, Texas and Wisconsin. (By February, Michigan and Pennsylvania may be added to the list.)
The data includes where a voter in, say, Memphis who was born in Memphis, would go to obtain a copy of his or her birth certificate and how to get there using public transportation.
We are a mobile society so a voter now living in Memphis may have been born in New Jersey, where it could take as long as ten weeks to get a certified copy of a birth certificate. So we will need data on how to obtain a birth certificate for all 50 states and the District of Columbia.
We need web designers. We need bloggers to help spread the message: No photo, no vote. In short, we need every citizen to get informed about photo ID requirements and inform their friends, family and neighbors.
The data gathered by the Cost of Freedom Project will help tell the underreported story of the high cost of a free voter ID."
I was witness to an interesting cultural moment today that exemplifies how fast technology is moving. Above, Phil Ma, owner of Mavelous SF, checks out his 23andMe results, fresh from the day's mail. Getting online access to your genetic code now costs just a couple hundred bucks for a year.
Over the past several years, cities like San Francisco have come a long way in putting data online. However, departments with budgets in the tens of millions of dollars - including the very agency tasked with policing government ethics - still have miles to go. I'm organizing an open data standards campaign in San Francisco and California because putting data online in formats that cannot be independently analyzed and resused by the public is simply poor government.
For the upcoming CityCampSF Hackathon, an event where volunteers will bring meaning to and extend the usefulness of public data, the muckracking newspaper CitiReport is sponsoring a challenge to cross-reference all kind of open data to make sense of public ethics in San Francisco. Sadly, of four departments who could help bring transparency to public ethics, only one is using open standards on its key data.
By way of background, it's been more than a year since SF passed a groundbreaking law urging departments to post their data on DataSF.org.
SF Planning: The Planning Department has several data sets on DataSF, but the project information on key developments active right now does not seem to be there. Instead, we get this nicely designed "Complete List of Projects" with no strutured data (such as a simple CSV file with headers that include the project description, location, lead planner and developers) in sight.
SF Ethics: The Ethics Commission has the legal duty of collecting huge amounts of campaign finance and good government data. But their datasets on OpenSF are horribly out of date, and their online list of lobbyists would take a master coder or many, many man hours to parse into an open format for analysis (again, simple CSV files with headers would do - name of lobbyist, who they work for, how much they got paid, who they contacted, when, what project they were trying to influence). This lack of functional transparency is glossed over with a great data display, but none of the underlying data is easily available on the site. This means you get the results that the display tells you are important, while independent analysis is nearly impossible.
SF Controller: The Controller's Office is responsible for keeping track of city salaries and city contracts. They don't have a fancy data display (which I bet saved them a lot of money), but they do allow you to search their contractor database and download data in structured open formats. Thank you!
SF Board of Supervisors: The Clerk of the Board publishes huge volumes of minutes of board agendas and votes. However, they are PDFs with no structure. Many of the attachments for Board items end up as PDF-formatted scans of documents without optical character recognition, rendering them unsearchable by people viewing them on the web, and keeping them out of open web search results. This lack of functional transparency makes it very difficult to know who's voting on which planning projects for example.
In summary, if we want to make connnections between who is lobbying for a contract, which developers are giving money to politicians, and who's voting on what, it is very, very difficult. Lack of functional transparency means that if you want to evaulate your government in San Francisco, you have to know almost exactly what piece of straw in a data haystack you're looking for before you even start.
Consider that SF has an annual budget of more than $6 billion. We can do better. We will do better.
That's why I'm working on a campaign to bring structured open formats to SF and California law.
And please, if you can help research, write open records requests, code or have any other skills you'd like to bring to bear on behalf of tech-enabled open government, join us this weekend at the CityCampSF Hackathon. It's time to make transparency work for the people.