There's been a big shift in how people use the web that caught up with Healthcare.gov and sister sites yesterday. You can build the most beautiful and "scalable" website for web visits, make it open source, put the code up on GitHub, talk about how innovative it is, then watch it crumble under the server strain of people trying to actually do something through your site.
Healthcare.gov's real challenge wasn't to build an alternative to a commercial CMS (content management system), it was to build an application that can handle event-oriented human behavior - for that you need the best systems engineering, not "10,000 authenticated users through GitHub" for your content delivery, as one of the Healthcare.gov contractors highlighted in this Atlantic profile of the project by Alex Howard.
Before the application process bogged down yesterday, Healthcare.gov got lots of nice gov tech insider buzz for its open source nature. But the project still had contractors on board, and based on how the service behaved on opening day of the Affordable Care Act, it could have stood a lot more testing of what people actually wanted to do with it. Kind of like Mitt Romney's Orca system on election day last November.
The Healthcare.gov site loaded fine, but trying to apply through it was kinda like buying first-time Comic-Con badges online.
Open source has changed the technology landscape for the better, underpinning many of our favorite startups. However, simply invoking it like a protection spell is no replacement for the architectural skill and planning required to pull off the systems needed for a successful Healthcare.gov launch. Health and Human Services, which managed the project, needed a little more "Puppet vs. Salt" and a little less "open" in its vernacular.
Adapting to a web where people are participants, not viewers, is the lesson we're all learning. Web infrastructure needs to support people, not publishing.
The reaction to failures of Healthcare.gov under heavy load won't work if the discussion is about how other services fail - it has to be about building infrastructure that's designed for peak interactivity and not for views.
Choice quotes from the Atlantic profile:
Bryan Sivak, CTO at Health and Human Services: "Instead of [running] farms of application servers to handle massive load, you're basically slimming down to two. ... The way it's being built matters."
Dave Cole from HSS contractor Development Seed: "You're just talking about content. There just needs to be one server. We're going to have two, with one for backup. That's a deduction of 30 servers."
Maybe there was a lot more infrastructure work going on behind the scenes, but the project leads' obsessive focus on the content framework is telling.
Healthcare.gov's scaling challenge was never about delivering content like a really popular website, it was the peak activity challenge that Twitter faces on a regular basis. Taking interaction-based scaling challenges seriously is why Twitter is stable now and wasn't in 2009 - those are the issues HHS should have been talking about.
Few updates after a bit of Twitter fun on these issues today:
Not faulting Alex's reporting in any way here - I believe if the HHS team was really focused on the infrastructure for supporting a signup rush at the time of the Atlantic article, that dedication would have shown up in the story. The omission of that kind of discussion (read the article - the project team seems to have an almost flippant approach to back-end server architecture). I also googled around looking for commentary on that front from earlier in the life of the project.
I didn't do a detailed investigation, this is an opinion blog piece not investigative journalism. As I said above, it's quite possible there was more going on - but the fact the site had so much persistent trouble as an actual application (while it functioned fine as what we call in the biz a "brochure site") means whatever was done fell dangerously short.
Finally, if an important initiative like Healthcare.gov is going to get 2.8 million views in a day, I want everyone who wants to apply through that site to do so smoothly. My ding on "open government buzzwords" is that it's really easy to do "innovative" things with government technology and get headlines, without actually delivering for constituents.
Another update from Twitter conversation:
Alex speculates the devs and designers who built the content framework aren't to blame here.
Fair enough. I think it's fairly clear from the above that I blamed HSS and a culture of thinking that web properties are publishing applications and not designing them for interaction. It's really time to stop talking about a "front-end" and a "back-end" for any kind of website. If it doesn't scale for interaction, it doesn't scale. Twitter's infrastructure challenge isn't displaying millions of tweets, it's keeping all of them threaded in real-time.
Open source content frameworks are nice (hey, Twitter released Bootstrap!), but HHS separated that issue from the kind of services needed to effectively scale the application process. It's like building a really shiny muscle car and then giving it a weak 2-liter engine. Fully integrated applications with content delivery and scalable interaction design are really, really hard. And that's where buzzwords fall short.
Sept. 7 update:
On Saturday, I wrote about these issues on GovFresh, "The openwashing of Healthcare.gov" and cited a Reuters article that laid the project on CGI Inc., a giant federal contractor.
Today, the Wall Street Journal quoted an HHS spokeswoman and IT experts regarding flaws in the system. The article mentions CGI and also says Experian had a contract around identity verification. Based on the analyses I've read, it seems like there could be timeouts or critical delays between security question submittal and verification, which would indicate architecture issues again, not an Experian issue per se.
What happened at CityCampSF Hackathon 2011 on Saturday and Sunday? Lots of great discussion about technology and open government, folks meeting for the first time over pizza, Red Bull and Peanut M&Ms, and some civic hacking on online lobbyists filings, timber harvest plans and text notifications for public meeting agenda keyword alerts. CityCampSF participants:
- Discovered some very interesting tidbits in the SF Ethics database, including a company that pays the City's biggest lobbying firm $3,800 a month, while the firm reports zero contacts with officials since Jan. 2010;
- Made a polite and official request for a complete raw data set of the entire lobbyist database, since the online version is a minor disaster (biggest recipient of lobbyist political contributions? "Not Applicable, Not Applicable"; third biggest? "NA, NA"). See my screenshots for more odd results in the SF Ethics lobbyists data display. Image here is from Ted Louie, looking at how an app might connect open data from various city agencies responsible for storing development, contracting, lobbying, political donation and legislative data;
- Built a demo app for public hearing agenda item alerts by text message. Interested in local liquor licenses or development projects in your neighborhood? With a little more municipal legislative open government (the tech is already in place), you can get a text every time those items are up for a vote;
- Discovered that forest timber harvest plans include geospatial-data rich MXD format files, then are turned into scanned and unsearchable PDFs before the state publishes them on an obscure FTP server;
- Built an interactive map of pending clear cuts and forest thinning in Northern California with Google Maps and research from the THP Tracking Center. See the great work by Granicus CTO Javier Muniz at forestsforever.heroku.com
This is the briefing document I've provided for prospective lead legislative sponsors (see questions and comments below):
Along with a number of other open government advocates, I've launched a campaign to put a definition of "open data online" into California and San Francisco law. The issue is that often when documents and data are published online, they cannot be accessed or used in a meaningful fashion because they cannot be searched, indexed by Google, or combined in a meaningful way with other documents for analysis. I want to tackle this not by mandating that certain documents and data be published online, but simply by creating a reference standard so that when new mandates pass, or new documents are published online as a matter of course under existing law or regular business, they are in accessible formats.
This has the benefits of making things easier for people who use screen readers, for developer who want to use public data to build applications, for transparency advocates, and is simply good policy. Publishing data in formats that can't be searched, compared to other documents or reused in a meaningful way is as useless as keeping it tucked away in an obscured file cabinets. Publishing in accessible formats online is as simply as education employees in how to properly save and store documents for online publication using the same software they already have on their computers. In an ironic demonstration of the current problem, San Francisco's current open data law was published by the Board of Supervisors as an unsearchable PDF.
- Javier Muniz, CTO and co-founder, Granicus (based in SoMa and one of the greatest open gov tech company success stories in the U.S.)
- Steve Ressler, founder, GovLoop
- Rep. Jason Murphey, Chairman of the House Goverment Modernization Committee, Oklahoma
- Scott Primeau, OpenColorado
- Luke Frewell, founder and publisher, GovFresh
- and many more who can be viewed online - http://www.wiredtoshare.com/structured_open_data_campaign
Comments meant for official consideration should be directed to Alicia Lewis, alicia.lewis [at] sen.ca.gov
Open data in San Francisco, the state of California, and throughout much of the U.S. and the world remains hobbled by a lack of legal definition. San Francisco's own open data law, for example, is posted online by the Board of Supervisors as a non-searchable PDF. On December 10-11, at the winter CityCampSF Hackathon, Gov 2.0 advocates will publicly launch an advocacy campaign to institute an open data standard in San Francisco municipal and California state law. The primary goal of this advocacy will be to achieve a clear and reasonable definition of open data for all materials required by law to be published online.
Please join us in endorsing this advocacy campaign, and encourage your friends and legislators to sign on as well.
For another definition of open data online that we will consider, see the CityCamp model Open Government directive, which describes open data as being published online in an "open format that can be retrieved, downloaded, indexed, sorted, searched, and reused by commonly used Web search applications and commonly used software."
This legislation should also encompass the goals of increased transparency in responses to SF Sunshine Ordinance requests and California Public Records Act requests - documents released in an electronic format after implementation of this ordinance would have to follow its standards of accessibility.
Machine-readability: Data should be published in structured formats easily processed by machines/software.
One of the great local government examples we've looked at is the Flickr activity of the Washington State Department of Transportation, managed by Jeremy Bertrand. Today, we opened an official Flickr account for the San Francisco City Attorney's Office, where we hope to not only highlight our great city, but also feature photos that illustrate the work of our office, from the hard-fought battle for marriage equality, to City Attorney Dennis Herrera's anti-gang initiatives.If you are a San Francisco photographer, or just interested in connecting with our content and San Francisco favorites, please add us on Flickr. We also welcome suggestions on how you think we can best use this channel.
Stephen Collins: Government 2.0...it can be a reality
David Forbes: I Do Not Believe in Collapse
Kristy Fifelski:ÃÂ 'Govsourcing' the Reno.gov homepage
Sarah Estes Cohen: Hybrid 2.0? How to leverage social media for emergency management and response
Dan Slee: Social Photo - 11 groovy ways local government can use Flickr
Andrea DiMaio: Human Resources and not Communication are the Front Line of Government 2.0
We have advocated for humanizing government, and for using new tools to bring more citizens into the deliberative process and to help shape the future of both our democracy and the bureaucracy. One of the main tools for the Gov 2.0 movement has been social media, as activists and line workers join technologists and political reformers in calling for more open communication between officials and agencies and the public they represent and serve.
Last week, Government 2.0 Ã¢ÂÂ a term first used by Bill Eggers in his 2005 e-gov-focused book of the same name, and that has become almost synonymous with Web 2.0 as developers have turned on to the promise of government-brokered data troves and universal open standards Ã¢ÂÂ won a significant victory. Twitter, the popular social media messaging service that has serves as a platform for thousands of startups using its architecture and user base, announced that it is hiring for its first field office, focused on the government sector.
Twitter Goes to DC
Twitter's job posting and further remarks by corporate spokesman Sean Garrett explain the DC-based position as the first step towards a public affairs unit, with support for innovative and engaging uses of Twitter in politics and policymaking. A new blog by Garrett and his team has since March been highlighting interesting government uses of the platform, from San Francisco's integration of Twitter and 311 non-emergency service requests, to construction updates and border crossing wait times by tweet, to the British Prime Minister's communications usage.
Twitter, thanks to millions of active and aggressive content-sharers and innovators around the world, has transformative powers. Conan O'Brien took to the service to recreate himself after losing his show, creating numerous accounts, rallying his fan base and using the free and frenetic publicity it to launch a comedy tour. Legendary film critic Roger Ebert, after panning Twitter as trite, has become one of its staunchest advocates, using it to deliver and amplify commentary on everything from film to politics to sport and humanism. Newark Mayor Corey Booker has used it to spread a hands-on philosophy of hope far beyond his New Jersey township.
Twitter Grows Due to User Innovations
Twitter's growth and popular features have often evolved from the minds and whims of its user base, from the intensely popular "retweet" convention for repeating and affirming others' messages, to the hashtag form of semantic tagging in its short messages, to Follow Friday, the day that tweeps around the world recognize friends and favorites.
Government 2.0 Ã¢ÂÂ which first hit Twitter's mainstream of "trending topics" during a March 16, 2009, pilot broadcast of the Gov 2.0 Radio podcast including govies, contractors and consultants calling in from South by Southwest and their DC-area homes Ã¢ÂÂ is now set to join the legacy of user-driven Twitter conventions. The first Twitter office outside of San Francisco will help connect politicians with their constituents and agencies with the public. It will help serve an engaged and innovative Government 2.0 movement, while that movement continues to shape and grow Twitter's utility.
Government 2.0 and the use of social media for politics and public service are still in their infancy, but it's safe to say that Twitter's new focus on this arena is a milestone of which we can be proud.
Clever Twitter Accounts ÃÂ¢ÃÂÃÂ Government
How Conan OÃÂ¢ÃÂÃÂBrien Learned to Stop Worrying and Love Twitter
Roger Ebert ÃÂ¢ÃÂÃÂ Tweet! Tweet! Tweet!
So, today I was doing some pruning of the folks I follow on Twitter. This can be tedious work, but itÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂs important to my networking efforts. I try to follow back most accounts that follow me, as long as they look like they have live people or organizations behind them. Plenty slip through the cracks, though, and I begin find my feed a bit overrun with people using FriendFeed, Facebook and a slew of other services to pipe content to Twitter with zero interaction there. Unless itÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂs content highly useful to me ÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ like feeds from a few blogs and news agencies ÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ I generally unfollow those sorts of accounts.
Cutting loose spammy and dead accounts
During this exercise, I also notice two kinds of accounts from people who are obviously trying to use Twitter as a networking tool, but are going astray. There are the accounts obviously auto-following people (look for 1-to-1 follower-following ratios) and having little luck at engagement, and then there are those whoÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂve simply stopped tweeting.
Reviewing these accounts, itÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂs often clear that they had purpose in getting started, whether to tweet at a conference, to promote their business, or simple to build that network before itÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂs needed. Many of the folks who stop tweeting donÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂt say why, but enough do that IÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂm guessing itÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂs because they simple arenÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂt getting the kind of engagement they were promised or expecting. Sometimes theyÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂre discouraged because theyÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂve got hundreds of Twitter followers but only a few of those click on the links they share.
My advice for networking on Twitter ÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ and I believe the informational networking there is tremendously valuable ÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ is to be strategic in how you build out your community. For example, if youÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂre trying to market SEO services, and sign up for a service that auto-follows anyone who tweets the words ÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂsocial media,ÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ youÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂve totally missed any sort of practical audience. Sure, you can all retweet each othersÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ links and tidbits of wisdom, and yes, that may increase your personal SEO (which is one of the few good reasons to crank out content on Twitter without and personal engagement). But itÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂs not likely to get you customers. What if instead you identified local businesses and Chamber of Commerce members engaging on Twitter who might be interested in your services? Start interacting with them; build a relationship that will lead to real business.
If youÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂre the conferencegoer, figure out what Twitter hashtag people are using to tweet about the event, and make connections before, during and after by merging your Twitter and offline networking. Chances are, Twitter connections established there will continue due to shared interest or profession.
Twitter has been an extremely valuable tool for the Government 2.0 movement. Last week, Gov 2.0 consultant Maxine Teller commented on why she thinks itÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂs important that Twitter is hiring a government liaison, explaining how Mark Drapeau convinced her to start using Twitter actively in 2008 after sheÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂd stopped:
The whole reason that you and I were jazzed about Twitter back then was because it was ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ and still is ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ a great way for us to find and connect with like-minded folks who believe ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ and are using ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ emerging tools and technologies enable us to more efficiently and effectively achieve our government missions.
To repeat the mantra that we've all chanted in our Gov 2.0 conference and event presentations umpteen times, Gov 2.0 (despite its software release naming convention) is not about the tools and technologies; it's about the collaborative interactions, innovative thinking, and revolutionary approaches that these tools and technologies catalyze and enable.
In late 2009, Gartner consultant Andrea DiMaio published a research noted defining Government 2.0 as ÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂthe use of IT to socialize and commoditize government services, processes and data.ÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ His definition is one of the most solid and comprehensive IÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂve seen, and it encapsulates many of the reasons social technologies are important to other businesses sectors as well:
The socialization of information has multiple facets (government to citizens, citizens to government and government to government) and the boundaries between these facets are increasingly blurred. The next step will be the socialization of services and processes by engaging individuals and communities to perform part of existing government processes or transform them by leveraging external data and applications.
Commoditization ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ which has already started with consolidation and shared services to reduce the diversity of infrastructure and horizontal application ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ¢ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ will gradually move toward services and business processes.
Government 2.0 has seven main characteristics:
* It is citizen-driven.
* It is employee-centric.
* It keeps evolving.
* It is transformational.
* It requires a blend of planning and nurturing.
* It needs Pattern-Based Strategy capabilities.
* It calls for a new management style.
Food for thought.
Twitter Strategy for Agencies and Causes
Why and How: Local Twitter Lists
Government 2.0: A Gartner Definition
Drapeau: Government 2.0 Movement Seemingly Passes by Twitter, Inc.
There's been a lot of reading between the lines of Twitter's job posting for a DC-based government liaison (and even one instance of actual follow-up reporting). One post really caught my attention - because I disagree with it so vehemently.
My friend Alan W. Silberberg, a Gov 2.0 innovator and founding organizer of Gov 2.0 LA, argued that, "Twitter needs a government relations expect who is also a social media expert. Not the other way around." His five-point post went on to urge a traditional (if exceptionally well-qualified in the type) Beltway insider for the new post, which Twitter envisions heading up an emerging public affairs shop.
Wrote Silberbeg (who said he is not applying):
Because of the Giants amongst us like Microsoft, Google, Facebook - Twitter's entry into the Government space has to be taken carefully. The Giants have armies of lobbyists, lawyers, pr firms, etc. The Twitter person needs to be able to navigate these waters with firm decision making. Time spent getting up to speed will only hurt the company, and its investors like Union Square's Fred Wilson. This goes back to my first point. Twitter needs to hire someone known in the Gov 2.0 space - but also known in DC. IN Government. No offense to my peers and friends applying for this job - but it clearly says that they are looking for a DC area person who already has Government experience. That really means connections, access and understanding of the policies and ethics surrounding these changing times.
I'm not going to do a point-by-point, because Alan's arguments are sound from the perspective of traditional government relations.
But our times urgently call for the non-traditional. I often say that my social media-fueled campaign for Congress last year was a few years too early.
Hiring anyone but a visionary for Twitter's first government-facing employee would be be a few years too late.
As a friend in government recently said to me, "We have the next 10 years to shape the next 30." Our government is a massive public engagement fail, and aping its nature of privilege and insider connections would be a disaster.
Another friend, Shaun Dakin, anti-robocall activist and dot-com era veteran, is applying for the post and today gave his reaction to Silberberg's post and the job description's inclusion of "entrepreneurial" qualities.
People used to working in Gov't and big companies (I was there, big time, with Fannie Mae and FedEx) are used to WAITING for permission to do things. They do research. They go to meetings. They brainstorm. They rarely DO anything.
Entrepreneurs, on the other hand, don't really ask anyone for permission. They just do.
... Critically, I think, they know how to get things done with few resources.
Perhaps Twitter thinks that whomever is in this role (he or she) will be really "starting up" not just a new office but also a new line of business for Twitter.
So, my recommendation to Twitter would be to look hard at if the person has had to DO.
In the past couple years, hundreds of driven and innovative political and government media and engagement strategists have qualified themselves for this position. I hope Twitter picks from that number.
Silberbeg: Gov 2.0 and Twitter Finally Tweet-up!