Announcing and BC launches open data portal

Today the BC government launched their new open data portal and open information sites. A little secret that I happened to be in-on along with some of the other influential #opendata folks in BC.

What you need to know:

1) Open Data is awesome. Open FOI information is huge.

2) The civil servants working on this one deserve a ton of credit for engaging with the development community. They definitely collected all the right requirements and had a solid understanding of what developers want out of open data in BC.

3) This is a _huge_ shift in government thinking and Christy Clark has to get the credit here. Progressive politics at its finest.

4) It will have consequences both good and bad, depending on what viewpoint you have. There's no longer any privacy when it comes to the public sector. Make more than 75k, your salary is public info. Get money from the government? I now know how much.

5) I plan to make sure that people can actually consume this information. So using some data that was available to me as part of the databc pilot, I've put together . Want to know what a civil servant's salary is? How much IBM got from the government last year? Who has the highest travel expenses in government? Search it out. A tip -- the site uses SQL 'like' searching, so %'s are wildcards. Search for a % symbol by itself to find the leaderboard.

Analysis after the break.

Ok, so you're still with me and not off looking up your co-workers salary. Good.

When I was asked to come up with feedback for a bc open data portal my suggestions were:

- Don't choose a license for everyone. Follow the award winning BC social media guidelines and empower the ministries and individual staff to use the standard license or pick their own if it is more appropriate.

The Good News:

They seem to have come up with a B- license. Its not all good, and its certainly not as good as the PDDL used by City of Surrey -- but I'll use it (unlike that awful Vancouver/Toronto license).

The Bad:

From reading the orders that accompanied the press release announcing the launch, one thing is clear, ministries are to use this new "Saurons Ring" license that will somehow magically be appropriate for all uses. Its ludacrous to think the same license could cover hansard and geobc at the same time and do justice to the legitimately restrictive traditions of hansard and the scientific commons of public information that could be geobc. What it will likely mean is either future revisions to the license, or that datasets simply wont get released. If hansard came out under this license for example, from the way I read it parody, satire and other mash-up style criticism would be valid uses -- I'd consider them so, but in my battles with hansard at a federal level I know very well that it would take an even bigger shift of thinking to allow hansard (and video clippings from proceedings) to end up being used, for example, in political campaigns or for bush-blair style mashup videos on youtube.

Definitely, an opportunity to leverage past success was lost here, but again, its a _good_ solid B- license.

Why do I give it a B-? Well, in addition to a few other minor items, the license they did come up with still puts a lot of liability on the part of developers. For example, if a dataset shows up in the BC catalog that they dont have the right to place there, it will likely be the developers problem to defend a copyright infringement suit. However, unlike the atrocious Vancouver/Toronto license, it doesn't expect you, dear developer, to also cover the province's costs. Basically you get to share the provinces misery for their mistake. My advice, make sure you know that the dataset you plan to use is original first-party work of the government before you use it, if not, figure out how it was made and who you need to contact to clear the rights first.

I've written many times before that its unreasonable to expect developers to do the government's rights clearing -- but I guess that one can't get past the legal wonks.

It's also an attribution license -- but its not firm attribution. I'm not actually sure if it is an attribution license, as if you use multiple datasets they dont expect you to nascar-style brand your site. Which is good I suppose, but the PDDL would be better. _Asking_ for attribution is the right balance IMO.

- Release high-value data sets.

Herb Lainchbury (founder of OpenDataBC and awesome opendata hacker) and I sat down and came up with a list that will be published shortly. It was a list of all the datasets we wanted from the BC government. I'm happy to say we got a couple -- financial disclosure datasets that are definitely high-value, and that form the basis for as well as some other fun stuff you'll discover in the catalog. However, data from BC Assessement, BC Laws, the freshwater fishing guide or the camping reservation system data weren't able to get released for launch time. I do however get the distinct impression that the wheels are now turning and that we can expect this data before too long. The next 12 months will really tell whether this is lip-service or a real change in government transparency and open policy. For those working on this file, we'll help by tracking the date a dataset was requested, when a dataset is released and developing some statistics about the quality and timeliness of releases. Some of this work already exists in the Open Data Usability Index, but more will come as we add the time vector to the statistics.

- Don't do portals.

Ok, I'll admit I lost this one. They made a portal. I think this is the wrong way to do open data. I want to see the power left to the ministries, and the awesome civil servants who want to help people find data.

This approach, especially combined with a standard license will add a pretty significant semophore to the release of open data. It creates central authorities that will slow the process down and add a formality to the process that is definitely not necessary. Worse, the BC government has a horrid track record when it comes to government standards. For example, what happens if they decide all financial data is to be released in some awful XML format as has been suggested by some opendata folk -- is it now a faux-pas to release a set of CSV's as Toronto just did? Are we better off with no data while we wait for some fancified XML format? It reeks of the all-govt-websites-must-use-frontpage standards of the past, and will certainly repeat the historic drag of classifying open-source software as a virus banned from official servers.

No. Portals are bad and authorities suck. They definitely heard us say empower the leaves of the tree, but we weren't listened to. Portals, I guess, are sexy to politicians as they're big projects that you can put your name on. Fair enough, but have we lost something in the process? Are they trying to embrace agile open-source while still trying to do waterfall? Has anyone ever achieved such control of open?

- Make it "default open"

The BC government knows what data is private -- they have an assessment tool called the PIA (privacy impact assessment)... what Herb suggested and I agree is that BC become open-by-default. That is, if its not PIA, or otherwise classified, that -anyone- can request the dataset and that their request should be filled within a service-standard timeframe -- like FOI requests are (in theory).

The jury is still out on this one I believe, but we'll see how it progresses in the weeks and months to come. Certainly the open info site is a massive, and unexpected step towards true government transparency and is a stepping stone on the path to truly open government.


In the end, I give the BC government's opendata efforts a B-, because to really be open means releasing data, licensing it permissively, and empowering people outside of central authorities to interact directly with an increasingly talented citizenry. Given time, I'm expecting that BC will gain an A+ opendata rating as there really does seem to be some nice, dedicated and hardworking folk working to make this vision a reality.

Until then, is fun to play with, as is

Just my $0.02 as always.