Government Abandonware
Governments order software for their allegedly very specific needs. And the EU government (The European Commission) orders “visionary” software that will supposedly one day be used by many member states that will be able to communicate with each other.
Unfortunately, a lot of that software is abandonware. It gets built, the large public doesn’t see the results, and it dies. A while a go I listed a couple of abandoned projects of the EC. Some of them, I guess, are really research projects and their results may be important for further development. But I couldn’t find the results. All there is are expired domains and possibly lengthy documents. But 100-page documents about software are not software. I haven’t seen any piece of code whatsoever. The only repo that I found is that of OntoGov, but all there is there are zip files with “bin” in their name.
Even though unused, the software maybe isn’t lost (although the agency behind some of the projects doesn’t exist anymore), and may be utilized by new projects or even businesses. But it’s just not accessible to the general public. It’s probably hidden in a desk, in a dark basement, with a leopard warning on the door (as in the Hithchiker’s Guide to the Galaxy).
But the problem is not only on EU level. It is on national level as well. Since June I’m an adviser to the deputy prime minister of Bulgaria (which I’ll write about another time), and from the inside it’s apparent that software has been built over the years and then forgotten. I’m pretty sure this is true in most countries as well.
Why this is bad? Not only because of the suboptimal public spending, but also because a lot of effort sinks into a bottomless pit. Software, even though not useful at the given moment, may be used as a base or a building block for future projects that will be really used. Now everything has to start from scratch.
A solution I’ve been advocating for for a while is open source. And, getting back to the EU, there’s an “Open Source observatory” to which I’m subscribed, and I get news of all sorts of state-level and EU-level open-source initiatives. And the other day I saw one pretty good example of the state of affairs.
I read that an open-source tool for collaboratively editing laws is available. Which sounded great, because in the past weeks I’ve been participating in law-making and the process of sending Word document with “Track changes” via email among a dozen people is not the most optimal editing process a developer like me can imagine. So I eagerly opened the link, and…
It got to the front page of /r/opensource, and eventually, after my request to the admins, the page is accessible. And guess what? There is no link to the code or to the product, and the contact person in the link hasn’t answered my email. Yes, it’s holiday season, but that’s not the point. The point is this is not real open source. All dead software from my tweet above is gone and useless, even though, maybe, you can contact someone somewhere to get it. Update: a while later the source finally appeared, but as a zip file, rather than an open-source repo.
Not only that, but currently important EU projects like the ones under e-SENS (an umbrella for several cross-border access projects, e.g. e-procurment, e-jusice, e-authentication (Stork)) are practically closed to the general public and even governments – it took me 10 days to get get the reference implementation (which, according to the implementors, will be made public once the instructions for its use are ready, in a few months)
I hope I’m wrong, but my prediction is that these new projects will follow the fate of the countless other abandonware. Unless we change something. The solution is not just to write “open source” in the title. It should be really open source.
The good news, for Bulgaria at least, is that all new publicly funded projects will have to be open source from day one. Thus not only the work is less likely to die, but also the process will be transparent. What software gets built, with what level of quality and for how much money.
We are currently in the process of selecting the best solution to host all the repositories – either on-premise or SaaS, and we are also including requirements for supporting this solution in proposed amendments to the e-governance law.
I hope that this way we’ll get way more reusable code and way less abandonware. And I hope to extend this policy to an EU level – not just claiming it’s open in the title, but being really open. Not for the sake of being open, but for the sake of higher quality. It won’t necessarily prevent abandonware altogether (because useless software gets created all the time, and useful software gets forgotten), but it will reduce the wasted effort.
Governments order software for their allegedly very specific needs. And the EU government (The European Commission) orders “visionary” software that will supposedly one day be used by many member states that will be able to communicate with each other.
Unfortunately, a lot of that software is abandonware. It gets built, the large public doesn’t see the results, and it dies. A while a go I listed a couple of abandoned projects of the EC. Some of them, I guess, are really research projects and their results may be important for further development. But I couldn’t find the results. All there is are expired domains and possibly lengthy documents. But 100-page documents about software are not software. I haven’t seen any piece of code whatsoever. The only repo that I found is that of OntoGov, but all there is there are zip files with “bin” in their name.
Even though unused, the software maybe isn’t lost (although the agency behind some of the projects doesn’t exist anymore), and may be utilized by new projects or even businesses. But it’s just not accessible to the general public. It’s probably hidden in a desk, in a dark basement, with a leopard warning on the door (as in the Hithchiker’s Guide to the Galaxy).
But the problem is not only on EU level. It is on national level as well. Since June I’m an adviser to the deputy prime minister of Bulgaria (which I’ll write about another time), and from the inside it’s apparent that software has been built over the years and then forgotten. I’m pretty sure this is true in most countries as well.
Why this is bad? Not only because of the suboptimal public spending, but also because a lot of effort sinks into a bottomless pit. Software, even though not useful at the given moment, may be used as a base or a building block for future projects that will be really used. Now everything has to start from scratch.
A solution I’ve been advocating for for a while is open source. And, getting back to the EU, there’s an “Open Source observatory” to which I’m subscribed, and I get news of all sorts of state-level and EU-level open-source initiatives. And the other day I saw one pretty good example of the state of affairs.
I read that an open-source tool for collaboratively editing laws is available. Which sounded great, because in the past weeks I’ve been participating in law-making and the process of sending Word document with “Track changes” via email among a dozen people is not the most optimal editing process a developer like me can imagine. So I eagerly opened the link, and…
It got to the front page of /r/opensource, and eventually, after my request to the admins, the page is accessible. And guess what? There is no link to the code or to the product, and the contact person in the link hasn’t answered my email. Yes, it’s holiday season, but that’s not the point. The point is this is not real open source. All dead software from my tweet above is gone and useless, even though, maybe, you can contact someone somewhere to get it. Update: a while later the source finally appeared, but as a zip file, rather than an open-source repo.
Not only that, but currently important EU projects like the ones under e-SENS (an umbrella for several cross-border access projects, e.g. e-procurment, e-jusice, e-authentication (Stork)) are practically closed to the general public and even governments – it took me 10 days to get get the reference implementation (which, according to the implementors, will be made public once the instructions for its use are ready, in a few months)
I hope I’m wrong, but my prediction is that these new projects will follow the fate of the countless other abandonware. Unless we change something. The solution is not just to write “open source” in the title. It should be really open source.
The good news, for Bulgaria at least, is that all new publicly funded projects will have to be open source from day one. Thus not only the work is less likely to die, but also the process will be transparent. What software gets built, with what level of quality and for how much money.
We are currently in the process of selecting the best solution to host all the repositories – either on-premise or SaaS, and we are also including requirements for supporting this solution in proposed amendments to the e-governance law.
I hope that this way we’ll get way more reusable code and way less abandonware. And I hope to extend this policy to an EU level – not just claiming it’s open in the title, but being really open. Not for the sake of being open, but for the sake of higher quality. It won’t necessarily prevent abandonware altogether (because useless software gets created all the time, and useful software gets forgotten), but it will reduce the wasted effort.
See if you can find someone at the Wisconsin statehouse. I know they were hiring for a project to build a collaborative editing app for drafting legislation a few years back.
I have experience with a few of the EU-sponsored FP7 projects – the ones in the ICT track (the ones you refer to are among them) are generally awful. It’s a horrible waste of money.
It all starts with the fact that during the application assessment stage no attention is spent on whether the people leading the project have EVER done any good work in the area of the project. I’ve seen quite a few work package leaders with little understanding of the area the project is, in theory, working in.
Things only get worse once the funding is granted as all the EU looks at is whether the time sheets and the expenses have been properly filed – nobody checks what sort of work has been done during that time and nobody really looks at the relationship between declared results and the initial goals. Most of the final reports are very fluffy essentially saying something like “A lot of research has been done, we made progress, but, of course, more research is needed.” That’s it.
At the very least the EU should at least collect some metrics (e.g. number of times a paper or a patent produced during the project has been cited) but this is not being done.
So software produced in these projects often has little to do with the topics of these projects and/or is of little use to anyone outside of the project consortium. This is only made worse by the fact that software produced by two thirds of the academic researchers generally sucks – test coverage is often non-existent and CI is unheard of. It is just awful. So the fact that is not being made open is just the tip of a very shitty iceberg.
damn..
well, we have to uncover that iceberg (or “shitberg”)
Hi Bozho,
as much as I agree that open source is the way to go I don’t think the problem here is open vs. proprietary. IMO it has to do with how software is ordered/purchased from the state, how the entire contract/process is managed and what happens after that.
If you don’t have clear software management goals whether or not it is open source doesn’t really matter. There are thousands of dead open source projects out there.
The problems I see here are:
1) During the development phase, who manages the requirements from the client, e.g. client acceptance testing ? Probably nobody.
2) When a software is deployed to some state organization what happens to the source code? My guess is it is not passed along with the binary files as a deliverable artefact to the customer. But it should be, regardless of whether it will be open sourced to the public or not. Looks like this changed recently which is good.
3) Who and how maintains the software after the initial release ? Requirements are always changing and if the software is not updated to follow them it becomes obsolete and is replaced with something else. I’ve seen this even in private companies who even had in-house devel teams but poor management.
IMO this is a classical software management issue which happens even in medium sized companies, not to mention banking, finance or state institutions. Switching to open source will not auto-magically solve this b/c it is not a technological problem, it’s organizational.
The first step in solving this is to have a state CIO and CTO and in-house devel teams which initially should be responsible for supporting the software. Also they should probably be involved in the creation process, if not entirely at least partially. This new organization should be independent from political parties and should have the power to control governmental spending on software nation wide.
That’s exactly what we are doing with creating the national e-government agency and a state-owned company to do the support. It will be sort-of a CIO (Bulgarian thoughts here http://blog.bozho.net/blog/1763)
But the reality is such, that the state cannot have enough control over its projects. Which is why we are opensourcing them, so that citizens can also be involved in the process
Whilst I can only be happy to read or hear quotes about the Hithchiker’s Guide to the Galaxy, the comment on the hidden desk in a hidden basement with a leopard warning is not really accurate.
I would rather describe it as a lot of fruit stands next to the road, each with a small-sized sign. You drive and maybe identify some of them, but chances are slim.
I know, it’s a post from 2015 and new projects are by default required to be open source. But in most EC-funded projects I have worked in for the last few years -also before 2015 and new calls for projects-, open source and doc was already the norm and EC has been pushing for it.
IMO the problem today is not that the source is not accessible, but that it is difficult to encounter -and thus reuse- by others. It’s not easy, when starting a project, to find work that we can reuse. Not even in GitHub/Bitbucket.
In any case, it’s nice to read that Bulgaria sums to the open source fostering initiatives. Signing off contracts by default for privative software is not necessarily the best option. Also, if software can be properly improved and audited by a public community and enhance public service functioning, that’s always a plus. There’s a lot of work to do on that.