The xz Backdoor Should Not Happen Again
A few days ago a significant supply chain attack attempt was accidentally revealed – the xz utiliy was compromised, likely by a nation state, in order to plant a backdoor which allows sniffing on encrypted traffic.
The xz library is a building block of many other packages and is basically ubiquitous. A famous XKCD strip describes the situation graphically:
This means that if it wasn’t accidentally discovered due to worsened performance, we would eventually have a carefully planted backdoor on practically every Linux server out there. This is a major issue and even though open source security is better than closed source security, even if just by allowing backdoors to be discovered by anyone, we need to address such nation state attempts of planting backdoors.
I propose two complementary measures:
- Public funding for open source – the EU and the US need to create a structured, not overly bureaucratic process to fund the maintenance of core open source projects (like xz). Germany has done a good job in setting up its Sovereign tech fund, but we need broader instruments that make sure there is no open source abandonware on which many other projects depend. Currently large corporations often fund the development of open source, but xz is an example that the little building blocks may fall through the cracks. Open source funding can also be directed at systematic security analysis of open source projects (like the one in point 2, but not limited the security services).
- Analyzing high-risk project – security services and other public and private organizations need to first pinpoint high-risk projects (ones that if compromised, cause a huge risk that trickles down to the whole ecosystem), rank projects based on risk, and then analyze no just source code, but also maintenance activities, maintainer recruitment and churn, commit patterns and so on. In hindsight, the xz backdoor could have been caught by monitoring such metadata and the suspicious activities by the “hacker”. We, of course, need (open source) tools to do these analysis, but also highly-skilled people in the security services of larger countries.
Overall, we can and should learn lessons and take measures based on this incident. Because the next one might not cause noticeable performance degradation and get into actual production, which will be devastating.
A few days ago a significant supply chain attack attempt was accidentally revealed – the xz utiliy was compromised, likely by a nation state, in order to plant a backdoor which allows sniffing on encrypted traffic.
The xz library is a building block of many other packages and is basically ubiquitous. A famous XKCD strip describes the situation graphically:
This means that if it wasn’t accidentally discovered due to worsened performance, we would eventually have a carefully planted backdoor on practically every Linux server out there. This is a major issue and even though open source security is better than closed source security, even if just by allowing backdoors to be discovered by anyone, we need to address such nation state attempts of planting backdoors.
I propose two complementary measures:
- Public funding for open source – the EU and the US need to create a structured, not overly bureaucratic process to fund the maintenance of core open source projects (like xz). Germany has done a good job in setting up its Sovereign tech fund, but we need broader instruments that make sure there is no open source abandonware on which many other projects depend. Currently large corporations often fund the development of open source, but xz is an example that the little building blocks may fall through the cracks. Open source funding can also be directed at systematic security analysis of open source projects (like the one in point 2, but not limited the security services).
- Analyzing high-risk project – security services and other public and private organizations need to first pinpoint high-risk projects (ones that if compromised, cause a huge risk that trickles down to the whole ecosystem), rank projects based on risk, and then analyze no just source code, but also maintenance activities, maintainer recruitment and churn, commit patterns and so on. In hindsight, the xz backdoor could have been caught by monitoring such metadata and the suspicious activities by the “hacker”. We, of course, need (open source) tools to do these analysis, but also highly-skilled people in the security services of larger countries.
Overall, we can and should learn lessons and take measures based on this incident. Because the next one might not cause noticeable performance degradation and get into actual production, which will be devastating.
We can do more to make sure that open source maintainers are properly paid and compensated for all the effort they put into the tools they create. Currently if you are an FOSS maintainer, you can hope for one of the following:
– your project be important for FAANG or IBM/Red Hatso they give you engineers as maintainers
– one of the FOSS foundations (linux, apache, canonical) to notice you (like what’s happening with redis atm)
– get free scans/ci/cd minutes and infra from the giants (aws/github etc)
– get government/EU funding (sovereign tech fund, there are other of other ones in NL/Germany where I currently live instead of my native Bulgaria, etc) if your project is deemed important
– get sporadic contributions on ko.fi / paypal / github sponsors that are rarely enough to pay the bills
So most of us are doing this not as a full time thing, but as a hobby project. Until the financial incentives are there, we simply can’t do more.
I simply can’t wait to put more actual time in open source instead of my 9-5 job, but currently it’s not at all financially viable.
You have the unique position of being in politics and IT at the same time, and to be able to potentially drive legislation to change that, or to challenge business to give their fair share towards FOSS. Until things for FOSS change financially, the xz thing, the terraform/redis license changes will be commonplace ocurrences.