XZ: How less than a second saved our lifes?

Posted on April 8, 2024

How Lady Luck saved us from cybersecurity disaster? Imagine something that is on almost every Linux machine and this thing has a backdoor. And what could it cause...

Introduction

OSS Community discovered shocking finding that new version of XZ Utils library - that is used for compression on many popular Linux systems had a backdoor created by developer Jia Tan. This backdoor allowed hackers that were in possession of a special key to connect as a user with administrator privileges.

But how was it discovered, and who is this "guy" Jia Tan?

Discovery

Solo developer Andres Freund from Microsoft noticed that logging into his Debian Sid system took longer than usual – by half a second. Further analysis showed that the xz package had become a Trojan horse, potentially affecting most Linux distributions.

How was it pushed?

Jia Tan leveraged the collaborative nature of open-source software development, where any individual can propose modifications to a program hosted on platforms like GitHub. These modifications are then evaluated by other developers before being merged into the existing software. Jia Tan's involvement in the open-source community began in November 2021 under the GitHub username JiaT75. Over the following year, they contributed to various open-source projects using the names Jia Tan and occasionally Jia Cheong Tan, eventually turning their attention to contributing to XZ Utils.

By January 2023, Jia Tan's contributions to XZ Utils started being accepted. Within the next year, they progressively assumed a dominant role in the project, overtaking the original maintainer, Lasse Collin. This shift in leadership was partly facilitated by a series of persistent emails from a handful of users to Collin, lamenting the slow pace of updates. It remains uncertain whether these users were inadvertently aiding Jia Tan or were in collusion with them to convince Collin to step down. None of these users responded to inquiries for comments from WIRED. In February of this year, Jia Tan ultimately inserted a covert backdoor into a version of XZ Utils.

Identity of Jia Tan

This precise, very patient approach and the technical skill required to do such thing with perfect OpSec execution lead some people to believe that Jia Tan must, in fact, be a handle operated by state-sponsored hackers.

Experts have observed that this individual maintains exceptionally high standards of operational security. Independent cybersecurity journalist Brian Krebs mentions that he was unable to find any evidence of Jia Tan's email address beyond the communications sent to other contributors in the open-source community, despite extensive searches through databases of compromised data. Moreover, it seems Jia Tan consistently used a VPN, masking their online activities with an IP address from Singapore.

Sources

  1. Tukaani Project's Website

  2. OpenWall - backdoor in upstream xz/liblzma leading to ssh server compromise

  3. Everything I Know About the XZ Backdoor - Evan Boehs

  4. The Mystery of ‘Jia Tan,’ the XZ Backdoor Mastermind

  5. Debian Bug Report