Ongoing Exploitation of the Log4j Vulnerabilities

On December 9th, Apache confirmed a critical zero-day vulnerability impacting the Log4j Java-based logging library that is being tracked as CVE-2021-44228. Known as Log4Shell or LogJam, CVE-2021-44228 is an unauthenticated remote-code execution vulnerability in versions 2.0-beta9 through 2.12.1 and 2.13.0 through 2.14.1.

The underlying issue resides in how Log4j handles user-supplied data when using the Java Naming and Directory Interface (JNDI). According to Apache, “An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled.”

The prevalence of Log4j across many open-source and commercial applications, ease of exploitation, and potential attack surface makes this one of the most severe vulnerabilities in some time with a CVSS score of 10.0.

Attacks Observed (So Far)

As of December 15th, eSentire has observed over 140,000 exploit events across our network telemetry. Attacks are increasing, peaking at over 40,000 events on December 14th.

Figure 1 Network Exploit Events Since December 11th

Observed Exploit Methods

Observed exploit attempts consist of opportunistic attacks against publicly facing systems as attackers rush to compromise exposed systems before fixes or mitigations are in place.

Initial observations comprised of attackers inserting strings to trigger the remote code execution (RCE) into various fields within the HTTP headers. This method is the quickest and most scalable way of widely exploiting many hosts, given HTTP is a common internet-facing protocol and there is a high-probability that these strings will be logged and processed by Log4j, thus triggering the exploit.

Figure 2 Log4j Exploit Attempt (Source: eSentire MDR for Network Detection Event)

Attacks were quick to evolve over the weekend. By Sunday, exploits with obfuscated strings were observed. The goal is to bypass filtering by Web Application Firewalls (WAFs) while still achieving the desired result. Fortunately, the security community has been quick to identify these obfuscation methods and update detections accordingly.

Figure 3 Log4j Exploit Attempt (Source: eSentire MDR for Network Detection Event)

Figure 4 Remote Java Class Retrieval from Successful Exploitation (Source: eSentire MDR for Network Detection Event)

JNDI protocols leveraged in observed exploit attempts include LDAP (and LDAPS), DNS, RMI and IIOP. As of December 16th, LDAP made up 60% of non-obfuscated exploit attempts, far more than other protocols. Exploit attempts to evade filtering through obfuscation are increasing, with 49% of events involving obfuscated strings on December 15th, compared to 24% on December 13th.

Exploit Protocols

Figure 5 Exploit Protocols in Observed Network Events

These initial exploitation methods are unlikely to be the last. We still do not know the full scope of the attack surface, and security practitioners continue to find avenues for exploitation. Ensuring applications are patched will continue to be a priority in the coming weeks.

Observed Activity

Multiple botnets including Mirai, Muhstik and coinminers have been observed. As is common with other web-facing vulnerabilities, the initial round of exploitation involves coinminer deployment. Coinminers are a rapid and scalable means to monetize compromised hardware with little effort. Concerningly, access brokers, ransomware, and APTs have also jumped into the fray.

Cobalt Strike Deployment

On December 14th, eSentire security teams responded to the successful exploitation of CVE-2021-44228 leading to attempted Cobalt Strike deployment on a customer’s web server. Analysis of endpoint telemetry showed PowerShell spawning with an encoded command for executing shellcode containing a Cobalt Strike payload which would call home to IP 152.136.226[.]175.

Figure 6 Log4j Exploitation Leading to Cobalt Strike Deployment

Pivoting on this IP, our cyber analysts identified matching exploit strings in application logs for the server:

Figure 7 Log Capture Of Log4j Exploitation

The above exploit strings are artifacts associated with this JNDI Exploit Kit (or a similar variant). The tool is used to generate JNDI links for connecting back to the attacker’s server where commands/payloads can be executed.

Figure 8 JNDI Exploit Kit Payloads

Artifacts of this tools can be seen in other exploit attempts:

Figure 9 JNDI Exploit Executing Encoded Commands

Figure 10 Decoded Commands

The Basic/Command option can be used to execute arbitrary encoded commands on the system once a successful JDNI callback is achieved, offering attackers flexibility in follow-on actions.

Recommendations from the Threat Response Unit (TRU)

eSentire released an updated advisory on this topic on December 13th.

This issue remains dynamic as organizations and vendors grapple with the scope of Log4j use and ongoing exploitation. We have provided two sets of recommendations below: (1) how your team can conduct further investigate possible exploitation; and (2) how you can mitigate the vulnerability.

How to Mitigate the Log4j Vulnerability

While Apache has released security updates, the fix is not as simple as applying a single patch across the enterprise. We recommend the following:

• Apply security updates for each vulnerable application in the supply chain.
• As of December 16th version 2.16.0 is the recommended Log4j version
  • Originally identified on December 14th, CVE-2021-45046 led to denial-of-service in version 2.15.0. Researchers have since found a method for data exfiltration and remote code execution, increasing the severity.
• Audit software vendors/applications for Log4j use and make applying security updates a top priority.
• Both CISA and NCSC-NL have released a vendor vulnerability status which can be correlated against software inventories.
• NCSC-NL has additionally compiled various methods and scripts for scanning applications and systems for Log4j use.

During this critical patch window, ongoing monitoring efforts will be essential. eSentire’s Managed Vulnerability Scanning service has enabled plugins for CVE-2021-44228 and will enable checks for vulnerable software as it becomes available. Our teams have also observed artifacts of Log4j exploitation across network, logs, and endpoint. Notably, endpoint telemetry has proven to be valuable in identifying and scoping exploit activity (see Notes for Defenders below).

Notes for Security Practitioners to Investigate Further Exploitation

• Network signatures are effective in identifying exploitation attempts and log/endpoint data is needed to trace post-exploitation activity.
• Endpoint telemetry has shown to be effective in investigating Log4j exploitation thus far.
• Exploitation in Windows occurs under the Java process executed by the application.
• Initial assumption that the string “Log4j-core-version.jar” would appear in the command line path for Java process does not appear to always be the case.
  • In multiple successful exploitation events, this string did not appear in the command line including the Cobalt Strike case above.
• We recommend tuning detections or queries to look for unusual activity under the Java process, regardless of the presence of Log4j in its command line.

Figure 11 Endpoint Process View of Log4j Exploitation

• Observed exploitation resulted in notable activity under Java.exe processes.
  • Network Activity
    • DNS requests
    • Outbound connections over unusual ports (1389,10009, 9999 etc.) and common ports (389, 443, 80)
  • Process Activity
    • Java.exe spawning Cmd.exe, PowerShell.exe, Ping.exe, Curl.exe, Whoami.exe
      • Note that for curl and whoami, both commands run as child processes of Cmd.exe (e.g. Java.exe → Cmd.exe → Curl.exe)

Log4j remains an evolving situation, so beyond following the current recommendations to mitigate the vulnerability, it’s critical to stay updated with ongoing updates on exploit methods and adapt quickly.

If you’re not currently engaged with a Managed Detection and Response provider, we highly recommend you partner with us for security services in order to disrupt threats before they impact your business.

Want to learn more? Connect with an eSentire Security Specialist.