Trouble in Paradise

A case study of Cloud compromise

Many organisations are increasingly moving to cloud solutions to solve their hosting needs, but outsourcing workload should not imply outsourcing security as well. The importance of security the cloud was recently highlighted by targeting of Microsoft Azure environments by Nobellium, the threat actor behind the SolarWinds Orion compromise. The threat actor notably exploited stolen SAML certificates for vertical movement, a rarely seen technique. Even without novel techniques, less sophisticated cybercriminal threat actors can also pose a threat to companies’ services in the cloud. Indeed, this week’s supply chain compromise operation by REvil is suspected to have been launched from a compromised web server hosted on AWS.

The Incident

Recently, DarkLab’s incident response team has helped a South Asian client in the media sector to remediate an incident involving multiple cloud environments breaches, a case study we think can help organisations better plan for secure implementations of their cloud environments.

The incident originated from a likely exploitation of a known remote code execution vulnerability in a Jenkins instance, an open source software development automation server. The server was hosted in an Amazon Web Service (AWS) environment and had a hardcoded root access key. With that, the threat actor was able to roam the compromised environment undetected for four months. Logs availability has been an issue due to the lack of CloudTrail log retention but we know that the threat actor created multiple IAM user accounts and accessed internal data, including those stored in S3 buckets via the free Windows client S3 Browser.

Their primary intent, however, was to use the victim as a jumping spot to identify other targets vulnerable to the same Jenkins RCE and move laterally to their servers. They did so by deploying Linux and Windows virtual machines in new EC2  instances  in the compromised environment to scan and exploit external IP addresses. The did so using T.2 micro sizing to avoid spikes in usage and remain hidden. The attacker deployed the additional EC2 instances in a different AWS region than that used by the victim, an anomaly that we suggest organisations monitor for.

A deeper dive into the system log of the Linux VMs shows that the attacker likely used Shodan to identify other vulnerable Jenkins instances online, suggesting their targeting was likely opportunistic. Similarly, analysis of the IP addresses used by the attacker to access our client – most of them AWS instances themselves – suggests the attack likely originated from multiple other compromised organisations.

From AWS, the threat actor managed to access a FTP server within a parallel Google Cloud Platform (GCP) environment. For this, they used a compromised hard-coded credential found in one of the configuration files in their BitBucket repository, also suspected to be compromised. After thorough environment and users’ enumeration, the attacker was able to obtain the password for another G-Suite user account, which they used to access data in the GCP environment and Google Drive.

Shortly after accessing the GCP,  threat actors attempted to cover their tracks by deleting the company’s entire production environment, all hosted on AWS, and the backup copies. Fortunately, AWS retained some copies of the deleted backups which were able to provide to the victim organisation.

However, while the victim restored their AWS system they were not aware to reset the root access key. Unsurprisingly, the attacker quickly re-established a presence in their cloud and a few days later they re-deleted the production environment, although no ransom demand was recorded. This was when our incident response team was called to help.

Assessment

Our investigation suggested that the threat actor behind this campaign is likely operating opportunistically and with a relatively low technical know-how. We often found traces of internet searches for open source tools or “how to” techniques. Nonetheless, such an actor could still pose significant operational damage to a large company by deleting their production environment.

The incident shows how even relatively unsophisticated threat actors are adopting an island-hopping approach by abusing imperfect implementations of commercial cloud platforms. Companies should ensure that standard security practices, like rotating passwords or access keys, monitoring suspicious activities, and prompt patching, are also applied to cloud environments.

What’s next?

Our experience suggests that this was not an uncommon attack path for adversaries targeting cloud environments. Monitoring for common attack vectors can help indeitifyuing supicious behaviour earlier and contain an incident before it is too late.

Below are some monitoring metrics mapped against Mitre ATT&CK tactics that we recommend organisations implement to AWS Config, Lambda, or their choice of CSPM platforms for automated detection and remediation.

Feel free to contact us at [threatintel at darklab dot hk] for the full set of 50 custom MITRE-based rules on AWS

TacticTechnique (custom)Log Source
Initial accessAWS user login failed multiple timesCloudTrail
Initial accessMultiple worldwide successful console login GuardDuty
Initial accessPotential Web scanning activities with multiple web server 400 error from same the source IPWeb access log
Privilege EscalationAWS “AssumeRole” from rare external AWS accountCloudTrail
DiscoveryAWS potential IAM enumeration ActivitiesCloudTrail
Defense Evasion/ PersistenceCreate/Update managed policy with excessive permissionCloudTrail
ImpactAWS Access Key EnabledCloudTrail
ExfiltrationEgress rule added to a security groupCloudTrail

Feel free to contact us at [darklab dot cti at hk dot pwc dot com] for any further information.