Monthly Archives: February 2012

Log Management

What and How much to Collect – Enterprise Security Logging

Gallery 2 Comments

One of the major challenges in Enterprise Log Management is “What to Collect and How much to Collect?”. The How to collect is the easy part with several Enterprise as well as open source tools available in the market that help with this. Source Device themselves have several log forwarding mechanisms and features which we can leverage to collect logs.
Most of the times, Log Collection and Management is driven by Standards, Compliance Requirements and other external drivers. Good Security Practice calls for Log Collection and Management to be driven by Enterprise Policy. So in order to implement a Log Management Program, the need to have a strong Enterprise Policy is paramount. This policy once defined should be implemented thoroughly right from the OS Build, Application Certification, IT Infrastructure Design to Solution Approval, Change Approval, and other Operations Process.

What to Collect:

“Never start Log Management Program without knowing your IT Infrastructure”. This is what I would suggest to anyone who wants to start Log Management Program. Knowledge of the network is important to decide on Log Collection Scope and Volume. Generally, we collect logs for several reasons ranging from Troubleshooting IT Issues to Forensic Investigation. My focus in this blog post is of course on Security Specific Log Collection. I would like to mention here that there is no right or wrong way in approaching Security Log Collection and hence what I am trying to catalogue here are my experiences of what has been useful in Security Investigations in my career.

  • Core Router– Though Router Logs are chatty, collection of Core Router Logs are important because, if your enterprise Core Router is over-run/compromised then your enterprise is wide open. These router logs can be heavily filtered to identify only DoS/DDoS attack scenarios as well as Exploit Scenarios (By exploiting a vulnerability and owning the router).
  • Network Firewall– Firewall Logs are the most helpful in identifying Security Incidents involving Malware, Compromised Hosts, Spam Relays and several other Network based Attacks and Compromises. Keep in mind that the logs don’t directly tell you that a Malware is found in the network, but it gives a great corroborating evidence (In terms of when and what connection was made) to any of your Security System Alerts.
  • Network IDS/IPS– Intrusion Detection Systems are a crucial part in the Security Incident Detection and Investigation process. The alerts received from the IDS/IPS gives a lot of information. This when combined with Firewall Logs help to identify the entire Attack Scenario.
  • Authentication Servers– In a Network there are several Authentication mechanisms for users to access IT Infrastructure. Most of them go off a Directory service or Database service. Most famous of the these Authentication servers are Active Directory/LDAP, RADIUS, TACACS, Internal User Database etc. From a Log collection perspective, in order to identify the User, his machine and the Authentication time stamp, we would need logs from Authentication Servers.
  • Endpoint Security Solutions– The last line of defense in any organization is the Security Systems put on the Servers or the End User machines. These tools can be of great help in a Security Investigation by providing information on whether the Host Defense system detected an intrusion or not.
  • Application Servers– Application servers are the main reason one has to protect the network. The application servers be it Web or Database are critical to an Enterprise. We would need to monitor these systems closely for any kind of breach of compromise by collecting logs from Operating System Level as well as Application Level
  • Misc Logs – In addition to the above critical log Sources, Enterprises may choose to collect Logs from other supporting IT Infrastructure as well. This however is an all-inclusive list of devices like Proxies, DHCP, DNS, Other Security Tools in the environment, Switches, WAN Devices etc

How much to Collect:

One of the oft ignored aspects of Log Management is Standardization of the Logging Infrastructure. This means that the format should be standardized so that all the Logs can be collected in a Centralized fashion and used for normalization, integration with SIEM Solutions, etc. Almost all the solutions support Syslog as a Logging Format. Network Devices and Unix Devices have Syslog daemons running natively and can be used to send logs to a Central Logging Infrastructure. As far as I know, most of the Security Appliances/Servers also have Syslog capability. Microsoft Windows servers don’t have the capability to send Event Logs through Syslog and hence we would need to use Third-Party tools like Snare, Syslog-NG, WinLogD, Kiwi etc.
Syslog has several levels as highlighted below in the diagram and one should carefully chose the level appropriate to the Organization.

From a Security Perspective, recommendation would be to choose either Level 5 or Level 6. Level 5 mostly covers all Security related incidents logging. Level 6 gives detailed incident logging but will be very voluminous. Level 6 enabled Syslog setting would need extensive filtering to ensure the Syslog Infrastructure is not overwhelmed in volume and subsequently Storage space.

To summarize, More and More information collection through Logs would be invaluable in Security Investigation. Many would say that Log Collection is a “Reactive Approach” to Security Management but remember “Logs Don’t Lie”
I would be writing a detailed Blog post very soon to give details on What Logs to keep and What Logs to Filter. I will try to do it by device type so that it will be easy to read.

Please visit the follow-up post – What Logs to Keep and What to Filter

Security Investigation Series

Episode 1 – Security Investigation Series – Torrents

Gallery 4 Comments

Preface: I would like to share a methodology I used to track down an individual who was involved in downloading illegal/copyrighted movies resulting in a legal lawsuit for the company. In this post, I would try to simplify the entire thought process into simpler steps so that the reader can get an understanding on how Security Investigation can be done. Several organizations block torrent use in the Network, but several organizations don’t have such a policy in place. This investigation methodology is for people who are part of organizations of the second type 🙂

The Tip: Every Security investigation starts with a tip. In this particular case, there were three tips to work off

1. Public IP Address Registered to the Company
2. File Name of the Torrent Download
3. Time Stamps of the Download

More and More Information – Logs

Based on this above information it will be very difficult to identify who was the exact person who downloaded the Torrent. In order to do this Security Investigation, we would need logs from several devices in the IT Infrastructure. The places from where we would collect these logs can range from Network Devices to Hosts. It really depends on the level of logging enabled in the enterprise. If there are no logs, identifying this type of incident would be impossible. More and More information will help in the investigation process.

What and How much Logs to Collect is a section where I talk about typical devices in IT Infrastructure and common ways to collect, filter, analyze and store logs. For this particular investigation, the following logs were of great use:

1. Firewall Logs
2. IDS/IPS Logs
3. Authentication Server Logs
4. VPN Logs
5. DHCP Logs (if any)

Apart from the logs, we would need Corporate Network Awareness, Good Investigation skills and Patience.

Methodology:

1. Based on the Public IP (given in Tip 1), we can determine the Firewall from which the Traffic has been detected. This is critical in the case of Global enterprise wide Networks because, there are several firewalls deployed World over with several different Public IP addresses. Zeroing on a specific Firewall will help reduce the amount of logs to look for.

2. The next thing is to look for the above Identified Firewall logs during the Time Stamps of the event(given in Tip 3) to narrow it down to a manageable subset (this would still be Gigs of logs in case of high volume firewalls).

3. Now is the time to filter the Firewall Logs for P2P Traffic pattern. If an IDS/IPS is available in the Network and is connected to the said firewall, it makes the job easier to filter on all P2P Activity based on P2P Detect Signatures. If no IDS/IPS is available then we will have to follow a manual and time-consuming analysis for P2P.
P2P Analysis can be done in two ways. One is to look for Filter for UDP connections and see Torrent Client Port Activity. The other way is to look for Network Devices logs for Application Protocol detection capability to identify P2P activity (Symantec has a Detailed Article on Traffic Analysis of P2P here). Once the analysis for P2P is done, we can narrow down on the P2P Client set. This set is the list of Client machines that were involved in doing P2P activity during the said Timeframe (1 hour before and after the timestamp in Tip 3. This is done in order to ensure that we capture as much P2P traffic events as possible to be able to determine the exact client involved in this activity. Also, in highly utilized Network or in High Log volume environment, latency is an issue therefore having a buffer time frame is always a good option).

4. Once we have the subset of logs from the above Step, its time to identify the list of Source IP Addresses that are P2P Clients. This is present in both the IDS/IPS logs as well as Firewall Log. Doing a Union of these two data sources to narrow down the list of IP Addresses should be easy.

5. With the IP Addresses in hand, it is time to identify the hostname these IP addresses were assigned to during the specified time frame. DHCP Logs (In case of dynamic environments) can give this information. In DHCP Logs, we would get Hostname, IP and MAC Address of all the machines that have been allocated an IP within the said time frame. In case of Static IP Addresses, we would already have the information about the Hostname based on the IP addresses assigned.

6. At this juncture, we have the List of IP Addresses, the Associated Hostnames and MAC Addresses of the suspected clients. Based on the Authentication Server Logs, we can identify the Usernames associated with the above information set. Authentication Server Logs in this case are Windows (AD Logs). By filtering on the Logon events for the IP/Hostname we can identify the users.

Note: In case of VPN Connections, the VPN Connection entry has the user name information as well because VPN has a backend Authentication Component to it.

7. Once the Users are identified, we need to proceed with the Forensic Analysis of the individual machines, either remotely or physically. The clue to identifying the exact Person who did this, would be to search for the specific Torrent Filename’s (given in Tip 2) presence in the Torrent Application directory (Chances are they are not deleted by the user) or in the entire file system. The Movie file may also be present if we are lucky. In case of Movie sharing between users, its highly likely you would see Torrent + Data on one machine and Data on all other machines.

8. Once the person is identified, the appropriate teams should be notified as per the Security Investigation procedures defined for the Organization.

Note: The entire Log Collection and Analysis can be done quite easily if a SIEM Tool is available in the Enterprise. Else, it will be a painstaking job involving few hours of Analyst Time.

Conclusion:
A Security Investigation is possible only when we have data in the right format. Right tools as well as Right mindset is also important in the Investigation process. As mentioned earlier in the post, there is no right or wrong way in Security Investigation, only a way that works best for the Current Situation and Current IT Infrastructure Setup. I would greatly welcome suggestions and additions to this post so that I can make this series more and more useful.