What and How much to Collect concentrated on giving a starting point for log collection and usage. Here, I am going to talk about Log Filtering for Security Investigations and Analysis purpose. If you are collecting logs from a troubleshooting perspective, you can collect without filtering, but for Security analysis, we would need less of haystack so that we can have a better chance of finding the needle.
In organizations, where several thousands of devices are sending Syslog, managing them is really a nightmare. In such scenarios, more focus is required on Log filtering. Log Filtering for Security Infrastructure use is one of the most ignored aspects of Log Management but it is the most important aspect for a cleaner and efficient log management and Security Event Analysis.
Log Filtering can be done at the Source or at the Syslog Server Location.
- Filtering at Source is the best approach when it comes to Log Filtering. This provides for better control of your infrastructure as you know what is being logged and what is not. Indirectly, it also aids in less utilization of System Resources. But the downside is, this is the most difficult to implement. Network devices have very little capability of Source filtering. Firewalls have a lot of options when it comes to Logging, but Routers and Switches offer very less. UNIX offers basic filtering natively and with third party tools like rsyslog, syslog-ng offer granular filtering capabilities. As all would know, there is no native Syslog capability in Windows and it would typically need a third party client to fill in for this lack of capability. This works in favor of filtering because; generally Third Party tools give filtering options.
- Filtering at Destination is the most practical and easiest to implement when it comes to Log Filtering. Here, Log Management Solutions or SIEM Tools take care of filtering what is needed and what is not. However, your Source Devices will be generating tons of logs and will be using System resources as well as Bandwidth.
Based on your organizations Architecture, it will help to decide on what is easier to implement and manage. Both have their pros and cons and its up to the Security Teams to decide. Once the filtering approach is decided, it is time to move on to the “Filtering” itself. This is the hard part. Before you start filtering, start understanding the device families present in the Environment. Every Vendor logs events differently and every event log means differently as well. So, care should be taken that we don’t end up filtering logs for one version and in reality we have another.
Example Log Filtering for Cisco ASA:
Let’s take an example of a Cisco ASA Firewall. The Cisco Adaptive Security Appliance (CISCO ASA in short) Operating System generates several logs. Out of the several thousands of messages, the most important events from a Security perspective are the following events given in the table. These events are built in protection defaults for Cisco ASA appliances. Apart from the following events, we can pick and choose Traffic Based events for analysis purpose. If interested to receive the full list, please comment below and I will put up that list too.
| %PIX|ASA-2-106016 | Deny IP spoof from (IP_address) to IP_address on interface interface_name. – Spoofing Detection |
| %PIX|ASA-2-106017 | Deny IP due to Land Attack from IP_address to IP_address – Spoofing Detection |
| %PIX|ASA-2-106020 | Deny IP teardrop fragment (size = number, offset = number) from IP_address to IP_address – Teardrop Attack |
| %PIX|ASA-1-106021 | Deny protocol reverse path check from source_address to dest_address on interface interface_name – Protocol Reverse Attack |
| %PIX|ASA-1-106022 | Deny protocol connection spoof from source_address to dest_address on interface interface_name – Spoofing Detection |
| %PIX|ASA-3-109010 | Auth from inside_address/inside_port to outside_address/outside_port failed (too many pending auths) on interface interface_name. (floodguard enable) |
| %PIX|ASA-4-109017 | User at IP_address exceeded auth proxy connection limit (max) – DOS Attack Possible |
| %PIX|ASA-4-109022 | exceeded HTTPS proxy process limit – DOS Attack Possible |
| %PIX|ASA-5-111001 | Begin configuration: IP_address writing to device – Privilege Use |
| %PIX|ASA-5-111003 | IP_address Erase configuration – Privilege Use |
| %PIX|ASA-6-113006 | User user locked out on exceeding number successive failed authentication attempts – Brute Force |
| %PIX|ASA-3-201002 | Too many TCP connections on {static|xlate} global_address! econns nconns – DOS Attack Possible Behavior |
| %PIX|ASA-3-201004 | Too many UDP connections on {static|xlate} global_address! udp connections limit – DOS Attack Possible Behavior |
| %PIX|ASA-4-209003 | Fragment database limit of number exceeded: src = source_address, dest = dest_address, proto = protocol, id = number – DOS Attack Possible Behavior |
| %PIX|ASA-4-209004 | Invalid IP fragment, size = bytes exceeds maximum size = bytes: src = source_address, dest = dest_address, proto = protocol, id = number – Possible Intrusion Event |
| %PIX|ASA-4-209005 | Discard IP fragment set with more than number elements: src = Too many elements are in a fragment set – Possible Intrusion Event |
| %PIX|ASA-3-210011 | Connection limit exceeded cnt/limit for dir packet from sip/sport to dip/dport on interface if_name – DOS Attack Possible Behavior |
| %PIX|ASA-4-405001 | Received ARP {request | response} collision from IP_address/MAC_address on interface interface_name to IP_address/MAC_address on interface interface_name – ARP Poisoning |
| %PIX|ASA-4-405002 | Received mac mismatch collision from IP_address/MAC_address for authenticated host – ARP Spoofing |
| %PIX|ASA-2-410002 | Dropped num DNS responses with mis-matched id in the past sec second(s): from src_ifc:sip/sport to dest_ifc:dip/dport – Possible DNS Attack Detected |
| %PIX|ASA-4-412002 | Detected bridge table full while inserting MAC MAC_address on interface interface. Number of entries = num – Possible L2 Attack Detected |
| %ASA-4-424001 | Packet denied protocol_string intf_in:src_ip/src_port intf_out:dst_ip/dst_port. [Ingress|Egress] interface is in a backup state – Possible Intrusion |
| %ASA-4-424002 | Connection to the backup interface is denied: protocol_string intf:src_ip/src_port intf:dst_ip/dst_port – Possible Intrusion |
| %PIX|ASA-6-605004 | Login denied from source-address/source-port to interface:destination/service for user “username” – Possible Intrusion |
| %PIX|ASA-5-304001 | user@source_address Accessed {JAVA URL|URL} dest_address: url |
If you carefully note the structure of the Event message itself you will notice that almost all of the Syslog Messages I have chosen have facilities that range from 1 to 5. These Event IDs can be used for analysis of different scenarios. Here, I have listed down some pointers on how to select these Events of Interest. Some approaches that we can use are:
- Focus on the Utility of an Event rather than the data it generates. The description of an Event ID will be misleading many a times, and hence looking at the Generated Data from the Event is important.
- Focus on what events will help you track back the actual Attacker/User/Service responsible for triggering the Event in the first place. This event independently or in tandem with another should be able to help you
- Look for what Event Collection is more suited to your environment in terms of Security Monitoring. For Example, if you have an Internet Firewall, Spoofing events don’t make sense to collect, whereas internal firewalls would need spoofing events.
- Look for Optimal Event ID Selection for auditing so that you get the right amount of data with One Event ID rather than many. This may be difficult in some cases with some devices, but still where ever possible this should be employed.
- Finally, see if your SIEM tools can parse these event properly for you to analyze
Now let us look at Juniper NetScreen Firewall Events using the same approach as we used for Cisco ASA.
| Critical (00031) Message | 〈string〉 detected an IP conflict (IP 〈IP address〉, MAC %m) on interface 〈string〉 |
| Notification (00031)Message | ARP detected IP conflict: IP address 〈ip〉 changed from interface 〈if_old〉 to interface 〈if_new〉 |
| Notification (00051) Message | Static ARP entry added to interface 〈string〉 with IP 〈IP address〉 and MAC %m |
| Notification (00052)Message | Static ARP entry deleted from interface 〈string〉 with IP address 〈IPaddress〉 and MAC address %m |
| Emergency Message | Ping of Death! From 〈src_ip〉 to 〈dst_ip〉, proto 1 (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number〉 times. |
| Emergency Message | SYN flood! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉, proto TCP (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number〉 times. |
| Emergency Message | Teardrop attack! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉, proto { TCP | UDP | 〈number1〉 } (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number2〉 times. |
| Alert Message | Address sweep! From 〈src_ip〉 to 〈dst_ip〉, proto 1 (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number〉 times. |
| Alert Message | ICMP flood! From 〈src_ip〉 to 〈dst_ip〉, proto 1 (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number〉 times. |
| Alert Message | IP spoofing! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉, proto {TCP | UDP | 〈number1〉 } (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number〉 times. |
| Alert Message | Land attack! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉, proto TCP (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number〉 times. |
| Alert Message | Port scan! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉, proto { TCP| UDP | 〈number1〉 } (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number2〉 times. |
| Alert Message | Source Route IP option! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉, proto { TCP | UDP | 〈number1〉 } (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number2〉 times. |
| Alert Message | UDP flood! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉, proto UDP (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number〉 times. |
| Alert Message | WinNuke attack! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:139, proto TCP (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number〉 times. |
| Critical Message – | Bad IP option! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉, proto{ TCP | UDP | 〈number1〉 } (zone 〈zone_name〉, int 〈interface_name〉).Occurred 〈number2〉 times. |
| Critical Message | EXE file blocked! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉, proto { TCP | UDP | 〈number1〉 } (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number2〉 times. |
| Critical Message | FIN but no ACK bit! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉, proto TCP (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number〉 times. |
| Critical Message | SYN and FIN bits! From 〈src_ip〉:〈src_port〉 to 〈dst_ip〉:〈dst_port〉,proto TCP (zone 〈zone_name〉, int 〈interface_name〉). Occurred 〈number〉 times. |
If you see in the case of Juniper, Alert and Critical messages are sent to Syslog and these events are not always identifiable using Event IDs. From a Syslog facility all these would be still at Level 5. These events are also default Security Messages generated from Juniper. In addition to these messages, we would be collecting the Traffic Logs (Event ID 000257) from Juniper that helps in Security analysis for correlation.
This same approach can be used for Windows, Unix, Security Application Devices etc. For more details of specific Device type filtering, please request in the comment section and I will post the filtering for the same.
If you effectively identify the Events of Interest for various devices and filter out the chaff, you would be able to harness the power of Syslog for your Security Investigation purposes. SIEM is getting bigger by the day in terms of data usage. The world is moving towards big data collection. But the key thing to note is, in spite of data growth, the quality of SIEM tools in log analysis has not been going North. Quality does not essentially come with Quantity. Filtering will always be needed no matter how much logs we collect.
Remember “Logs don’t lie”
