Scenario:

• A User has a machine installed with an Application Client and connects to the Web Application Server which in turn connects to the Database servers on the back-end.
• The Client machine has been reported to be being slow.
• Some basic SIEM based analysis of Network logs for the client machine shows some Network connections resembling a Port Scan from the client machine in its own subnet
• The machine is also doing some random internet connections to Public IP Addresses.
• On checking the Anti-Virus software on the machine it shows as being outdated.
• Suspicion is that the machine is infected with some kind of a malware.
• Since this client machine has connections to the Critical App and Database server, this event is critical

Should I press the panic button?

Before we press the panic button (raise an Official Security Incident which has SLA tagged to it), there are several questions we need answers for. This is something I learnt from one of my mentors (he blogs here). What I learnt was that before beginning a Security Incident investigation and press the panic button, we need to sit down and collect as much as corroborating data as possible. In order to do that we can follow a series of logical steps:

1. Collect the Public IP Addresses it connects to and find out more about them. Sometimes a simple Google search can help to determine the authenticity and reputation of the website/domain or public IP.
2. Establish Ground Zero for the infection. This can be determined by historical log data. Some SIEM tools do this natively for you. Else, you can use a white board or pencil/paper to draw this visualization yourself.
3. Gather as much as possible infected machine IPs and sort them geographically, Network Zone based and if possible VLAN based as well.
4. Other Transitive Infections that could be identified.
5. Analyze one of the infected machines to gather Malware analysis data

Next Steps – Remediation, Control etc

• Quarantine all the machines infected.
• In case of the server, ensure that all instances (if any) of the infected file are cleaned.
• In case of a database, if we exactly know the infection source and possible file names, try purging them from the database contents.
• Submit the samples of the Malware to the Security Vendor and ask them to start working on a Signature or definition.
• Perform a Reverse Engineering of the Malware to identify specific Network connections/Behaviors so that they can go into our prevention systems in the Network. (Read Network IPS, Host IPS, Proxy etc). Reverse Engineering skill set  is a good skill to have because for Zero day scenarios, vendors are slow to react. If a Reverse Engineer is available, the job of getting a control measure put in place (although a crude one) will be quicker.
• Document the characteristics of the malware, the response taken and the findings during the course of this entire incident.