This is a guest diary written by Mr. William Glodek – Chief, Network Security Branch, U.S. Army Research Laboratory

As a network analysis practitioner, I analyze multiple gigabytes of pcap data across multiple files on a daily basis. I have encountered many challenges where the standard tools (tcpdump, tcpflow, Wireshark/tshark) were either not flexible enough or couldn’t be prototyped quickly enough to do specialized analyzes in a timely manner.  Either the analysis couldn’t be done without recompiling the tool itself, or the plugin system was difficult to work with via command line tools. 

Dshell, a Python-based network forensic analysis framework developed by the U.S. Army Research Laboratory, can help make that job a little easier [1]. The framework handles stream reassembly of both IPv4 and IPv6 network traffic and also includes geolocation and IP-to-ASN mapping data for each connection.  The framework also enables development of network analysis plug-ins that are designed to aid in the understanding of network traffic and present results to the user in a concise, useful manner by allowing users to parse and present data of interest from multiple levels of the network stack.  Since Dshell is written entirely in Python, the entire code base can be customized to particular problems quickly and easily; from tweaking an existing decoder to extract slightly different information from existing protocols, to writing a new parser for a completely novel protocol.  Here are two scenarios where Dshell has decreased the time required to identify and respond to network forensic challenges.

1. Malware authors will frequently embed a domain name in a piece of malware for improved command and control or resiliency to security countermeasures such as IP blocking.  When the attackers have completed their objective for the day, they minimize the network activity of the malware by updating the DNS record for the hostile domain to point to a non-Internet routable IP address (ex. 127.0.0.1).  When faced with hundreds or thousands of DNS requests/responses per hour, how can I find only the domains that resolve to a non-routable IP address?

Dshell>  decode –d reservedips *.pcap

The “reservedips” module will find all of the DNS request/response pairs for domains that resolve to a non-routable IP address, and display them on a single line.  By having each result displayed on a single line, I can utilize other command line utilities like awk or grep to further filter the results. Dshell can also present the output in CSV format, which may be imported into many Security Event and Incident Management (SEIM) tools or other analytic platforms.

2. A drive-by-download attack is successful and a malicious executable is downloaded [2].  I need to find the network flow of the download of the malicious executable and extract the executable from the network traffic.
   Using the “web” module, I can inspect all the web traffic contained in the sample file.  In the example below, a request for ‘xzz1.exe’ with a successful server response is likely the malicious file.

I can then extract the executable from the network traffic by using the “rip-http” module.  The “rip-http” module will reassemble the IP/TCP/HTTP stream, identify the filename being requested, strip the HTTP headers, and write the data to disk with the appropriate filename.

There are additional modules within the Dshell framework to solve other challenges faced with network forensics.  The ability to rapidly develop and share analytical modules is a core strength of Dshell.  If you are interested in using or contributing to  Dshell, please visit the project at https://github.com/USArmyResearchLab/Dshell. 

[1] Dshell – https://github.com/USArmyResearchLab/Dshell
[2] http://malware-traffic-analysis.net/2015/01/03/index.html