Using Hashing to Improve Volatile Memory Forensic Analysis

I wanted take this opportunity to thank everyone who attended our presentation, "Using Hashing to Improve Volatile Memory Forensic Analysis", at the American Academy of Forensic Sciences 60th Annual Meeting on February 21, 2008 Washington, D.C.. This was joint work with my colleague Blake Matheny and Doug White from the National Institute of Standards and Technology, NIST. The American Academy of Forensic Sciences does a lot of great work furthering the application of science and law. I'm glad to see their renewed interest in the area of digital forensic sciences. In particular, I was encouraged that our peers in the forensic sciences community were able recognize the importance of volatile memory analysis to the future of digital investigations. I believe this is an extremely important step!

I also wanted to take this opportunity to thank our friends at NIST, especially Doug White and John Tebbutt, for all their help with this research. With their help, we are creating a standard reference data set to support the needs of the growing community of volatile memory analysts. A special thanks also goes to Jide for all his help and thoughtful discussions!

The slides from the AAFS presentation are now available.

It's about time...

As we mentioned in a previous blog post and in our presentations, we have recently been focusing our attention on the Reconstruction Phase of the digital investigation process. During the Reconstruction Phase, a digital investigator will attempt to organize the analysis results to help develop a theory about what happened during an incident. One method investigators have traditionally used to organize file system analysis is to elucidate the temporal relationships between digital artifacts. This technique is referred to as temporal reconstruction.

Dan Farmer demonstrated the usefulness of temporal reconstruction of filesystem events with the 'mactime' program. In fact, he called mactime "the most potentially valuable forensic tool in your digital detective toolkit" (Farmer, 2000). Rob Lee eventually extended this work with the 'mac_daddy' program and finally these tools were combined by Brian Carrier into the SleuthKit's versions of 'mactime' and 'mac-robber'. Recently, Florian Buchholz has also done a lot of interesting research exploring the characteristics of these temporal relationships and demonstrating the value of being able to combine disparate data sources, Zeitline.

In this blog entry, we will demonstrate how digital artifacts extracted from volatile memory analysis can be combined with artifacts from file system analysis to help reconstruct a more complete understanding of the digital crime scene. In fact, volatile memory analysis often provides the context necessary to link seemingly disparate events and their related artifacts, in ways that are not possible with typical live response tools. Using these temporal relationships, we have also been able to develop "temporal incident patterns" allowing us to quickly discern tools and techniques that may have been involved in an incident based on their "temporal footprints". We have also found that the ability to visualize these temporal relationships is invaluable for both presentation and knowledge discovery.

The following images will help demonstrate how a digital investigator can use both volatile memory analysis and visualization to improve temporal reconstructions of the digital crime scene. The file system events used to populate the time line in the images were generated using the Sleuthkit's 'mactime' program. These instantaneous events are represented in the image with blue dots and relate to the time attributes (LastWriteTime, LastAccessTime, CreationTime, etc) associated with files and directories in a file system, MACtimes. The following image is a visual time line representation of a filtered set of file system events.



In the next image, we augment the time line with events extracted using live response techniques, one type of run time state analysis. Live response allows us to extract events about objects that were active on the system when acquisition was performed. This could be extracted with your typical live response toolkit (RAPIER, WFT,etc). The red dots in this image are used to denote when a process was created. Unlike the file system events, this is a duration event since it has both a start time and an end time. In this image, the end time relates to when the live response was performed, represented by the gray dot. The green dot, another instantaneous event, represent when a process binds a specific port address to its socket. This augmented time line can be seen in the following image.






The final image demonstrates how using volatile memory (RAM) analysis to perform run time state analysis can be used to further augment our temporal reconstruction of the digital crime scene. In this case, the temporal events were extracted from volatile memory using Volatility. In contrast to the previous image, we are not only able to augment the time line with those objects that where active when live response was performed but also with objects that may have been relinquished by the operating system. The blue dots in the image once again represent file system events. The red dots represent process creation events, except this time a process duration event ends with memory acquisition or when a process exited. The green dots relate to binding sockets and the grey dot relates when memory acquisition was performed. The final augmented time line can be seen in the following image.





The purpose of this blog entry was to demonstrate the usefulness of being able to augment temporal reconstruction with both visualization and volatile memory analysis. In the final image, we can easily see how including volatile memory analysis and visualization allow us exploit temporal locality and volatile context to develop theories about the incident. We have found this to be invaluable during the reconstruction phase of the digital investigations process.


Despite the usefulness of these techniques, it is important to keep in mind that timestamps can be manipulated by a determined adversary and recently tools, such as timestomp, have been created to frustrate temporal reconstructions of filesystems. Recent research has also discussed important considerations for the digital investigator as they work with temporal data. Temporal reconstruction is not the panacea but a digital investigator should combine many types of analysis techniques during their digital investigations.

More details to follow ...

Commercial Support for Volatility!

While at DoD Cyber Crime last week, numerous members of the Volatility community made me aware of a company attempting to spread misinformation about Volatility. It was broadly suggested that there was no support being offered for Volatility. The goal behind the open development of Volatility was to bring together systems researchers who believed in bettering the state of the digital forensics community. One way that we have been able to continue this open development is by offering customizations and support.

Volatile Systems, LLC has been providing commercial support and maintenance for Volatility (and our other products) for the past 8 months. In fact, one of the main reasons Volatile Systems, LLC formed was to support the forensics needs of our users who required commercial support contracts. The added benefit of our commercial support contracts is that you are not only getting guaranteed support and access to our team of unparalleled memory analysts, but you are also actively contributing back to the volatile memory analysis community by allowing us to continue the open development of Volatility.

At this point, we also decided to extend a new offer to those who may be considering spending the thousands of dollars to purchase one of those other commercial products, as they become available. If you are considering investing in one of those products because you think it provides extraction functionality not currently supported in Volatility, contact us and let us know! In most cases, we would be more than willing to use those funds to build you custom modules providing the same capabilities you desire but tailored to your exact needs. In addition, we would provide you access to the source code, training on how to use the modules, and share information on how they were developed. As we have learned from our experience performing volatile memory analysis, the most valuable thing is often not the tool but the experience and training of the analyst. Knowledge is power!

On a tangential note, it was encouraging to get all the positive feedback about Volatility at the conference. We are committed to this growing open community of volatile memory analysts and we are highly appreciative of their support. I also wanted to extend a special thanks to the Volatility community for keeping me updated on this evolving issue. Little do they know, the Order of Volatility is everywhere!

They are playing you for a fool!

I have previously talked about this issue before, but based on a number of conversations I had last week at Cyber Crime, I felt it was worth bringing up again. Every time this issue comes up, it reminds me of one of my favorite blog posts, which talks about the ethical conflict in the rootkit community. I also recently came across this blog post from my former advisor, Spaf, which I found relevant as well.

One of the main reason why I dedicated myself to researching volatile memory analysis was the fact that the offensive communities and projects were flourishing. As a result, the sophistication of methods and accessibility to knowledge was continuing to grow unabated in the offensive community. At the time, I felt we drastically needed to have a similar revolution in the defensive community. A way of bringing together strong systems researchers who were interested in securing our infrastructure.

Based on the research we were doing at the time, I knew that volatile memory analysis would be an important component of securing those systems and had the potential to disrupt much of the offensive research being performed. As a result, members of our project have spent a great deal of time over the last couple of years writing research papers, giving talks, educating, and developing an open source architecture, in order to inspire research and increase the communal knowledge of the investigative community. In the process, we have had over 20 different contributors from multiple countries across the world. This includes contributions from numerous law enforcement and forensic agencies. In fact, I have been contacted by many universities that are now, or soon will be, using Volatility in their digital forensic courses.

It seems that the work being done in the live memory analysis community has also been successful at getting the attention of the offensive community (esp. rootkit). In fact, they have attempted many times in the last couple of years to disrupt the communal aspects of these projects. They began by trying to convince people that volatile memory analysis wouldn't work and was ineffective. Their methods changed last year, when they began trying to deceptively patent techniques that members of the volatile memory analysis community had already presented at conferences. Recently, I have learned that they are now trying to use their companies as real life Trojan horses to undermine and divide the open nature of the volatile memory analysis community. They are now trying to sell the techniques they had previously argued were ineffective. Once again, trying to capitalize on the problem they created.

Let's consider the following analogy:

Sadly, your child has been struggling with drug addiction for a number of years. He was recently busted by the police and mandated by the court to attend drug rehabilitation. Your child's drug dealer was a notorious individual by the name of B.S. Hary. B.S. Hary has never hidden the fact that he sells drugs and, in fact, even wrote a book and teaches classes about advanced drug dealing techniques. Often flaunting his drug dealing in the face of local law enforcement, who are overburdened dealing with the myriad of drug dealing pupils B.S has released on the streets. As a result, B.S. Hary's drugs and drug dealing techniques account for the majority of the drug problem currently faced by your community.

Recently, B.S. became concerned about the popularity of drug rehabilitation in pop culture. On the one hand, he realized that rehabilitation could be bad for business, but he also figured there was a lot money to be made in rehabilitation. As a result, he decided that he could not sit idly by and watch his drug business be swept out from under him, so he formulated a plan. He decided to capitalize on the rehabilitation market while undermining its effectiveness by starting his own rehabilitation company called Addiction Responder. B.S. Hary even had the brazenness to open Addiction Responder right next door to his crack house.

B.S. Hary is hoping to play the community for a fool!

As a parent, would you be willing to send your child to the Addiction Responder clinic? Knowing that Addiction Responder is run by a notorious drug dealer, do you think the court would be willing to trust a report that acknowledges your child's successful completion of its drug rehabilitation program? Knowing that the owner of Addiction Responder has a crack house right next door to the clinic, do you think the court would have faith in the fidelity of Addiction Responder's rehabilitation capabilities? Knowing that B.S. sells manufactured drugs out of the crack house right next door, would you be willing to ingest his magic rehabilitation pills? Knowing that the money you give to Addiction Responder for rehabilitation will be used to further his drug cartel, will you be willing to help fund the problem that is tormenting both your family and your community?

Your child's current drug dealer wants to perform his rehab.
You said, no, no, no!

On that note, it seems utterly absurd to me that anyone would consider buying volatile memory (RAM) forensics tools from an organization that freely admits to having armed and which continues to arm the enemy with "technology being used to evade forensics and response today." As a taxpayer, I'm not happy to see that all the government funding they have received for research and development has contributed to the majority of the rootkits currently found on the Internet today. As a person involved in forensic investigations, I would not want to be the person responsible for presenting those tools or results in court.

Defense Attorney: Is it true that developers of this "investigation" tool are responsible for the techniques found in the majority of rootkits found on the Internet today?
Forensic Examiner: Yes.
Defense Attorney: Is it true that the makers of this tool also sell "undetectable" software agents that allows people to secretly spy on a person/companies computers (similar to malware or spyware)?
Forensic Examiner: Yes.
Defense Attorney: Do the developers of this software also develop tools to exploit software, cheat at online games, and build rootkits?
Forensic Examiner: Yes.


One of the most important things that I have learned from the forensics and digital investigation communities is that the integrity and trust that can be placed in the collected evidence is often the most important standard. I have been confronted with many situations where we have had to forgo certain types of evidence, because it had the potential to compromise the integrity of investigation and/or case. How would you like to walk into court knowing that the evidence you collected and analyzed will immediately be called into question and, as a result, ruin the case? What happens when the malware you are investigating, as part of an incident, was written by the same people who wrote your forensic tool? Can you trust that they weren't involved?

The question is, are you willing to listen to B.S. and be played the fool?

And you wonder why I'm angry.....

PyFlag Using the Volatility Framework!

It was only a matter of time....

In case you might have missed it during the holidays, the latest version of PyFlag now leverages the Volatility Framework to add volatile memory analysis (RAM Forensics) to it's outstanding list of capabilities. As a result, making PyFlag the first and only tool publically available that allows the digital investigator to correlate disk images, log files, network traffic, and RAM captures all within an intuitive interface. While the current functionality is still preliminary, just imagine the possibilities!

Since PyFlag loads memory images through its standard IO source interface, it is also now possible to store your memory images using the EWF format, commonly used in commercial tools. Once the memory image is uploaded to PyFlag, information can either be accessed through a browseable /proc interface or through the Stats view. Michael Cohen and his team have provided a tutorial and image to get you started.






As I mentioned in a previous post, a special thanks to Europol for bringing our teams together through the High Tech Crime Expert Meeting. I also want to thank Michael Cohen for the great work he has done with PyFlag and his contributions to Volatility! Stay tuned for further exciting collaborations and future Volatility releases in 2008!

The Hague

I've just returned from a wonderful trip to the Netherlands. I want to thank the extremely nice people from Europol who invited me to brief their high tech crime experts about the latest advancements in volatile memory analysis. It's encouraging to see that the major police organizations of the European Union appreciate both the importance of and potential for volatile memory analysis. They also understand the role it will play in the future of digital investigations.

During this trip, I also had the pleasure of meeting a number of people whom I've exchanged emails with over the years. For many of them, it was great to finally be able to associate a face with a name. In fact, this was definitely one of the major highlights of my trip. In particular, I look forward to future collaborations with both Jon Evans and Michael Cohen, who both gave outstanding presentations.

While in the Netherlands, I also had the opportunity to meet up with Robert Jan Mora, Bas Kloet, and Joachim Metz from Hoffmann Investigations. Hoffmann Investigations is lucky to have such a bright group of forensics researchers. Not to mention, they are also really good at reading a menu for those not brave enough for the mystery meal! Hopefully, they will make it back to the US for DFRWS 2008.

2008 Cyber Crime Conference

The agenda for the 2008 DoD Cyber Crime Conference has been posted. I'll be giving a talk during the Research and Development Track at 0830 January 16, 2008. In this talk I will be discussing the latest advancements in the area of Volatile Memory Analysis and how they affect the way we perform digital investigations.

Title:
Advanced Volatile Memory Analysis

Abstract:
This session will focus on advanced techniques being used in volatile memory analysis (VMA) and our experiences while performing VMA. We will also discuss a number of open source tools and resources we have made available to the digital investigation community. The session will also explore how we are using VMA to perform automated malware analysis. Finally, we will demonstrate how we are combining VMA with file system analysis to help reconstruct and visualize the digital crime scene.