Detection Engineering 4: Documenting and Expanding Investigations

As you analyze and interpret detection results, effective documentation becomes essential. Documentation provides a clear record of what has been observed, what steps have been taken, and where the investigation is headed. In this post, we’ll discuss how to document findings, organize investigation steps, and expand the investigation by connecting related events.

Importance of Documentation in Detection

Documentation is more than just record-keeping. It serves as a valuable reference for the entire team, ensuring that analysts stay aligned and do not repeat work. A well-documented investigation clarifies the reasoning behind each decision and highlights the context around detected events. This clarity is essential for maintaining consistency, especially in cases where new analysts may need to step in mid-investigation.

Key Documentation Techniques

When documenting, aim to capture the investigation’s core steps without overwhelming detail. The goal is to make your documentation actionable and accessible, not overly complex. Three primary methods help organize your notes:

1. Red and Blue Timelines
   Red timelines focus on documenting adversarial actions observed during the investigation. They detail each malicious event chronologically, which helps in understanding the adversary’s tactics, techniques, and procedures. Blue timelines track defensive actions taken, capturing how the team responded, which analytics were adjusted, and which hypotheses were tested. Keeping these timelines updated ensures that the entire investigative process is transparent.
2. Activity Graphs
   An activity graph visually connects related events, highlighting connections between hosts, users, and processes. By mapping out suspicious actions, you can spot patterns or gaps in knowledge that text alone might not reveal. Activity graphs help clarify lateral movement within the network and can be especially useful for understanding multi-stage attacks.
3. Event Summaries
   Summarize each significant event with details like process name, command-line arguments, user ID, timestamp, and associated IP addresses. This concise format keeps key information available at a glance, aiding quick decision-making during future analysis.

Expanding the Investigation

With initial findings documented, it is often necessary to expand the scope of the investigation. Connecting related events adds context, clarifying the scale and depth of the potential threat.

1. Identify Causal Links
   Look for causal relationships between events. For example, if a suspicious PowerShell command initiates a network connection, investigate the processes or systems involved in that connection. Following these links helps build a more complete picture of the adversary’s behavior and can reveal additional indicators.
2. Investigate Temporal Connections
   Events occurring within close timeframes often relate to each other, particularly in the case of a coordinated attack. Grouping events by time, especially those within a few seconds or minutes of each other, can help identify related actions, such as privilege escalation followed by data exfiltration.
3. Correlate Across Multiple Hosts
   Many adversaries use lateral movement to reach valuable systems. If an event appears suspicious on one host, investigate whether similar activity is present across other hosts in the network. Expanding across hosts helps ensure you’re capturing the full extent of the activity and not missing connected actions on other parts of the network.

Avoiding Common Pitfalls in Documentation

When documenting, it is easy to fall into common traps. Avoid excessive detail that makes documentation too lengthy or dense for other team members to follow. Keep records concise and focus on clarity. Another common issue is inconsistency. Define a standard documentation format and approach, which could include templates for event summaries or specific fields for timelines, to keep entries uniform.

Building a Collaborative Documentation Process

For teams, documentation should be a collaborative effort. Schedule regular check-ins to discuss findings and update documentation with the latest information. Team members should openly share insights from their own investigations to prevent overlap and ensure that each new discovery contributes to the overall understanding of the incident. Collaboration ensures that documentation remains current, accurate, and useful to everyone involved in the investigation.

Conclusion

Documentation in detection engineering is more than just a record. It is a strategic tool that guides the investigation, enables collaboration, and forms a basis for continuous improvement. By maintaining clear, concise records of findings, timelines, and event connections, you ensure that your team has a comprehensive understanding of each investigation. In the final post, we’ll discuss how to take actionable steps from these findings, moving from detection and analysis to response and continuous refinement.