Cribl pipeline modification of Zeek logs

Cribl’s pipeline capabilities make it a powerful tool for processing and enriching logs. In this post, we’ll explore how to apply advanced modifications to Zeek logs in Cribl.

Each modification needs to be placed in individual Eval function blocks within your pipeline to avoid errors and ensure smooth processing.

In the scenarios below, Cribl edge is installed on the Zeek system, and a file input is used on /opt/zeek/logs/current. Output is configured for Splunk and can be used for verification, but you can also do this with Cribl’s built in data capture and preview functionality.

Why Modify Zeek Logs with Cribl?

Zeek logs are very detailed and provide insight into network traffic. However, their volume can be overwhelming, and not every field is always necessary. Advanced pipeline modifications allow you to:

• Filter logs.
• Enrich logs.
• Parsing.
• Normalization.

Key Considerations for Cribl Pipelines

Cribl’s language is not the same as full JavaScript, and some common programming constructs, like variable assignments or multi-line expressions, can trigger errors. Cribl uses functions to modify logs and expects all expressions to resolve cleanly without manual variable assignments like =. You’ll need to use Cribl’s built-in functions and chaining instead.

1. Filtering Zeek Logs by Conditions

Filtering logs based on specific conditions is essential for reducing noise. Here’s how you can filter for HTTPS traffic in conn.log by checking if the destination port is 443. Create a drop function in your pipeline and set the filter.

source == "/opt/zeek/spool/zeek-logger/conn.log" && dst_port == 443

This expression checks:

• The log type is conn.log.
• The destination port is 443.

If conditions are true, the event is dropped through the pipeline. Otherwise, it is passed.

2. Enriching Zeek Logs with Lookup Tables

Cribl does not allow direct external API enrichment using HTTP requests in all environments, so a common solution is to use lookup tables for enrichment. You can preload data like geolocation information into a file and use Cribl’s lookup function to add this information to your logs.

Here’s how you would enrich Zeek logs with geolocation data using a lookup file:

In this example:

• The dst_ip field is used to look up geolocation information from a preloaded CSV file.
• The result is stored in the geo_data field of the event.

3. Parsing

It’s nice to have your logs parsed before they even get to their destination to save compute and time. Below, I set zeek to send logs in json format and configure a parser in Cribl which is insanely easy.

vim /opt/zeek/share/zeek/site/local.zeek
#add this at the bottom
@load policy/tuning/json-logs.zeek

Add a parser function in your pipeline. Set Operation mode to extract and Type to JSON Object and you’re done.

4. Rename Fields

You’ll probably want to rename some fields to align with your data model. Here’s an example that renames zeek’s non-standard fields to some that may fit with your strategy. Add a Rename function to the pipeline and set old and new field names. Data must already be parsed.

Conclusion

Cribl’s powerful pipeline engine allows you to efficiently process and enrich Zeek logs. These advanced techniques will help you optimize log processing, reduce unnecessary data, and ensure that your logs are enriched with meaningful context before they reach your SIEM or analytics tools.