Supercharge Your Cloudflare Logs to Mitigate Risk

Around 20% of the web runs through Cloudflare — an expansive, connectivity cloud platform offering services such as cybersecurity, data compliance, and digital modernization. Organizations leverage Cloudflare’s comprehensive data insights to fuel their analytics and SIEM platforms, which results in the blocking of billions of cyber threats daily. 

To further improve your security posture, Edge Delta offers a customized Cloudflare Pack for our Security Data Pipelines. The pack intelligently enhances Cloudflare logs to improve analysis and reduce data noise. 

Edge Delta Security Data Pipelines help you standardize, enrich, and stream data to security platforms and archives, while delivering a clear view into how data streams are configured, all in real-time to outpace adversaries.

‍

With efficiency in mind, the Cloudflare Pack automatically parses, suppresses, and transforms your logs with a pre-built collection of custom processors. Once finished, the Cloudflare logs can be routed — via Edge Delta’s Security Data Pipelines — to any downstream destination for deeper investigation or to simple storage for later analysis or compliance purposes. 

In this post, we’ll talk about installing the Cloudflare Pack, how it processes your data, and what you can expect once things are up and running. 

How Does the Cloudflare Pack Work? 

‍

‍

The Cloudflare Pack includes three different processors designed to improve the usability of the data being generated by your Cloudflare logs. The overall five-step sequential processing journey — which begins with a Source node and ends with a Processed compound node — consolidates log data for more efficient analysis. 

Processing Pathway: JSON Body Parsing

To begin, the Cloudflare logs enter an Edge Delta pipeline through the Source node. Once the logs are flowing into the Cloudflare Pack, its first processing step is a Parse JSON body node, which is an OpenTelemetry Transformation Language (OTTL) node. 

The main function of this node is to decode the message body as UTF-8, parse it into JSON, and store the result under the attributes ["parsed_body"] attribute. The transformation from raw JSON data into a more structured format increases the ability to analyze and react to log content efficiently.

- name: Parse JSON body
 type: ottl_transform
 statements: set(attributes["parsed_body"], ParseJSON(Decode(body, "utf-8")))

‍

Processing Pathway: Suppress Processor

The suppress processor node restricts the visibility of repetitive log entries by inspecting specific fields (ClientRequestHost, ClientASN, and EdgeResponseStatus, in this case) and allowing only one log with the same property values to be listed within every 30-second interval.

- name: Suppress Processor
 type: suppress
 interval: 30s
 key_field_paths:
 - attributes["parsed_body"]["ClientRequestHost"]
 - attributes["parsed_body"]["ClientASN"]
 - attributes["parsed_body"]["EdgeResponseStatus"]
 number_to_allow: 1

‍

Processing Pathway: Reserialize Body

Next, logs are then restructured by the Reserialize body node, which is the last OTTL Transform node.

This node performs the following functions:

• delete_matching_keys: erases all keys from the attributes ["parsed_body"] map that match the regex pattern ClientRequest.
  
  
• edx_delete_empty_values: removes keys with values that are empty from the attributes["parsed_body"].
  
  
• set: defines the value of the body field. The value is obtained by encoding attributes ["parsed_body"] into a byte array using the EDXEncode custom function with "utf-8" encoding.

• delete_key: discards the key parsed_body from the attributes map, which assists with disposing of unnecessary data.

- name: Reserialize body
 type: ottl_transform
 statements: |-
   delete_matching_keys(attributes["parsed_body"], "ClientRequest")
   edx_delete_empty_values(attributes["parsed_body"], [], ["", "unknown"], ["deleteNull", "deleteZero"])
   set(body, EDXEncode(attributes["parsed_body"], "utf-8", true))
   delete_key(attributes, "parsed_body")

Processing Pathway: Processed Output

The last step is making the newly-processed logs available for routing to one or more destinations of your choice via the Processed compound output node. From here, Cloudflare logs can be shipped to cold storage or to a SIEM, for example, for further processing and analysis.

Cloudflare Pack in Practice

To get started with the Cloudflare Pack, you’ll first need an existing pipeline within Edge Delta. If you don’t have an Edge Delta pipeline created yet, you can navigate to Pipelines, and then select New Fleet. From there, you can select whether you want your pipeline to be configured with an Edge Fleet or Cloud Fleet — which correspond with the environments in which they’re hosted — and then follow the remaining setup steps.

Once you have a pipeline up and running, navigate to the top-level Pipelines menu, and select “Knowledge,” and then “Packs.” From there, scroll to the Cloudflare Pack and click “Add pack.” This will send the Cloudflare Pack to your Pack library, which you can access from the Pipelines menu under “Packs.” Then head back to your main Pipelines dashboard, select the pipeline where you’d like to add the Cloudflare Pack, and enter “Edit Mode” to configure the pipeline with the Visual Pipelines builder. Once in edit mode, navigate to “Add Processor,” then “Packs,” and find the Cloudflare Pack. 

You’ll then have the option to rename the pack and choose some sample logs to test. When you’re ready, hit “Save Changes,” and the Cloudflare Pack will be added to your pipeline. From there, use the Visual Pipelines builder to simply drag and drop the connection from the initial Cloudflare logs source into the pack. 

Next, Edge Delta Security Pipelines allow you to ship your processed Cloudflare data to a destination of your choosing. For example, processed Cloudflare logs can be routed to your current SIEM vendor — whether that’s Splunk, Microsoft Sentinel, CrowdStrike Falcon, or others. You can also route your data into Edge Delta, and take advantage of our log analysis capabilities. Additionally, you can send a full copy of your raw log data to archival storage destinations such as S3 for compliance or later analysis. 

Getting Started 

Ready to see Edge Delta’s Cloudflare Pack in action? Check out our packs list and add the Cloudflare Pack to any running pipeline. New to Edge Delta? Visit our Playground to get a feel for how we can help you change the way you manage your security data. 

‍