Making data CIM compliant can be a daunting and confusing exercise for new Splunkers and experienced ones alike. Often the biggest misconceptions have to do with the approach rather than the exercise itself. My biggest piece of advice – start with the end in mind. (i.e. you should be familiar enough with your data to understand which data model it should be mapped to, what fields the data model requires, and what fields your data actually has).
In this exercise I’m going to do a simple walkthrough making Checkpoint firewall data (sourcetype=cp_log) CIM compliant with the Network Traffic data model.
You should already be comfortable performing extractions in props.conf and transforms.conf. If you’re not, here’s a cool cheat sheet.
Before we begin, you should do the following:
- Identify the app for our given data’s sourcetype
We get TA-check-point-app-for-splunk
- Familiarize ourselves with our app’s default/props.conf and transforms.conf
- Familiarize ourselves with the Network Traffic data model
- Consult vendor documentation to understand Checkpoint’s fields. Here is Checkpoint’s data dictionary. Not all vendors make this readily available online. Thank you Checkpoint <3. If I can’t find one online, I would consult with the device’s administrator, if I can’t make sense of it myself. But this one is pretty straight forward.
Now, let’s begin
Step 1. Map Checkpoint’s fields to Splunk CIM fields in the Network Traffic data model
|Splunk CIM Field (Network Traffic data model)||Splunk CIM required values||Checkpoint Equivalent Fields||Checkpoint possible field values||Making it CIM Compliant|
|Action||Allowed, Blocked||Action||Accept, Reject, Drop…||Looks like all this requires is a simple eval…|
But there’s a gotcha… After reviewing default/props.conf, I noticed EVAL-action had already been defined in default/props.conf. 10 Pro points for checking the defaults first!
So now, how do we define a new EVAL-action statement while keeping the vendor’s definitions? Here’s how I would do it, you can probably think of more clever ways!
Let’s test it out in SPL first
|app||N/A (Application protocol of the traffic)||service_id||https, domain-udp, microsoft-ds, ldap,….||local/props.conf:|
FIELDALIAS-service_id = service_id AS app
… |rename service_id AS app
|bytes||N/A (numerical. Total bytes)||bytes||289234,23093,…||No action required|
|dest||ip or hostname||dst||10.x.x.x, 172.x.x.x, …||local/props.conf:|
FIELDALIAS-dst = dst AS dest
Step 2. Test our changes as SPL before we jump into *.conf files
Before I jump into the .conf files, I like to test out my changes as SPL so we cut down on rework and from disappointing your client/end users when you’re constantly reloading Splunk. (Pro Tip: Download CIM Vladitator and test your search out on the scoring dashboard).
An easy visual using CIM Validator:
Step 3. Prepare our props and transforms.conf changes to be deployed.
Step 4. Create event types (if not defined by default). Event types will allow us to filter in only Checkpoint’s firewall data and filter out non-firewall data.
Reviewing the default/eventtypes.conf we see Firewall data if being filtered with the value eventtype=Network_Traffic. No changes required here.
Step 5. Create tags (if not defined). This should always be the final step because this is when the data will be imported into the data model.
Reviewing the default/tags.conf we also see the tags required for the Network Traffic data model have been defined. No changes required here either. If it wasn’t there, we would have to create a local/tags.conf with the content below.
Step 6. Deploy the Search Heads
7. Check whether our datamodel is pulling in our new sourcetype (assuming the Network_Traffic data model has been enabled)
In the next article, I’ll do a more advanced walk through and explain additional concepts and tricks. You should get to a point where you no longer need a table and be able to do all of this in your head!
Aditum’s Splunk Professional Services consultants can assist your team with best practices to optimize your Splunk deployment and get more from Splunk.
Our certified Splunk Architects and Splunk Consultants manage successful Splunk deployments, environment upgrades and scaling, dashboard, search, and report creation, and Splunk Health Checks. Aditum also has a team of accomplished Splunk Developers that focus on building Splunk apps and technical add-ons.
Contact us directly to learn more.