Using Netskope Log Streaming, the solution can add/remove certain fields being ingested into your hyperscaler bucket to drive cost savings. Additionally Microsoft Sentinel Optimization Workbook also helps to identify areas where you can optimize cost reductions which further helps in Analyzing which field is taking more space and how much we can save by eliminating the particular field. Following the below architecture you can enable cost savings on your Microsoft Sentinel spend.
In today’s scenario why reducing cost on your logs becomes an important part:
- Global data is exploding: This exponential growth means raw telemetry volumes keep rising year-over-year
- SIEM/Sentinel cost is driven by ingestion & retention: every extra GB you send and retain directly increases your bill.
- A large fraction of log volume provides minimal security value: Industry assessments and SIEM optimization projects routinely find that ~60–70% of ingested log volume has little to no security value, and that a small set of sources often account for most of ingestion cost (e.g., “top 5 sources ≈ 80% of ingestion”)
Prerequisites:
- Netskope Log Streaming
- Azure Sentinel workspace with Microsoft Sentinel Optimization Workbook installed from the content hub
How to remove fields in Netskope Tenant:
Go to your Netskope tenant > Settings > Tools > Click on Log Streaming.
Configure the Netskope Log streaming following below guide for your Microsoft Azure Environment
https://docs.netskope.com/en/stream-logs-to-azure-blob/
Now how to remove fields or Add fields from the Netskope Log Streaming. Click on your particular configuration for the Web Transaction Logs. Click on Transaction Events it will list down all the fields present for the particular configuration.
Now we have all the fields which we can select/deselect on the basis of our requirement. This will help us to do the cost saving and reduce unnecessary fields. After the change in the configuration click on save. The configuration change can take up to 60 minutes to reflect the same on the Microsoft Sentinel.
How to save costs in Microsoft Sentinel:
Please use the below guide to deploy the template and the Web transaction events connector
After completing the connector configuration we will start seeing the data as seen in the image below. We will be receiving the required fields only from the Netskope Log Streaming as we already removed them from the configuration.
To further save costs within Microsoft Sentinel we will be utilizing the Microsoft Sentinel Optimization Workbook and installing the particular solution from the Microsoft Sentinel Content hub.
After installing the solution you can select your particular workbook for which you want to do the analysis.
Go to the Cost and Ingestion Optimization and you will be able to find which fields are consuming what amount of space to further help us reduce any costs.
Additionally you can add the below field in your custom template to remove the field with hyphen. Hyphen represents the empty spaces in your data. You can add the check for all the fields by doing the changes with the code below. This addition in your template can lead to saving you up to 5% on your total cost. Note: using the code below can lead to a slow down in ingestion.
extend y'cs-content-type'] = iff("'cs-content-type'] == '-','',l'cs-content-type']) | extend y'cs-referer'] = iff("'cs-referer'] == '-','',l'cs-referer']) | extend t'x-cs-app-from-user'] = iff(='x-cs-app-from-user'] == '-', '', t'x-cs-app-from-user'])
