How To Identify The Available Data
Last updated
Last updated
When working with the Elastic Stack, understanding the available fields and values is crucial for constructing effective KQL queries. Below are two approaches to identifying fields and values: leveraging KQL's free text search and consulting Elastic's documentation.
The feature in Kibana is a powerful tool for exploring data and identifying available fields and values. Here’s how you can use it:
Step-by-Step Process
Access the Discover Feature:
Navigate to the Discover tab in Kibana.
Select the appropriate index pattern (e.g., logs from Winlogbeat or Filebeat).
Perform Free Text Searches:
Use KQL's free text search to explore specific terms or patterns in your data.
Example: Search for "4625" to identify records related to Windows event code 4625 (failed login attempts).
Inspect Returned Records:
Expand the returned records to view the fields associated with the data.
Example:
event.code:4625 ().
winlog.event_id:4625 ().
@timestamp (timestamp extracted from the original event).
Identify Relevant Fields:
For disabled accounts, search for "0xC0000072" to find records where the SubStatus indicates a disabled account.
Example: Notice the field winlog.event_data.SubStatus in the returned records.
Understand Field Context:
Learn which fields are part of the Elastic Common Schema (ECS) and which are specific to tools like Winlogbeat or Filebeat.
Prefer ECS fields (event.code) for consistency across the organization.
Example Query Construction
Using the insights gained from the Discover feature, you can construct precise KQL queries:
This query identifies failed login attempts against disabled accounts between March 3rd and March 6th, 2023.
Elastic provides comprehensive documentation that helps you understand the structure of fields and their usage. Below are key resources to consult:
Key Resources
Elastic Common Schema (ECS):
Standardizes field names and types for consistent querying across the Elastic Stack.
ECS Event Fields:
Provides details about event-related fields such as event.code, event.action, and event.created.
Winlogbeat Fields:
Lists fields specific to Windows event logs ingested by Winlogbeat.
Winlogbeat ECS Fields:
Maps Winlogbeat fields to the ECS for compatibility.
Winlogbeat Security Module Fields:
Focuses on security-related fields, such as those for Windows event logs.
Filebeat Fields:
Covers fields for logs ingested by Filebeat.
Filebeat ECS Fields:
Maps Filebeat fields to the ECS.
Why Use ECS Fields?
Consistency: ECS fields ensure uniformity across all data sources, making queries easier to write and maintain.
Interoperability: ECS fields are compatible with other Elastic Stack components, enabling seamless integration.
Scalability: As organizations grow, ECS fields simplify the management of large datasets.
By combining free text searches in Kibana's Discover feature with Elastic's documentation, you can:
Quickly identify relevant fields and values in your data.
Understand the context and purpose of each field.
Construct precise and efficient KQL queries.
Search for Failed Login Attempts:
Use free text search in Discover: "4625".
Identify fields like event.code:4625 and @timestamp.
Focus on Disabled Accounts:
Search for "0xC0000072".
Identify the field winlog.event_data.SubStatus.
Construct the Query:
Validate with Documentation:
Confirm the meaning of event.code and winlog.event_data.SubStatus using Elastic's documentation.
Identifying available fields and values is a critical step in effectively using the Elastic Stack for security operations. By leveraging KQL's free text search in Kibana's Discover feature and consulting Elastic's documentation, SOC analysts can:
Gain a deep understanding of the data structure.
Construct precise and actionable KQL queries.
Detect and respond to security incidents more efficiently.
