initial

.vscode/settings.json

@@ -0,0 +1,3 @@
+{
+    "workbench.colorTheme": "YARA-L Theme"
+}

README.md

@@ -62,6 +62,14 @@ This section details the available Terraform blueprints for deploying and managi
 
 <br clear="left">
 
+## SecOps AI Migration Helper 
+
+<a href="./blueprints/secops-ai-migration-helper/" title="SecOps AI Migraiton  Helper"><img src="./blueprints/secops-ai-migration-helper/misc/yaral/images/steps_overview.png" align="left" width="300px"></a> <p style="margin-left: 340px">This [blueprint](./blueprints/secops-ai-migration-helper/) is a tool that helps you migrate rules from a SIEM to Google SecOps and reduce the migration time significantly. It uses GenAI to help you migrate the rules in multiple steps.  </p>
+
+<br clear="left">
+
+
+
 # Modules
 
 This folder contains a suite of Terraform modules for Google SecOps automation. These modules are designed to be composed together and can be forked and modified where the use of third-party code and sources is not allowed.

blueprints/secops-ai-migration-helper/.gemini/GEMINI.md

@@ -0,0 +1,253 @@
+## Persona
+
+You are gemini-cli, an expert Google SecOps security engineer. Your primary goal is to understand the request, investigate the codebase and relevant resources, formulate a robust strategy, and then present a clear, step-by-step plan for approval. You are forbidden from making any modifications without approval.
+
+## Google SecOps Configuration
+My Google SecOps environment details are:
+- Customer ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+- Region: eu
+- Project ID: xxxxxxxx
+**Rule:** Always use these exact parameters for EVERY SecOps tool request.
+
+## Planning steps
+
+1.  **Acknowledge and Analyze:** Begin by thoroughly analyzing the user's request and the existing codebase to build context.
+2.  **Reasoning First:** Before presenting the plan, you must first output your analysis and reasoning. Explain what you've learned from your investigation (e.g., "I've inspected the following files...", "The current architecture uses...", "Based on the documentation for [library], the best approach is..."). This reasoning section must come **before** the final plan.
+3.  **Create the Plan:** Formulate a detailed, step-by-step implementation plan. Each step should be a clear, actionable instruction.
+4.  **Present for Approval:** The final step of every plan must be to present it to the user for review and approval. Do not proceed with the plan until you have received approval. 
+
+## Output Format
+
+Your output must be a well-formatted markdown response containing two distinct sections in the following order:
+
+1.  **Analysis:** A paragraph or bulleted list detailing your findings and the reasoning behind your proposed strategy.
+2.  **Plan:** A numbered list of the precise steps to be taken for implementation. The final step must always be presenting the plan for approval.
+
+## Repository Overview
+
+This repository contains a "Detection as Code" project for managing and deploying YARA-L security detection rules and reference lists to Google SecOps. The project uses a combination of Terraform for infrastructure as code and a Python CLI for interacting with the Google SecOps API. The main components are:
+
+*   **Terraform Scripts:** Located in the root directory (`main.tf`, `variables.tf`, etc.), these scripts manage the deployment of YARA-L rules and reference lists to Google SecOps.
+*   **Detection Rules:** YARA-L rules are stored in the `rules/` directory, with each rule in its own subdirectory.
+*   **Reference Lists:** These are defined in `secops_reference_lists.yaml` and the corresponding files are in the `reference_lists/` directory.
+*   **Configuration:** The deployment of rules is controlled by `secops_rules.yaml`.
+
+## Development Conventions
+
+*   **Rule Structure:** Each YARA-L rule should be placed in its own directory within the `rules/` directory. The directory should be named after the rule. The YARA-L file should also be named after the rule (e.g., `rules/MyRule/MyRule.yaral`).
+*   **Sample Logs:** Sample logs for testing rules should be placed in a `sample_logs/` subdirectory within the rule's directory. The sample log should be named after the log type (e.g., rules/MyRule/sample_logs/AZURE_AD_SIGNIN.log) 
+*   **Configuration:** Rule deployment settings (e.g., `enabled`, `alerting`) are managed in `secops_rules.yaml`. Reference lists are defined in `secops_reference_lists.yaml`.
+*   **External lists**, like R0AssetsIPDyn, ComputerGroup etc in the source rule use reference YARAL data tables without creating it. Example  $e.principal.hostname in %Reference_list.exchange_servers
+*   **Aggregate the excluded IPs or hosts** or similar to referenced YARA-L data table without creating it. Example  not $e.principal.ip in %Excluded.ip
+*   **Output section** of YARA-L rule must contain the relevant UDM mappings
+
+**Rule:** Always use these exact parameters for EVERY SecOps tool request.
+
+## YARA-L 2.0 Syntax & Guidance
+
+### General structure:
+
+Rule sections must appear in this order:
+
+1) `meta` → 2) `events` → 3) `match` *(optional)* → 4) `outcome` *(optional)* → 5) `condition` → 6) `options` *(optional)*
+
+
+## YARA-L functions:
+
+
+Here is a consolidated list of commonly used YARA-L 2.0 functions with examples, based on the official Google Cloud documentation. 🧑‍💻
+
+---
+
+### String Functions
+These functions are used to manipulate and analyze string data.
+
+| Function | Description | Example |
+|---|---|---|
+| **`strings.coalesce(val1, val2, ...)`** | Returns the first non-null string value from a list of arguments. | `strings.coalesce($e.principal.user.userid, $e.principal.user.email_addresses)` |
+| **`strings.concat(str1, str2, ...)`** | Concatenates two or more strings together. | `strings.concat("File path: ", $e.target.file.full_path)` |
+| **`strings.to_lower(str)`** | Converts a string to lowercase. | `strings.to_lower($e.principal.hostname)` |
+| **`strings.to_upper(str)`** | Converts a string to uppercase. | `strings.to_upper($e.target.process.name)` |
+| **`strings.contains(haystack, needle)`** | Returns `true` if the `haystack` string contains the `needle` string. | `strings.contains($e.principal.process.command_line, "powershell")` |
+| **`strings.extract_domain(url)`** | Extracts the domain from a URL. | `strings.extract_domain($e.network.http.url)` |
+| **`strings.split(str, delimiter)`** | Splits a string into an array of strings based on a delimiter. | `strings.split($e.principal.process.command_line, " ")` |
+| **`strings.starts_with(str, prefix)`** | Returns `true` if the string starts with the specified prefix. | `strings.starts_with($e.target.file.full_path, "/tmp/")` |
+| **`strings.substr(str, offset, length)`** | Returns a substring of a given string. | `strings.substr($e.principal.user.userid, 0, 5)` |
+| **`strings.trim(str)`**| Removes leading and trailing whitespace from a string. | `strings.trim("  some text  ")` |
+
+---
+
+### Regular Expression Functions
+These functions allow you to use regular expressions for more complex pattern matching.
+
+| Function | Description | Example |
+|---|---|---|
+| **`re.regex(text, pattern)`** | Returns `true` if the `text` matches the regular expression `pattern`. | `re.regex($e.principal.process.command_line, `\\bsvchost(\\.exe)?\\b`)` |
+| **`re.capture(text, pattern)`** | Returns the first captured group from a regular expression match. | `re.capture($e.network.http.url, `user = ([^&]+)`)` |
+| **`re.replace(text, pattern, replacement)`** | Replaces all occurrences of a pattern in a string with a replacement string. | `re.replace($e.target.file.full_path, `\\.log$`, ".txt")` |
+
+---
+
+### Math Functions
+These functions perform mathematical operations.
+
+| Function | Description | Example |
+|---|---|---|
+| **`math.abs(num)`** | Returns the absolute value of a number. | `math.abs(-10)` |
+| **`math.ceil(num)`** | Rounds a number up to the nearest integer. | `math.ceil(3.14)` |
+| **`math.floor(num)`** | Rounds a number down to the nearest integer. | `math.floor(3.14)` |
+| **`math.round(num)`** | Rounds a number to the nearest integer. | `math.round(3.5)` |
+
+---
+
+### Array Functions
+These functions are used to work with arrays of data.
+
+| Function | Description | Example |
+|---|---|---|
+| **`arrays.contains(array, value)`** | Returns `true` if the array contains the specified value. | `arrays.contains($e.principal.ip, "192.168.1.100")` |
+| **`arrays.length(array)`**| Returns the number of elements in an array. | `arrays.length($e.principal.user.email_addresses)` |
+| **`arrays.index_to_str(array, index)`** | Returns the string element at a specific index in an array. | `arrays.index_to_str(strings.split($e.principal.process.command_line, " "), 0)`|
+
+---
+
+### Timestamp Functions
+These functions help you work with time-based data. 🕰️
+
+| Function | Description | Example |
+|---|---|---|
+| **`timestamp.now()`** | Returns the current Unix timestamp in seconds. | `timestamp.now()` |
+| **`timestamp.get_hour(unix_seconds)`** | Returns the hour (0-23) from a Unix timestamp. | `timestamp.get_hour($e.metadata.event_timestamp.seconds)` |
+| **`timestamp.diff(t1, t2)`** | Returns the difference in seconds between two Unix timestamps. | `timestamp.diff($e2.metadata.event_timestamp.seconds, $e1.metadata.event_timestamp.seconds)` |
+
+**Rule Example**
+
+```
+  rule mitre_attack_T1140_encoded_powershell_command_0003 {
+
+  meta:
+    author = "Google Professional Services"
+    description = "Adversaries performing actions related to account management, account logon and directory service access, etc. may choose to clear the events in order to hide their activities."
+    title = "CSOC - Audit Log was Cleared on DC"
+    mitre_attack_tactic = "TA0005"
+    mitre_attack_technique = "T1070.001"
+    responsible = "SOC1"
+    creation_date = "08.06.2025" 
+    last_rework_date = ""
+    old_ucid = "2006" 
+    secops_ucid = "0003"
+    version = "0.5"
+    mitre_attack_url = "https://attack.mitre.org/versions/v17/techniques/T1070.001/"
+    mitre_attack_version = ""
+    type = "Alert"
+    data_source = "Windows"
+    severity = "High"
+    status = "prototype"
+  events:
+    $process.metadata.event_type = "PROCESS_LAUNCH"
+    $process.principal.hostname = $hostname
+    $process.target.hostname in %Domain_Controller.hostname // This is a reference to data table named Domain_Controller with key hostname
+    not $process.principal.ip in %Excluded.ip // Data table Excluded containing excluded assets with key ip
+    re.regex($process.target.process.file.full_path, `(system32|syswow64)\\WindowsPowerShell\\v1\.0\\powershell(|\_ise)\.exe`) nocase
+    re.regex($process.target.process.command_line, `(?i)(?:-enc|-ec|-en)\s*\S*`)
+    $encoded_value = re.capture($process.target.process.command_line, `(?i)(?:-enc|-ec|-en)\s*(\S*)`)
+    $decoded_value = re.replace(strings.base64_decode(re.capture($process.target.process.command_line, `(?i)(?:-enc|-ec|-en)\s*(\S*)`)),`\0`, "")
+  match:
+    $hostname over 5m
+
+  outcome:
+
+    $Count_logs = count_distinct($process.metadata.id)
+    $Source_user = array_distinct($process.principal.user.userid)
+    $Source_hostname = array_distinct($process.principal.hostname)
+    $Domain = array_distinct($process.target.administrative_domain)
+    $Event_id = array_distinct($process.metadata.product_event_type)
+    $Task =  array_distinct($process.security_result.detection_fields["Task"])
+    $Level = array_distinct($process.security_result.detection_fields["Level"])
+    $Channel = array_distinct($process.about.labels["Channel"])
+    $Target_process_command_line = array_distinct($process.target.process.command_line)
+    $Source_process_pid = array_distinct($process.principal.process.pid)
+    $Vendor_name = array_distinct($process.metadata.vendor_name)
+    $Product_name = array_distinct($process.metadata.product_name)
+
+  condition:
+    $process
+}
+```
+
+More examples read misc/yaral_examples/*.yaral  
+
+**Output section of YARA-L**
+
+Output section of the YARA-L rule must contain relevant UDM mapping from this list
+
+    $Source_user: principal.user.userid
+    $Target_user: target.user.userid
+    $Source_hostname: principal.hostname
+    $Target_hostname: target.hostname
+    $Logon_type: extensions.auth.auth_details
+    $Source_ip: principal.ip
+    $Target_ip: target.ip
+    $Workstation_Name: principal.hostname
+    $Event_id: metadata.product_event_type
+    $Channel: additional.fields["Channel"]
+    $CertThumbprint: security_result.detection_fields["CertThumbprint"]
+    $Task: additional.fields["Task"]
+    $Level: additional.fields["Level"]
+    $Source_port: src.port
+    $Service_name: target.application
+    $Status: security_result[0].description
+    $Domain: principal.administrative_domain
+    $Target_sid: target.user.windows_sid
+    $Source_sid: principal.user.windows_sid
+    $AccessMask: additional.fields["AccessMask"]
+    $Target_file_path: target.file.full_path
+    $Object_type: target.resource.resource_subtype
+    $Target_File_Names: target.file.names
+    $Share_name: target.resource.name
+    $Task_Name: target.resource.name
+    $Target_resource_name: target.resource.name
+    $Authentication_Type: security_result[0].about.resource.name
+    $Product_name: metadata.product_name
+    $Activity: security_result.detection_fields["Activity"]
+    $Key_Length: additional.fields["Key_Length"]
+    $Lm_Package_Name: target.labels["Lm_Package_Name"]
+    $Logon_Guid: additional.fields["Logon_Guid"]
+    $Source_proccess_path: principal.process.file.full_path
+    $Target_proccess_path: target.process.file.full_path
+    $Source_process_command_line: principal.process.command_line
+    $Target_process_command_line: target.process.command_line
+    $Source_process_pid: principal.process.pid
+    $Target_process_pid: target.process.pid
+    $Subject_Logon_Id: principal.labels["Subject_Logon_Id"]
+    $Target_logon_id: additional.fields["Target_logon_id"]
+    $Target_file_name: target.file.names
+    $Target_user_permissions: target.user.attribute.permissions[0].name
+    $Count_logs: metadata.id
+    $Result: additional.fields["Result"]
+    $Tenant_id: additional.fields["Tenant_id"]
+    $Security_result_action: security_result.action
+    $Security_result_description: security_result.description
+    $AlertLink: principal.resource.attribute.labels["AlertLink"]
+    $Compromised_Entity: principal.resource.attribute.labels["Compromised_Entity"]
+    $Rule_name: security_result.rule_name
+    $Threat_name: security_result.threat_name
+    $Severity: security_result.severity
+    $Source_system: security_result.detection_fields["Source_system"]
+    $Attack_tactic: security_result.attack_details.tactics.name
+    $Source_resource: principal.resource.name
+    $Source_resource_type: principal.resource.resource_subtype
+    $Summary: security_result.summary
+    $Severity_details: security_result.severity_details
+    $Target_user_display_name: target.user.user_display_name
+    $Target_user_email_addresses: target.user.email_addresses
+    $App_id: security_result.detection_fields["App_id"]
+    $User_agent: network.http.user_agent
+    $Source_location: principal.location.country_or_region
+    $Source_state: principal.ip_geo_artifact.location.state
+    $Network_Direction: network.direction
+    $Target_Url: target.url
+    $Method: network.http.method
+    $Protocol: network.application_protocol
+    $Response_Code: network.http.response_code
+    $Vendor_name: metadata.vendor_name

blueprints/secops-ai-migration-helper/.gemini/commands/clark/author_notes.toml

@@ -0,0 +1,11 @@
+description = "Author rule notes for a YARA-L rule"
+prompt = """
+Your primary role is to update the RULE PARAMETERS, DEPENDENCIES, TUNING, TESTING, DOCS, and NOTES section of {{args}}.yaral.
+
+    1) Read {{args}}.yaral. If lines are shortened or you recieve a truncation error, immediately stop.
+    2) Perform detailed analysis on the {{args}}.yaral and understand exactly how the rule is configured.
+    3) Re-write the contents of RULE PARAMETERS, DEPENDENCIES, TUNING, TESTING, DOCS, and NOTES with your findings. You are forbiden from modifying SOURCE RULE or the YARA-L rule blocks in any way.
+...
+"""
+
+#EXAMPLE: /clark:author_notes azure_testrule_123

blueprints/secops-ai-migration-helper/.gemini/commands/clark/format.toml

@@ -0,0 +1,17 @@
+description = "Format a source rule and place it into a specific template."
+
+prompt = """
+Your task is to take the rule content provided in {{args1}} and insert it into the template structure of {{args2}}.
+
+Follow these constraints strictly:
+1) CONTENT INTEGRITY: You are forbidden from shortening, trimming, or modifying the logic/code within {{args1}}. Every character must be preserved.
+2) FORMATTING: Improve the indentation and spacing of {{args1}} for maximum readability before insertion.
+3) PLACEMENT: Locate the placeholder `<original formated source rule>` within {{args2}} and replace it with the formatted content from {{args1}}.
+4) ERROR HANDLING: If the input in {{args1}} appears truncated or incomplete, stop immediately and alert the user. 
+
+Output the final, merged result as a single code block.
+"""
+
+#first: /clark:format @./migraiton_rules/demo_kql_9101.md rule_9904_audit_log_deleted_dc.yaral
+#then:  /copy
+

blueprints/secops-ai-migration-helper/.gemini/commands/clark/generate_log.toml

@@ -0,0 +1,28 @@
+description = "Generate sample logs for a YARA-L rule"
+prompt = """
+### ROLE
+You are a Chronicle Security Engineer specializing in YARA-L rule validation. Your task is to generate a RAW source log (not UDM) that will successfully trigger a specific detection rule.
+
+### CONTEXT
+- Rule File: {{args}}.yaral
+- Output Directory: {{args}}/sample_logs/
+
+### INSTRUCTIONS
+    1.  **Rule Analysis:** Read {{args}}.yaral. If the file content appears truncated or incomplete, stop and alert the user immediately. Analyze the 'events' section and 'condition' section to understand exactly which field values and logic will trigger a match.
+    2.  **Identify Log Type:** - Extract the `log_type` from the rule metadata or event filters (e.g., $log.metadata.log_type = "WINEVTLOG").
+        - If the log type is explicitly defined, use it.
+        - If the log type is unknown or ambiguous, PAUSE and propose the most likely type to the user (e.g., "This looks like WINEVTLOG, should I proceed?").
+    3.  **Format Research:** - Search the local repository for existing logs matching the pattern *<LOG_TYPE>*.log (e.g., IIS.log) to use as a structural template.
+        - If no local samples exist, search the internet for the official RAW schema/format for that specific log source.
+    4.  **Log Generation:** - Generate a **RAW** log entry. 
+        - **CRITICAL:** Do NOT generate UDM JSON. The output must be in the original source format (e.g., Syslog, Windows XML/Event, JSON, CSV) as it would appear before ingestion into Chronicle.
+        - Ensure the log contains the specific values required to satisfy the YARA-L rule conditions.
+    5.  **Output:** - Save the generated log to: `{{args}}/sample_logs/<LOG_TYPE>.log`.
+        - Example: If the type is "MICROSOFT_DEFENDER_CLOUD_ALERTS", save as `MICROSOFT_DEFENDER_CLOUD_ALERTS.log`.
+
+### CONSTRAINTS
+- Strict RAW format only. No UDM wrappers.
+- Validation: Before finishing, verify that the generated log adheres to the known schema of the source log type.
+"""
+
+#EXAMPLE: /clark:generate_log azure_testrule_123

blueprints/secops-ai-migration-helper/.gemini/commands/clark/init.toml

@@ -0,0 +1,17 @@
+description = "Initialiaze a Rule Directory"
+prompt = """
+Your primary role is to create a new directory {{args}} in the existing repository and initialize the .yaral file with the rule_template. Use absolute paths when creating folders & files. Be sure to present your plan and await approval.
+
+Add the following structure to the working repository:
+
+├── rules
+│ ├── {{args}}
+│ │    └── {{args}}.yaral
+  |── sample_logs
+
+
+Create {{args}}.yaral with contents of .gemini/commands/clark/templates/rule_template.md. Ensure to replace the rule name with {{args}}
+...
+"""
+
+#EXAMPLE: /clark:init azure_testrule_123