Austin LimaCharlie Workshop

• Austin LimaCharlie Workshop
  • Resource List
    • Lab Resources
    • Lab Downloads
  • Setting Up LimaCharlie
    • Lab 1: Configuring LimaCharlie and Ingesting Logs
    • Lab 2: Configuring Add-ons
    • Lab 3: Ingesting Lookups
    • Lab 4: Configuring Outputs
  • Writing Detection and Response Rules from Scratch
    • Lab 5: Writing a Detection and Response Rule for Known Malicious IPs
    • Lab 6: Writing a Detection and Response Rule for Known Malicious Hashes

Important

This guide will provide you with a step-by-step of all the commands we will use throughout this workshop. Please reference it as we move forward. If you have questions, feel free to ask any LimaCharlie team member.

Tip

Terms Used in this Workshop

• Sensor - A tool that collects and acts on data across endpoints, networks, and cloud environments to provide full visibility and security
  • Endpoint Agent - A lightweight sensor installed on an endpoint that collects security telemetry and enables real-time monitoring and response.
  • Adapter - Sensor that provides flexible data ingestion mechanisms for both on-premise and cloud environments
  • External Adapter - A LimaCharlie adapter that lives on a system outside of the LimaCharlie cloud platform (e.g., on a syslog server or other on-premise system)
• Extension - A handy add-on that boosts the platform’s features by hooking into third-party tools and custom setups, and once created, any subscribing organization can use them across their environments.
• Organization - A tenant within LimaCharlie
• OID - The organizations unique ID
• SID - The sensor's unique ID

Resource List

Lab Resources

• LimaCharlie
• LimaCharlie Docs
• SOCRadar
• LCQL Cheat Sheet
• LimaCharlie Community Forum
  • Please feek free to ask questions on the forum after the workshop if you encounter any issues. Mutiple people from LimaCharlie monitor and will respond to any help needed!

Lab Downloads

• Workshop Files
  • Extract the archive after downloading. You should end up with four files, shown here.
    
• LimaCharlie Adapter
  • Download the adapter specific to your system.
• Austin Workshop Secrets
  • This contains the URLs and API keys you'll utilize for the workshop. These credentials will only be valid for a short time (< 1 week)

Important

Make sure you save these files to a convient location as you'll need them throughout the workshop

Setting Up LimaCharlie

In these labs we will be configuring a LimaCharlie organization, loading in sample data, building detection rules, and utilizing response actions to create an alert when a detection is made.

Lab 1: Configuring LimaCharlie and Ingesting Logs

To begin, we will need to create an account within LimaCharlie and ingest the first set of logs for this workshop. We will use the community tier of LimaCharlie for this workshop, which will be yours to keep for free after the workshop. The community edition is limited to two sensors per organization and two organizations (four sensors in total).

1. Create a free LimaCharlie account

   • Please select LimaCharlie Workshop as the event

2. Once you have an organization setup in LimaCharlie, log in to your LimaCharlie account and click on your organization's name

3. Switch to the "Modern Theme" by clicking the "Try Modern Theme" button at the top of the screen

4. Since we're not going to be using a built-in adapter, click "Skip for now" at the bottom of the UI if you're prompted to install your first sensor

5. First, we need to create an installation key that will be used to authenticate the sensor that will be used to ingest the lab data. To access your installation keys, click on "Sensors" and then select "Installation Keys"

6. Create a new installation key by clicking the "Create Installation Key" button

7. Add a description that will allow you to associate the installation key with the intended purpose

8. The installation key allows for sensors to automatically be tagged when they authenticate to LimaCharlie. For this workshop, use the tag lab-data and then click "Create"

Tip

Tags can be used for a variety of purposes, including classifying endpoints, automating detection and response actions, creating powerful workflows, and triggering automations

9. Get the new installation key's GUID by clicking the copy icon in "Adapter Key" column for the installation key you created.

Important

This will be used in the next few steps, so it's recommended to put this into a text file to make referencing it easier

10. You'll also need your organizations ID (OID), so copy it from the URL and add it to your text file. The OID will be the GUID after /orgs/ in the url

Tip

The OID is also available by viewing a sensor's details, but grabbing it from the URL is simpler.

11. Next, on the sidebar, click "Sensors" and then click on "External Adapters"

Note

LimaCharlie offers multiple ways to ingest logs into the platform. For this lab, we're going to read in the file from your system using a local adapter. To simplify the management of local adapters, the External Adapters menu allows you to centralize management of all your on-premise adapters.

12. Click "Add External Adapter"

13. Give your adapter a descriptive name.

14. Copy and paste the following YAML into the "External Adapter Definition" box:

sensor_type: file
file:
  client_options:
    identity:
      installation_key: YOUR_INSTALLATION_KEY
      oid: YOUR_OID
    platform: lc_event
    sensor_seed_key: austin_workshop
    serialize_files: false
  backfill: true
  inactivity_threshold: 0
  no_follow: false
  file_path: YOUR_FILE_PATH\\small_sample-1.jsonl

15. Change the YOUR_OID field to the OID you saved in step 8

16. Change the YOUR_INSTALL_KEY field to the installation key you saved in step 6

17. Change the YOUR_FILE_PATH field to point to the small_sample-1.jsonl file you extracted in the Resources to Download section

Important

If you are on a Windows system, ensure you use double backspaces when putting in the file path
Example: C:\\Users\\cbot\\Downloads\\workshop_files\\small_sample-1.jsonl

18. Switch back to your web browser where you were creating the external adapter within LimaCharlie

19. Copy and paste the entire configuratin into the "External Adapter Definition" area in your web browser and then click "Create"

20. Copy the GUID into your text editor. You will use this to configure the adapter you downloaded in the Resources to Download section

Note

If you do not see the GUID field on your screen, make sure you're using the Modern Theme

21. Open a command prompt on your system and change to the directory where you extracted the archive to

22. Execute the adapter setting the type as cloud, then pass the parameter conf_guid and paste in the GUID value from the External Adapter configuration we created in step 17 along with the oid parameter and OID from step 8. The output should look like the following. If you do not see the "opening file: PATH/small_sample-1.jsonl" line, then check your file_path and make sure it points to correct file and location

• Example command:

lc_adapter.exe cloud conf_guid=2cb02e19-2e21-4584-8313-49202147d6e9 oid=9d4817ea-9369-4b8b-a109-5101fe75a1b1

1. Switch back to your web browser. On the side menu, click "Sensors" if it's not already open and then select the sensor you created by clicking on the sensor with your hostname. If you cannot find the sensor you created, wait a minute and refresh the page. Verify your sensor shows as online

2. Change to the default theme by clicking the "Go Back to Default Theme" button at the top of the screen

3. On the sidebar, click on "Query Console"

4. Enter the following query and then click "Submit"

-2h | * | NETWORK_CONNECTIONS | event/NETWORK_ACTIVITY/?/SOURCE/IP_ADDRESS is public address

5. Verify data is returned. The results show the logs you ingested in the previous steps. If you do not see logs after a few minutes, feel free to ask your moderator for assistance.

🎉 Lab 1 is now complete!

Lab 2: Configuring Add-ons

Add-ons, also known as extensions, extend LimaCharlie's capabilities and allow you to add functionality such as managed rulesets, API connections, investigative tools, and threat intelligence feeds. While many add-ons are provided by LimaCharlie, users are encouraged to develop their own and make them available to other users through the add-ons marketplace.

1. If you are not already logged in to your LimaCharlie account, log in

2. Click "Add-Ons", located at the top-right of the LimaCharlie UI. Feel free to explore the add-ons that are available

Note

Some add-ons may have a cost associated with them. This cost is not included in the free tier.

3. Search for the ext-lookup-manager add-on and click on it

4. The screen displays information about the extension, the developer, as well as any associated cost. To enable an add-on click on "Subscribe". Add-ons are subscribed on a per-organization basis, so only the organization displayed is subscribed to the add-on

5. On the sidebar, click "All Add-ons" to go back to viewing all available add-ons

6. Using the same process as before, subscribe your organization to the following add-ons. This will provide your organization with a base set of detections:

   • ext-sigma
   • ext-snapattack

🎉 Lab 2 is now finished. On to lab 3!

Lab 3: Ingesting Lookups

Lookups are lists that can contain various types of information such as malicious hashes and IPs and can be used in detection rules. These can be manually added, or they can be managed via the lookup manager to simplify adding and maintaining them. The lookup manager will automatically synchronize the list every 24 hours, or the lookups can be manually synchronized by clicking the "Manual Sync" button within the lookup manager extension.

The lookup manager utilizes Authenticated Resource Locator (ARL) strings to define where to get the lookup from. ARLs can point to standard websites, websites with authentication, or even specialized resources such as Google Cloud Storage and GitHub repos. For this lab, we'll be configuring the Lookup Manager that was enabled in lab 2, which will allow us easily import and manage lookup lists from an unauthenticated URL.

1. If you are not already logged in to your LimaCharlie account, log in

2. On the sidebar menu, click on "Extensions" and then click "Lookup Manager"

Note

If you do not see the Lookup Manager under Extensions, ensure you subscribed to it in Lab 2: Configuring Add-ons

3. Click "Add New Lookup Configuration"

4. Give the lookup a descriptive name that you will reference later

Tip

This name will be how you reference the lookup within rules, so make sure to name it something memorable

5. Copy and paste the first ARL from the workshop secrets list you were provided. If you do not have this list, please reach out to your coordinator

   • If you are working on this outside of a LimaCharlie workshop, please feel free to use the following ARLs that are similar to the SOCRadar lists used during the workshop:
     • Malicious IPs: [github,romainmarcoux/malicious-ip/full-40k.txt]
     • Malicous Hashes: [github,romainmarcoux/malicious-hash/full-hash-sha256-aa.txt]

6. For the format, select "newline" since each entry can be found on a separate line

Tip

To view the additional formats supported by LimaCharlie, visit Lookups

7. Click "Save" to save the lookup.

8. In order to sync the lookup without waiting for the 24 hour period, click on "Manual Sync" at the top-right corner of the UI and then click "Manual Sync" on the popup

9. If everything was done correctly, a message will pop up in the bottom-right corner of the UI that says the sync was successful. If there is an error, review your configuration and attempt again. If you still encounter issues, please reach out to the workshop coordinator for assistance

10. Repeat steps 3 through 8 for the remaining lookups

11. To view the lookups you just created, select "Automation" from the sidebar and then click "Lookups"

12. Open one of the lookups you created by clicking on the lookup name. A modal will pop up displaying the information contained within the lookup'

Note

Notice the information is now JSON formatted even though the data came in separated by newlines. LimaCharlie normalizes lookups into JSON objects for you, simplifying the import.

🎉 Lab 3 is finished and you now have working lookups that will automatically syncronize every 24 hours! Head over to lab 4 to configure outputs.

Lab 4: Configuring Outputs

LimaCharlie output can be used for many different tasks. Outputs can send events to a third-party location for long-term (> 1 year) retention, trigger webhooks, send notifications, etc.

1. You know the drill at this point - if you are not already logged in to your LimaCharlie account, log in

2. On the sidebar, click "Outputs"

3. Click "Add Output"

4. Since we only want to send to our output on specific detections, select the "Tailored" output stream

Tip

LimaCharlie's data pipeline allows you to output as much or as little data as desired. This can take the form of raw events, detections, audit logs, etc. For more information, please see Outputs in the LimaCharlie documentation

5. Choose the "Slack" output destination

Tip

The output destination selection screen shows all of the output destinations supported by LimaCharlie. In addition to the vendor-specific destinations such as Tines, generic destinations are also supported via webhook and SMTP. This allows LimaCharlie to easily integrate with ticketing systems such as ServiceNow, ZenDesk, etc. For more information, please see Output Destinations

6. Give your destination a memorable name. You will need to reference this name during D&R rule creation

7. Set your API token to the token found in the workshop secrets file

Note

If doing this outside of a physical workshop, you will need to create a free Slack workspace and follow the instructions located here to create the API token: Getting a Slack API Token

8. Set your channel to #austin-workshop

9. Click "Advanced Options" to display additional options for configuring the output

10. In the "Message Text" box, put your name and then click "Save Output"

🎉 Your LimaCharlie environment is now set up! Next, it's time to create some detection and response rules using all the work you've done thus far.

Writing Detection and Response Rules from Scratch

LimaCharlie allows users to create rules to detect activiy and then response actions to do something with that detection. These responses can range from a generating a simple alert to complex, multi-step processes intended to automate IR tasks such as isolating hosts, dumping memory, running playbooks. Additionally, these reponse actions can trigger actions in third-party applications such as Tines to enhance response activities.

All rules are written in YAML and are made up of two descriptors, detection and response.

Tip

Within the LimaCharlie UI, these are broken into separate text boxes, so when copying and pasting from the workshop files and LimaCharlie documentation, you will copy the section beginning with # Detection into the detection box and the section beginning with # Response into the response box.

Lab 5: Writing a Detection and Response Rule for Known Malicious IPs

Rules can either be created completely from scratch or a basic rule can automatically be generated from an event. Events can be found directly from a sensor or from the LimaCharlie Query Console. For this lab, we'll utilize a LCQL query to find an event representative of what we want to detect on, and then use that rule generate our starter rule that we'll then expand on.

Currently, the Query Console is only available within the old UI, so if you have changed to the new UI, please click the "Go back to default theme" button at the top of the UI.

1. Click "Query Console" from on the sidebar

2. Enter the following query and then slick save:

-2h | * | NETWORK_CONNECTIONS | event/NETWORK_ACTIVITY/?/SOURCE/IP_ADDRESS is public address

Tip

Save queries to save time and avoid having to remember the query every time you want to look for events or detections

3. Click "Save New Query" button to go to the save options

4. Give the query a descriptive name, and then click "Save"

5. Click on the "Saved" tab to view the query you just created. Since this query is already loaded, just click on the "Query" tab to go back to your query.

6. Click Submit to query the data you loaded during lab 1. If you do not see any results, ensure you correctly entered the query. If you need assistance, please feel free to ask.

7. These events show processes making connections to internal and external IPs. Click on any event to view the fields available in the events.

Tip

You can download the results of the query in json or csv format

8. On the sidebar, click "Automation" and then select "D&R Rules"

9. Click the "Add Rule" button on the top-right of the UI

10. In the "Name" field, give your rule a descriptive name

11. Copy and paste the following YAML into the "Detect" box, replacing the IP_LOOKUP_NAME field below with the IP lookup you created in Lab 3: Ingesting Lookups

event: NETWORK_CONNECTIONS
op: lookup
path: event/NETWORK_ACTIVITY/?/SOURCE/IP_ADDRESS
resource: hive://lookup/IP_LOOKUP_NAME 

Tip

Explanation of what is going on here
The event type we want to detect on: event: NETWORK_CONNECTIONS
The operator that determines how the rules are interpreted, here it's a lookup: op: lookup
The path within the event we want to look at for detection: path: event/NETWORK_ACTIVITY/?/SOURCE/IP_ADDRESS
Finally, the lookup we want to compare the "path" against: resource: hive://lookup/IP_LOOKUP_NAME

10. Copy and paste the following YAML into the "Respond" box, replacing the SLACK_OUTPUT_NAME with the name of the slack output you created in Lab 4: Configuring Outputs:

- action: report
  name: WORKSHOP - Attempted connection to known malicious IP - {{ .routing.hostname }}
- action: output
  name: SLACK_OUTPUT_NAME

Important

Make sure your indentations are correct. YAML is an indentation-based language, so incorrect indentation will cause errors

Tip

Explanation of what is going on here
The action we want to occur, in this case "report," which generates a detection: - action: report The name we want our detection to display. This also includes a variable from the event's routing information to dynamically include the hostname in the detection name: name: WORKSHOP - Attempted connection to known malicious IP - {{ .routing.hostname }} The second action tells LimaCharlie to send the detection to an output: - action: output Finally, the second "name" tells LimaCharlie which output to use: name: SLACK_OUTPUT_NAME

11. After you have entered the YAML in both text boxes, click "Create" to create your rule.

12. Open the external adapter you created in Lab 1: Configuring LimaCharlie and Ingesting Logs and change the file_path to point to the file sample_logs-1.jsonl. The adapter will automatically restart within a few minutes and read in the new file

Note

To speed up the process, feel free to restart the lc_adapter process. When it restarts, it'll pull down the new configuration and start reading the new file.

13. If you are attending an in-person LimaCharlie workshop, join the #austin-workshop channel in the testing Slack workspace at: LC Testing Slack Workspace

Note

If you are working on this workshop on your own, you will need to utilize your own Slack workspace

14. You should see message in the Slack channel with your name and the event information. If you do not see this after a few minutes, please verify your YAML was correctly created and the Slack output is correctly configured. If you need assistance, please don't hesitate to ask.

🎉 You've completed the first D&R rule lab! In the next lab, you'll take what you've learned and create your own rule.

Lab 6: Writing a Detection and Response Rule for Known Malicious Hashes

Using the information you learned in this workshop, create a D&R rule that alerts on a known-bad hash using the following information:

1. First sample log file: small_sample-2.jsonl

Tip

Reconfigure the external adapter to point to this file

2. LCQL Query: -2h | * | NEW_PROCESS | event/HASH != ''
   • Pick any of the NEW_PROCESS events to create a D&R rule from

Tip

For help with LCQL, check out the LCQL Cheat Sheet

3. Match the hash contained at the following path against the hash lookup created in Lab 3: Ingesting Lookups
   • event/HASH
4. Create a report action with a name of your choosing
   • Extra credit: Include the file_path field in the name
5. Create another action to output the detection to the Slack output you previously created
6. Use the sample_logs-2.jsonl file to test your newly created rule