This article pertains to the Elastic Defend (Endpoint) on Windows.
This article aims to provide a greater understanding of the causes of Elastic Defend’s system resource utilization, and provides workarounds for some common problems that users may encounter, especially when deploying Defend alongside other security software.
The Elastic Endpoint (part of Elastic Defend) monitors activity across your system. In response to the actions of other programs, it collects information including:
- Process Creation/Termination
- File Access/Creation/Modification/Rename/Deletion
- Registry modifications
- Network activity
- DNS activity
- Windows Security Logs
- Threat Intelligence API Activity (such as process injection)
It may or may not forward these events to your Elastic Stack. If events are disabled in policy, Defend won’t stream these events to ElasticSearch, but it may still collect and enrich them to support other features such as Behavioral Protections.
Beyond collecting the base event information, Defend also collects additional information to enrich each event. For example, it computes and verifies digital signatures to include signer information in every process event. It also parses PE files to extract their Original File Names. These are just some examples - there are many more enrichments.
Collecting this information requires CPU cycles, and may require I/O. For example, when a file is written, the Defend must then read that file to scan it for malware. This involves checking its digital signature, computing file hashes, computing file entropy for its Machine Learning model, and scanning the file with yara.
If Behavioral Protections are enabled, Defend runs all collected events through a stateful rules engine that checks for (and quickly reacts to) hundreds of known-malicious patterns of behavior. This evaluation process requires CPU cycles.
Defend reacts to activity on your system, generating its own activity in response. Problems can arise on systems running other software that does the same thing. Example of such software include:
- Anti-Malware (AM) / Anti-Virus (AV)
- Endpoint Detection and Response (EDR)
- eXtended Detection and Response (XDR)
- Endpoint Protection Platform (EPP)
- Data Loss Prevention (DLP)
- Employee Monitoring Software
- Application Virtualization Software
If two or more applications react to system activity by generating their own activity, then feedback loops are possible. These feedback loops can cause spikes in resource usage for either or both products, or lead to deadlocks that cause the system to hang.
Imagine the following scenario with hypothetical third-party AV product:
- A user downloads a file with their web browser
- Elastic Defend's filesystem minifilter driver intercepts this file creation and asks its user-mode component,
elastic-endpoint.exe, to scan the file. elastic-endpoint.exeattempts to open the file to scan it.- AV's filesystem minifilter driver sees an application (
elastic-endpoint.exe) opening a file and intercepts it, asking its user-mode process to scan a file. - AV's user-mode process
AV.exeattempts to open the file to scan it. - Elastic Defend's filesystem minifilter driver intercepts
AV.exe's activity and asks its user-mode component,elastic-endpoint.exe, to scan the file. elastic-endpoint.exeattempts to open the file to scan it.- AV's filesystem minifilter driver sees an application (
elastic-endpoint.exe) opening a file and intercepts it, asking its user-mode process to scan a file. - AV's user-mode process
AV.exeattempts to open the file to scan it. - ... the loop continues
Such feedback loops degrade system performance and responsiveness, and can lead to spikes in CPU and I/O utilization. There are variations of this too, such as where the AV makes a temporary copy of the file to scan it asynchronously. Interactions can get even more complex when there are more than two products installed on a system.
Generally, it's not recommended to run multiple AV applications simultaneously. Here is AV Comparatives' take on it, titled "Why you should never have multiple antivirus programs on your computer." Despite this, some users prefer to run multiple security products simultaneously. In response, we created Trusted Applications to help deal with these conflicts. By having Defend ignore the activity of the other security software on your system, we can break this cycle, reduce wasted resources, and improve system performance. By also adding Defend as a Trusted Application in the third-party security product, we can break this cycle even sooner for better performance and fewer wasted resources. In the above example, even if both AV applications trust each other, both will still scan the file saved by the web browser.
While not guaranteed to resolve performance issues, Trusted Applications are a common first step when deploying new security software to an already-protected environment. If you intend to run multiple security applications in your environment and are encountering performance problems, we strongly recommend you deploy Trusted Applications ASAP.
Trusted applications work on a process level. Many security products also include kernel-level components (drivers) that can generate activity in system worker threads and/or arbitrary thread contexts. System worker threads run inside the System process in Task Manager, which should not be added as a Trusted Application. Activity generated within an arbitrary thread context can come from any thread (in any process) on the system while it is executing in kernel mode, such as performing a system call.
Many security products also inject DLLs into processes throughout the system to perform user-mode hooking. For example, an EDR may inject a DLL into Microsoft Office in order to intercept specific intra-process activity that is not easily accessible from its kernel driver. In this example, activity generated by this injected DLL appears to come from Microsoft Office, not the EDR. Microsoft Office should not be added as a Trusted Application, so Trusted Applications will likely not be able to work around issues stemming from this activity.
While adding your existing AV/EDR/EPP/DLP/etc software as a Trusted Application in Elastic Defend can help performance, better performance will be achieved (with fewer compatibility issues) if the trust is mutual. Defend calls these exclusions Trusted Applications, but other products may call them Process Exclusions, Ignored Processes, or Trusted Processes. It is important to note that file-, folder-, and path-based exclusions/exceptions are distinct from Trusted Applications and will NOT achieve the same result. The goal here is to ignore actions taken BY a process, not ignore the file that the process was spawned from. Files are different from processes.
The Elastic Defend’s main executable is “C:\Program Files\Elastic\Endpoint\elastic-endpoint.exe”. It is signed by “Elasticsearch, Inc.” (spaces included, sans quotes). There may be a secondary signature from “Elasticsearch B.V.”, though this may change in future releases. When adding Defend as a Trusted Application in a third-party product, you should require both the path and the signer to match if possible. This will reduce the risk of an attacker exploiting the gap created by this trust.
Here is an example of the process exclusion UI in Microsoft Defender:
Below are some resources to help you add Defend as a Trusted Application in your third-party security software. If you use a product not listed here, try searching for “PRODUCTNAME add process exclusion”
| Product | Resources |
|---|---|
| Microsoft Defender | How to add a file type or process exclusion to Windows Security Configure exclusions for files opened by processes |
| Symantec Endpoint Protection | Preventing SEP from scanning files accessed by a trusted process |
| Carbon Black Protection (Bit9) | Anti-Virus Exclusions for Agent (Windows) Antivirus Exclusions for Server |
| Carbon Black Cloud | How to Set up Exclusions in the Carbon Black Cloud Console for AV Products |
| Trend Micro | Adding exclusion for Anti-Malware Real-Time Scan in Deep Security |
| SentinelOne | SentinelOne - Path Exclusion (SentinelOne appears to combine path and process exclusions) |
| Cisco Secure Endpoint / AMP | Configure and Identify Cisco Secure Endpoint Exclusions |