面向XDR的数据质量度量方法（一）

Main Detection Types

None

The vendor is unable to detect red activity due to capability limitations or other reasons. If data is available that is not directly relevant to the procedure tested, this will be categorized as “None.” In these cases, a vendor may receive a categorization of “None” with additional notes and screenshots about that data. 可概括为：无检测能力，无对应实际攻击行为的可观测数据。

No data is collected related to the particular action being performed.An alert fires at the same time that the red team performs the procedure, but it is not related to the technique under test.The tool allows a user to run forensic analysis of a file. This forensic analysis shows the file's capabilities but does not directly show that a procedure was performed.

Telemetry

The capability produces some minimally processed data that is accessible to an end user and directly indicates that the red team activity occurred after the user performs human analysis. There is no evidence of complex logic or an advanced rule leading to the data output, and no labeling occurs other than simple field labeling. The detection needs to be demonstrably and logically related to the actual procedure performed. Proof of detection could include the view, query, or API search used to access the data and/or the detection output (e.g., table view or process tree). 可概括为：基本对应原始采集日志，且日志中明确包含攻击行为线索。

正面例子：Command-line output is produced that shows a certain command was run on a workstation by a given username.反面例子：The capability shows all the potential behaviors a malicious file could perform but does not indicate which behaviors actually occur. This is not a Telemetry detection because the detection is not demonstrably related to the procedure being performed.

Indicator of Compromise

The vendor identifies the red team activity based on known hashes, IP addresses, C2 domain, tool names, tool strings, or module names. Proof of detection could include the rule name, API/query used to access the data, and/or detection output. 可概括为：基于已知IOC匹配的检测结果。

The red team C2 IP address is identified as malicious.

Enrichment

The capability captures data (usually data as described above in the “Telemetry Available” category) and then enriches it with additional information such as a rule name, labels, tags, or ATT&CK tactics or techniques that would assist in a user's analysis of the data beyond what would have been originally presented. The enrichment must be demonstrably and logically related to the ATT&CK technique under test. There is no evidence of complex logic or an advanced rule leading to the data output beyond a simple “if X, tag with Y” condition. Proof of detection could include the view, query, or API search used to access the data and/or the detection output (e.g., table view or process tree). 可概括为：基于原始日志，实现对行为的ATT&CK技战术基本映射和标签富化。

正面例子：A simple rule looking for any execution of foo.exe produces an alert called “Foo Discovery Occurred” with the supporting data “cmd.exe foo”.Data showing “cmd.exe foo” that is tagged with the information “Reconnaissance observed.”Data showing “cmd.exe foo” that is tagged with the information “ATT&CK Technique T9999 Foo Discovery.”反面例子：Data showing “Process: cmd.exe foo” that is tagged with “Process created” counts as Telemetry rather than Enrichment because “Process created” would already be apparent to an analyst and is not related to the technique under test.

General Behavior

The capability produces an alert detection for suspicious or potentially malicious behavior based on some type of reported complex logic or rule (beyond a simple “if X, display Y Rule Name,” which would be categorized as Enrichment). The alert must be demonstrably and logically related to the technique under test, and the capability must have a visual designation indicating this is an alert (e.g. an icon indicating an alert, a notification, an impact score, or a similar visual representation). This detection may provide an ATT&CK “tactic”-level description (e.g. Discovery, Execution, etc.) and/or a general description indicating the behavior is anomalous but does not provide specific details on the procedure detected (i.e. a “black box”). (An ATT&CK reference is not necessary for detection to be categorized in this way, though would be noted if included.) Proof of detection could include the rule name or module that performs the detection as well as detection output. 可概括为：通过分析手段实现了对攻击行为的显示告警，该告警是ATT&CK Tactic战术层级的通用行为告警。

An alert called “Malicious Discovery” is triggered on a series of discovery techniques. The alert has a score indicating the alert is likely malicious. The alert shows does not identify the specific type of discovery performed.A “Suspicious File” alert triggered upon initial execution of the executable file.

Specific Behavior

The capability detects suspicious behavior based on some complex rule or logic and provides an ATT&CK “technique”-level description of the activity (beyond a simple “if X, display Y Rule Name,” which would be categorized as Enrichment). (An ATT&CK reference is not necessary for detection to be categorized in this way, though would be noted if included.) The detection includes additional language and/or explanation to provide more detail beyond a general designation that the behavior is malicious. The alert must be demonstrably and logically related to the = technique under test, and the capability must have a visual designation indicating this is an alert (e.g. an icon indicating an alert, a notification, an impact score, or a similar visual representation). Proof of detection could include rule name or module that performs the detection as well as detection output.可概括为：通过分析手段实现了对攻击行为的显示告警，该告警是ATT&CK Technique技术层级的特定行为告警。

The capability produces an alert named “UAC Bypass.” The alert contains details showing the capability detects a change in process integrity levels over a sequence of related events.The capability produces an alert named “Credential Dumping” for Mimikatz logonpasswords credential dump.

Modifier Detection Types

Delayed

The capability does not detect the activity in real-time or near-real-time when the red team executes the action, but subsequent alerts, data, enrichment, or additional processing produce a detection for the activity. The Delayed category is not applied to data based solely on the time it takes for regular detections to appear in the capability, nor is it applied due to range or connectivity issues that are unrelated to the capability itself. Proof of detection must include explanation for the delayed detection and proof that could include other alerts, narrative, queries used to obtain the detection, and/or the detection output. 可概括为：相关检测数据未能及时触发，例如云端检测和MSSP分析等，可见示例。

The capability does not detect the activity in real-time or near-real-time when the red team executes the action, but subsequent alerts, data, enrichment, or additional processing produce a detection for the activity. The Delayed category is not applied to data based solely on the time it takes for regular detections to appear in the capability, nor is it applied due to range or connectivity issues that are unrelated to the capability itself. Proof of detection must include explanation for the delayed detection and proof that could include other alerts, narrative, queries used to obtain the detection, and/or the detection output.

Tainted

The capability detects the activity based on previously identified suspicious/malicious behavior that is related to or “tainted by” the detection. An example could include previously identifying a process as malicious and marking subsequent events related to that process as malicious, either by direct action/result or other relationship. Proof of detection must include clear visual evidence indicating that the detection uses tainted propagation, including what other activity has led to this detection. 可概括为：基于前序检测结果，通过某种机制传播（如关联）形成的新的检测结果，可见示例。

The capability produces a General Behavior alert for “Malicious Process Detected,” and then shows in a tree view that cmd.exe ran ipconfig. The ipconfig Telemetry detection is shown to be tainted due to a line going from the parent General Behavior detection to the child Telemetry detection, so the Telemetry detection would be categorized as both Telemetry and Tainted.

Configuration Change

A detection is made possible by a special configuration change or additional API access that allows data not normally accessible to the end user to become available. This category is applied to detections that are produced because of changes made to the initial configuration (as described by the vendor at the start of the evaluation). Proof of detection should include the output of the detection that results from the change along with notes about the reason for the change, how it is changed, and how an end user could request it. 可概括为：厂商经过检测机制、配置文件中的配置参数修改后，才产生或展现出来的检测结果。

Data showing account creation is collected on the backend but not displayed to the end user by default. The vendor changes a backend setting to allow Telemetry on account creation to be displayed in the user interface, so a detection of Telemetry and Configuration Change would be given for the Create Account technique.The vendor toggles a setting that would display an additional label of “Discovery” when the foo, foo1, and foo2 discovery commands are executed. A detection of Enrichment and Configuration Change would be given (as opposed to a detection of Telemetry that would have been given before the change).A rule or detection logic is created and applied retroactively or is later retested to show functionality that exists in the capability.

Main Detection Types

None

No data available that was automatically collected, processed, and was made available within the capability related to the behavior under test. If data is available that is not directly relevant to the procedure tested, this will be categorized as “None.” In these cases, a vendor may receive a categorization of “None” with additional notes and screenshots about that data. 可概括为：无检测能力，无对应实际攻击行为的可观测数据。与APT3基本一致。

No data is collected related to the particular action being performed.An alert fires at the same time that the red team performs the procedure, but it is not related to the technique under test.

Telemetry

Minimally processed data collected by the capability showing that event(s) occurred specific to the behavior under test. (i.e. showing the procedure/command that was executed). Evidence must show definitively that behavior occurred and be related to the execution mechanism (did happen vs may have happened). Evidence must be related to what caused the behavior. There is no evidence of complex logic or an advanced rule leading to the data output, and no labeling occurred other than simple field labeling 可概括为：基本对应原始采集日志，且日志中明确包含攻击行为线索。与APT3基本一致。

Command-line output is produced that shows a certain command was run on a workstation by a given username.

MSSP

Data is presented from a managed security service provider (MSSP) or monitoring service based on human analysis and indication of an incident occurring. MSSP are inherently delayed due to the manual analysis necessary and will be marked as delayed to remain consistent with other delayed detections. 可概括为：基于远程MSSP的检测结果。相对APT3为新增。

An email was received from an analyst describing the context of actions related to data exfiltration.

General

Processed data specifying that malicious/abnormal event(s) occurred, with relation to the behavior under test. (i.e. cmd.exe /c copy cmd.exe sethc.exe, is abnormal/malicious activity or an identifier stating that "suspicious activity occurred". No or limited details are provided as to why the action was performed (tactic), or details for how the action was performed (technique). 可概括为：一般类型检测数据，如异常行为等，不包含对技战术的详细映射和识别。无APT3的对应类别。

An alert describing "cmd.exe /c copy cmd.exe sethc.exe" as abnormal/malicious activity, but not stating it's related to Accessibility Features or a more specific description of what occurred.A “Suspicious File” alert triggered upon initial execution of the executable file.An alert stating that "suspicious activity occurred" related to an action but did not provide detail.

Tactic

Processed data specifying ATT&CK Tactic or equivalent level of enrichment to the data collected by the capability. Gives the analyst information on the potential intent of the activity, or helps answer the question "why this would be done" (i.e. Persistence was set up or there was a sequence of Discovery commands). 可概括为：该数据映射到ATT&CK Tactic并包含了ATT&CK Tactic战术层级的信息，能够给出并解释告警对应行为的“Why”问题。基本对应APT3的General Behavior类别。

An alert called “Malicious Discovery” is triggered on a series of discovery techniques. The alert has a score indicating the alert is likely malicious. The alert does not identify the specific type of discovery performed.An alert describing that persistence occurred but not specifying how persistence was achieved.

Technique

Processed data specifying ATT&CK Technique or equivalent level of enrichment to the data collected by the capability. Gives the analyst information on how the action was performed, or helps answer the question "what was done" (i.e. Accessibility Features or Credential Dumping). 可概括为：该数据映射到ATT&CK Technique并包含了ATT&CK Technique技术层级的信息，能够给出并解释告警对应行为的“How”问题。基本对应APT3的Specific Behavior类别。

An alert called "Credential Dumping" is triggered with enough detail to show what process originated the behavior against lsass.exe and/or provides detail on what type of credential dumping occurred.An alert for "Lateral Movement with Service Execution" is triggered describing what service launched and what system was targeted.

Modifier Detection Types

Alert

Data is presented as priority notification to the analyst as an indication of a suspicious or malicious event occurring for further investigation (e.g.: icon, queue, highlight, popup, etc.). Not a modifier of Telemetry. 可概括为：指示检测结果为告警类型。相对APT3把数据粒度更精细化了，主分类只提数据和标签，不区分是否是告警。这更符合ATT&CK行为识别而非攻击识别的中性数据假设。

A visual notification occurred in a dashboard and/or alert queue that "Lateral Movement" occurred.A recognizable identifier populated to a dashboard so that an analyst recognizes that a high severity event may have occurred.

Correlated

Data is presented as being descendant of events previously identified as suspicious/malicious based on an alert or high suspicion activity that was brought to the attention of an analyst. Examples of correlation evidence include annotated process trees or tags applied to chains of events showing the relationship between the suspicious/malicious event and data from the technique under test. 可概括为：基于前序检测结果，通过某种机制传播（如关联）形成的新的检测结果，可见示例。基本对应APT3的Tainted类型，语义表达更明确了。

A process tree or chain of events is annotated showing the relationship between a net.exe process and a prior alert on "Credential Dumping".Telemetry in a dashboard shows a relationship between a process launch for ipconfig.exe and a prior alert on an IOC to show the linage of activity.

Delayed

Detection (alerts, telemetry, tactic, technique, etc.) is unavailable due to some factor that slows or defers its presentation to the user, for example subsequent or additional processing produce a detection for the activity. The Delayed category is not applied for normal automated data ingestion and routine processing taking minimal time for data to appear to the user, nor is it applied due to range or connectivity issues that are unrelated to the capability itself. The Delayed modifier will always be applied with modifiers describing more detail about the nature of the delay.Delayed is subdivided into:Manual — Processing was triggered by human action and not initiated automatically. In the case of detections provided by a MSSP, human analysts reviewed and produced the outputs that were later presented to an analyst.Processing - Detection incurred a delay based on additional data processing to apply complex logic to the events where the results were later available to an analyst. 可概括为：相关检测数据未能及时触发和展示，例如云端检测和MSSP分析等。基本对应APT3的Delayed子类型，增加了分类。

The capability's cloud service component uses machine learning algorithms that trigger a detection on credential dumping hours after the red team performs it. This detection would receive the Main detection category of Technique with a Modifier detection category of Delayed-Processing.The capability sends data back to an analyst team, who manually analyze the raw data and create an alert called “Malicious Behavior Detected” that appears in the interface three hours after the red team performs the procedure. This detection will receive a Main detection category of MSSP and a Modifier detection category of Delayed-Manual.

Host Interrogation

Data is manually pulled from an endpoint via the capability for analysis. This category represents possible behavior identified through manual analysis from data that is not automatically ingested and analyzed by the capability to show an analyst that event(s) occurred specific to the behavior under test. Though useful as a capability to some security teams, capturing data through these means may be difficult and/or depend on the skill level of the analyst to derive actionable information. Host Interrogation is a modifier that will only apply to the None category. It will be marked as delayed to remain consistent with other delayed detections. 可概括为：通过非自动化手段直接拉去目标主机数据进行分析，只针对None这一主类型。相对APT3为新增子类型。

There is a remote shell component to the capability that can be used to pull native OS logs from a system suspected of being compromised for further analysis.

Residual Artifact

Data, such as a binary or process memory, that requires additional analysis to determine what capabilities or behaviors may have been used. This category represents possible behavior identified through manual analysis from data that is not automatically ingested and analyzed by the capability to show an analyst that event(s) occurred specific to the behavior under test. The collected data is more a byproduct of adversary actions and less indicative of adversary behavior. Though useful as a capability to some security teams, capturing data through these means may be difficult and/or depend on the skill level of the analyst to derive actionable information. Residual Artifact is a modifier that will only apply to the None category. It will be marked as delayed to remain consistent with other delayed detections. 可概括为：通过非自动化的手段（通过专家）对特定数据进行再分析，以得出进一步的检测结果和信息。只针对None这一主类型。相对APT3为新增子类型。

Process memory of svchost.exe was collected for later analysis because it was identified as a suspicious process. Later strings analysis showed that there may have been a keylogger present on the system.PowerShell scripts are collected automatically upon execution. Later analysis of a script shows it contains the functionality to capture a screenshot of the user's desktop.

Configuration Change

The configuration of the capability was changed since the start of the evaluation. This may be done to show additional data can be collected and/or processed. The Configuration Change modifier may be applied with additional modifiers describing the nature of the change.Configuration Change is subdivided into:UX - Change was to the user experience and not to the capability's ability to detect behavior. Changes could include display of a certain type of data that was already collected but not visible to the user.Detection - Change was to the capability's ability to capture or process information that impacts its ability to detect adversary behavior. Changes could include collecting a new type of information by the sensor or new processing logic that was deployed. 可概括为：厂商经过检测机制、配置文件中的配置参数修改后，才产生或展现出来的检测结果。基本对应APT3的Configuration Change子类型，增加了分类。

Data showing account creation is collected on the backend but not displayed to the end user by default. The vendor changes a backend setting to allow Telemetry on account creation to be displayed in the user interface, so a detection of Telemetry and Configuration Change-UX would be given for the Create Account technique.The vendor toggles a setting that would display an additional label of “Discovery” when the foo, foo1, and foo2 discovery commands are executed. A detection of Tactic and Configuration Change-Detection would be given (as opposed to a detection of Telemetry that would have been given before the change).A rule or detection logic is created and applied retroactively or is later retested to show functionality that exists in the capability. This would be labeled with a modifier Configuration Change-Detection.

Innovative

Designation applied to innovative and useful ways to detect a technique under test. Not all techniques will have or can get this designation applied to vendor solutions. It is meant to highlight accurate and robust approaches that bring value and deeper insight to consumers. This modifier will be applied at the Evaluation Team's discretion and will take into account data collected, method of detection, accuracy of detection, context provided to the end user, and display of information. 可概括为：经评估团队认定的鲁棒、准确的分析方法产生的检测结果。相对APT3为新增子类型。

/

Main Detection Types

Not Applicable

Vendor did not have visibility on the system under test. The vendor must state before the evaluation what systems they did not deploy a sensor on to enable Not Applicable to be in scope for relevant steps. 可概括为：无检测数据采集能力，无法收集相关数据。相对APT29为新增。

No sensor was deployed in the Linux systems within the environment to capture command-line activity, which would have been required to satisfy the detection criteria of the technique under test.

None

No data was made available within the capability related to the behavior under test that satisfies the assigned detection criteria. There are no modifiers, notes, or screenshots included with a None. 可概括为：无检测能力，无对应实际攻击行为的可观测数据。与APT29基本一致。

/

Telemetry

Minimally processed data collected by the capability showing that event(s) occurred specific to the behavior under test that satisfies the assigned detection criteria. Evidence must show definitively that behavior occurred and be related to the execution mechanism (did happen vs may have happened). This data must be visible natively within the tool and can include data retrieved from the endpoint. 可概括为：基本对应原始采集日志，且日志中明确包含攻击行为线索。与APT29基本一致。

Command-line output is produced that shows a certain command was run on a workstation by a given username. There is a remote shell component within the capability that can be used to pull native OS logs from a system suspected of being compromised for further analysis.

General

Processed data specifying that malicious/abnormal event(s) occurred, with relation to the behavior under test. No or limited details are provided as to why the action was performed (tactic), or details for how the action was performed (technique). 可概括为：一般类型检测数据，如异常行为等，不包含对技战术的详细映射和识别。与APT29基本一致。

A detection describing "cmd.exe /c copy cmd.exe sethc.exe" as abnormal/malicious activity, but not stating it's related to Accessibility Features or a more specific description of what occurred. A “Suspicious File” detection triggered upon initial execution of the executable file. A detection stating that "suspicious activity occurred" related to an action but did not provide detail regarding the technique under test.

Tactic

Processed data specifying ATT&CK Tactic or equivalent level of enrichment to the data collected by the capability. Gives the analyst information on the potential intent of the activity or helps answer the question "why this would be done". To qualify as a detection, there must be more than a label on the event identifying the ATT&CK Tactic, and it must clearly connect a tactic-level description with the technique under-test. 可概括为：该数据映射到ATT&CK Tactic并包含了ATT&CK Tactic战术层级的信息，能够给出并解释告警对应行为的“Why”问题。与APT29基本一致。

A detection called “Malicious Discovery” is triggered on a series of discovery techniques. The detection does not identify the specific type of discovery performed. A detection describing that persistence occurred but not specifying how persistence was achieved.

Technique

Processed data specifying ATT&CK Technique, Sub-Technique or equivalent level of enrichment to the data collected by the capability. Gives the analyst information on how the action was performed or helps answer the question "what was done" (i.e. Accessibility Features or Credential Dumping). To qualify as a detection, there must be more than a label on the event identifying the ATT&CK Technique ID (TID), and it must clearly connect a technique-level description with the technique under-test. 可概括为：该数据映射到ATT&CK Technique并包含了ATT&CK Technique技术层级的信息，能够给出并解释告警对应行为的“How”问题。与APT29基本一致。

A detection called "Credential Dumping" is triggered with enough detail to show what process originated the behavior against lsass.exe and/or provides detail on what type of credential dumping occurred. A detection for "Lateral Movement with Service Execution" is triggered describing what service launched and what system was targeted.

Modifier Detection Types

Configuration Change

The configuration of the capability was changed since the start of the evaluation. This may be done to show additional data can be collected and/or processed. The Configuration Change modifier may be applied with additional modifiers describing the nature of the change, to include:Data Sources - Changes made to collect new information by the sensor.Detection Logic - Changes made to data processing logic.UX - Changes related to the display of data that was already collected but not visible to the user. 可概括为：厂商经过数据源、检测机制、配置文件中的配置参数修改后，才产生或展现出来的检测结果。与APT29基本一致，增加了分类。

The sensor is reconfigured to is created to enables the capability to monitor file activity related to data collection. This would be labeled with a modifier for Configuration Change-Data Sources. A new rule is created, a pre-existing rule enabled, or sensitivities (e.g., blacklists) changed to successfully trigger during a retest. These would be labeled with a modifier Configuration Change-Detection Logic. Data showing account creation is collected on the backend but not displayed to the end user by default. The vendor changes a backend setting to allow Telemetry on account creation to be displayed in the user interface, so a detection of Telemetry and Configuration Change-UX would be given for the Create Account technique.

Delayed

The detection is not immediately available to the analyst due to additional processing unavailable due to some factor that slows or defers its presentation to the user, for example subsequent or additional processing produce a detection for the activity. The Delayed category is not applied for normal automated data ingestion and routine processing taking minimal time for data to appear to the user, nor is it applied due to range or connectivity issues that are unrelated to the capability itself. The Delayed modifier will always be applied with modifiers describing more detail about the nature of the delay. 可概括为：相关检测数据未能及时触发和展示，例如云端检测等。与APT29基本一致，删除了MSSP相关的示例和描述。

The capability uses machine learning algorithms that trigger a detection on credential dumping after the normal data ingestion period. This detection would receive a Modifier detection category of Delayed with a description of the additional processing time.

Protection Categories

Not Applicable

Vendor did not deploy protection capabilities on the system under test. The vendor must state before the evaluation what systems they did not deploy a sensor on to enable Not Applicable to be in scope for relevant steps. 可概括为：无防护能力。

No sensor was deployed in the Linux systems within the environment to block red team activity.

None

The technique under test was not blocked and/or the technique was unsuccessful and there is no evidence provided to the user that the capability blocked the activity. 可概括为：未成功防护。

The technique under test was successful. The technique under test was unsuccessful, but no evidence was displayed within the capability showing that the behavior was explicitly blocked by the tool.

Blocked

The technique under test was blocked and the user was explicitly informed that the capability blocked the activity. 可概括为：成功阻断攻击行为。

A detection was generated for “Potential Malicious Credential Dumping” specifying that the capability detected potential credential access activity and successfully blocked the behavior.

Protection Modifier Categories

User Consent

The technique under test was blocked after confirmation/consent was manually provided by the user. 可概括为：通过人工介入实现攻击行为阻断。

A detection was generated for “Potential Malicious Credential Dumping” specifying that the capability detected potential credential access activity and successfully blocked the behavior only after the user accepted a prompt to confirm that the behavior should be blocked. This protection would receive a Modifier protection category of User Consent.