Skip to content

Commit 6dbed5e

Browse files
committed
Address review comments
1 parent 4094fa9 commit 6dbed5e

1 file changed

Lines changed: 32 additions & 56 deletions

File tree

Lines changed: 32 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
# Query classification and display
22

3-
Attributable Queries
3+
## Attributable Queries
44

55
The results of some queries are unsuitable for attribution to individual
66
developers. Most of them have a threshold value on which they trigger,
77
for example all metric violations and statistics based queries. The
88
results of such queries would all be attributed to the person pushing
9-
the value over (or under) the threshold. Some queries only trigger
9+
the value over (or under) the threshold. Some queries only trigger when
1010
another one doesn't. An example of this is the MaybeNull query which
1111
only triggers if the AlwaysNull query doesn't. A small change in the
12-
data flow could make an alert switch from AlwayNull to MaybeNull (or
12+
data flow could make an alert switch from AlwaysNull to MaybeNull (or
1313
vice versa). As a result we attribute both a fix and an introduction to
1414
the developer that changed the data flow. For this particular example
1515
the funny attribution results are more a nuisance than a real problem;
@@ -20,106 +20,82 @@ many others, where "duplicate function" only triggers if "duplicate
2020
file" didn't. As a result adding some code to a duplicate file might
2121
result in a "fix" of a "duplicate file" alert and an introduction of
2222
many "duplicate function" alerts. This would be highly unfair.
23-
Currently, on the duplicate and similar code queries exhibit this
23+
Currently, only the duplicate and similar code queries exhibit this
2424
"exchanging one for many" alerts when trying to attribute their results.
2525
Therefore we currently exclude all duplicate code related alerts from
2626
attribution.
2727

2828
The following queries are excluded from attribution:
2929

30-
- Metric violations, i.e. the ones with metadata properties like `
31-
@(error|warning|recommendation)-(to|from) `
32-
- Queries with tag ` non-attributable `
33-
34-
` `
35-
36-
<div>
37-
38-
<div>
30+
- Metric violations, i.e. the ones with metadata properties like
31+
 `@(error|warning|recommendation)-(to|from)`
32+
- Queries with tag `non-attributable`
3933

4034
This check is applied when the results of a single attribution are
4135
loaded into the datastore. This means that any change to this behaviour
42-
will only take effect on newly attributed revisions and the historical
43-
data is unchanged.
44-
45-
</div>
36+
will only take effect on newly attributed revisions but the historical
37+
data remains unchanged.
4638

47-
</div>
39+
## Query severity and precision
4840

49-
 
50-
51-
Query severity and precision
52-
53-
We currently classify queries in on two axes, with some additional tags.
41+
We currently classify queries on two axes, with some additional tags.
5442
Those axes are severity and precision, and are defined using the
5543
query-metadata properties `@problem.severity` and `@precision`.
5644

5745
For severity, we have the following categories:
5846

59-
- Error
60-
- Warning
61-
- Recommendation
47+
- Error
48+
- Warning
49+
- Recommendation
6250

6351
These categories may change in the future.
6452

6553
For precision, we have the following categories:
6654

67-
- Very-high
68-
- High
69-
- Medium
70-
- Low
55+
- Very-high
56+
- High
57+
- Medium
58+
- Low
7159

7260
As [usual](https://en.wikipedia.org/wiki/Precision_and_recall),
7361
precision is defined as the percentage of query results that are true
7462
positives, i.e., precision = number of true positives / (number of true
7563
positives + number of false positives). There is no hard-and-fast rule
7664
for which precision ranges correspond to which categories.
7765

78-
We expect these categories to remain unchanged for the forseeable
66+
We expect these categories to remain unchanged for the foreseeable
7967
future.
8068

8169
### A note on precision
8270

83-
Intuitively, precision measures how well the query does at finding the
71+
Intuitively, precision measures how well the query performs at finding the
8472
results it is supposed to find, i.e., how well it implements its
8573
(informal, unwritten) rule. So how precise a query is depends very much
8674
on what we consider that rule to be. We generally try to sharpen our
8775
rules to focus on results that a developer might actually be interested
8876
in.
8977

90-
91-
9278
## Which queries to run and display on LGTM
9379

9480
The following queries are run:
9581

96-
<div class="table-wrap">
97-
98-
| Precision: | V. high | High | Medium | Low |
99-
| -------------- | ----------- | ------- | ------- | --- |
100-
| Error | ****Yes**** | **Yes** | **Yes** | No |
101-
| Warning | ****Yes**** | **Yes** | **Yes** | No |
102-
| Recommendation | ****Yes**** | **Yes** | No | No |
103-
104-
</div>
82+
Precision: | Very high | High | Medium | Low
83+
---------------|-----------|---------|---------|----
84+
Error | **Yes** | **Yes** | **Yes** | No
85+
Warning | **Yes** | **Yes** | **Yes** | No
86+
Recommendation | **Yes** | **Yes** | No | No
10587

10688
The following queries have their results displayed by default:
10789

108-
<div class="table-wrap">
109-
110-
| Precision: | V. high | High | Medium | Low |
111-
| -------------- | ----------- | ----------- | ------ | --- |
112-
| Error | Yes | ****Yes**** | No | No |
113-
| Warning | ****Yes**** | ****Yes**** | No | No |
114-
| Recommendation | ****Yes**** | No | No | No |
115-
116-
</div>
117-
90+
Precision: | Very high | High | Medium | Low
91+
---------------|-----------|---------|--------|----
92+
Error | **Yes** | **Yes** | No | No
93+
Warning | **Yes** | **Yes** | No | No
94+
Recommendation | **Yes** | No | No | No
11895

11996
Results for queries that are run but not displayed by default can be
12097
made visible by editing the project configuration.
121-
12298

123-
 Queries from custom query packs (in-repo or site-wide) are always run
124-
and displayed by default. The can be hidden by editing the project
99+
Queries from custom query packs (in-repo or site-wide) are always run
100+
and displayed by default. They can be hidden by editing the project
125101
config, and "disabled" by removing them from the query pack.

0 commit comments

Comments
 (0)