Skip to content

Commit 37f5b5f

Browse files
PLukas2018PLukas2018
committed
Docs: Added info to README about development, ...
To README were added: - info about usage of bot and its options, - information about basic bot behaviour, - info about how to contribute to development, - info about automatic tests, - info about script for evaluating past PRs, - info about the logger, - useful links.
1 parent 38315f0 commit 37f5b5f

1 file changed

Lines changed: 154 additions & 18 deletions

File tree

‎README.md‎

Lines changed: 154 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -2,35 +2,100 @@
22

33
GitHub Bot used in the DiffKemp development process.
44

5+
Bot allows to evaluate impact of DiffKemp PRs, by comparing semantic comparison
6+
results gained for selected experiments by using master version of DiffKemp
7+
and PR version of DiffKemp.
8+
59
## Requirements
610

711
- Node.js v22
812
- Podman
913

10-
## Setup to run the bot
14+
## Usage
1115

12-
```sh
13-
# Create podman image
14-
./create_image.sh
15-
# Install dependencies
16-
npm install
17-
# Translate TypeScript to JavaScript
18-
npm run build
19-
# Run the bot
20-
npm start
21-
```
16+
Create comment on a PR in the repository containing `\evaluate` on a single
17+
line (you need to have `write`/`admin` permissions to the repository).
2218

23-
## Usage
19+
It is possible to specify options to the `\evaluate` command:
20+
21+
### Options
2422

25-
1. Install the bot application to a repository.
26-
2. Create comment on a PR in the repository containing `\evaluate` on a single line.
23+
- `--run <experiments...>`: Selection of experiments to be run (`eqbench`,
24+
`rhel-sysctl`, `rhel-functions`), by default all experiments are run.
25+
- `--pr-cmp-opt <options...>`: Allows adding options for `diffkemp compare`
26+
command which will be used when comparing experiments using a PR version
27+
of DiffKemp.
28+
- `--cmp-opt <options...>`: Allows adding options for `diffkemp compare`
29+
command which will be used when comparing experiments using both a PR and
30+
master version of DiffKemp.
31+
- `--rebuild`: Does not use cached snapshots from master branch instead of it,
32+
it builds them.
33+
34+
## About (behavior)
35+
36+
- The bot automatically compares experiments when a new push to a DiffKemp
37+
master occurs, caching the results of comparison and builded snapshots.
38+
For the comparison the master version of DiffKemp is used.
39+
- Reviewer can [trigger evaluation](#usage) of a PR. Bot by default uses
40+
snapshots created on a master branch and compares them using PR version of
41+
DiffKemp. After the comparison is done bot posts comments on the PR informing
42+
about differences in experiments' results gained by master version of
43+
DiffKemp and PR version of DiffKemp.
2744

2845
## Development
2946

30-
- Bot created using [Probot](https://probot.github.io/docs/) framework.
31-
- Written in [TypeScript](https://www.typescriptlang.org/).
32-
- Uses [typescript-eslint](https://typescript-eslint.io/) as linter.
33-
- Uses [Prettier](https://prettier.io/) as formatter.
47+
0. Create a repository fork and clone the project.
48+
1. Install the [requirements](#requirements).
49+
2. Install dependencies:
50+
51+
```bash
52+
npm install
53+
```
54+
55+
3. Create container image used for running experiments (this will take a while):
56+
57+
```bash
58+
./create_image.sh
59+
```
60+
61+
4. Build the bot (translate TypeScript to Javascript):
62+
63+
```bash
64+
npm run build
65+
```
66+
67+
5. Run the bot:
68+
69+
```bash
70+
npm start
71+
```
72+
73+
go to <http://localhost:3000> and follow the instructions (*Register GitHub
74+
App*) for creating GitHub App and installing the Bot for your development
75+
purposes:
76+
- Register the App (the app is specified by app manifest in `app.yml`).
77+
6. Restart the server in your terminal (press ctrl + c to stop the server, run
78+
`npm start`).
79+
80+
7. Install the [created application](https://github.com/settings/apps) to your
81+
fork of [DiffKemp](https://github.com/diffkemp/diffkemp) repository, where
82+
you will test the bot. Keep the bot running for some time -- it will run
83+
experiments on the master branch of your DiffKemp fork and cache the
84+
snapshots and results.
85+
86+
8. Try triggering the bot by creating a PR on your DiffKemp fork and running
87+
`\evaluate` command (see [usage](#usage)).
88+
89+
9. Develop new features:
90+
91+
- Add new feature.
92+
- [Check and fix code style](#code-style-checking).
93+
- Add tests and [run them](#tests).
94+
- Test the feature manually on your fork of DiffKemp repository.
95+
- Test the feature [on the previous PRs](#running-past-prs-evaluation),
96+
review the results.
97+
- Update bot documentation.
98+
- Create pull request on the main bot repository.
3499

35100
### Code style checking
36101

@@ -48,3 +113,74 @@ npm run format:check
48113
# Auto-corrects format
49114
npm run format
50115
```
116+
117+
### Automatic tests
118+
119+
Bot contains few tests that checks basic logic, they can be run following
120+
command:
121+
122+
```bash
123+
npm run test
124+
```
125+
126+
### Running past PRs evaluation
127+
128+
By running:
129+
130+
```bash
131+
npm run past-prs
132+
```
133+
134+
selected previous PRs from DiffKemp main repository will be analyzed.
135+
The results will be saved to `past-prs-logs-<DATE-AND-TIME>` directory.
136+
The directory contains markdown file for each PR named by the PR number.
137+
The files contain:
138+
139+
- master commit SHA,
140+
- PR commit SHA,
141+
- **verdict** - if the PR provides same or different results than the master,
142+
- Detailed information which would be comment on the PR if the bot would be run
143+
on it.
144+
145+
The script caches lots of information (snapshots, results, results directories)
146+
to simplify additional analysis. For this reason it can be useful to use
147+
`--cache-dir` for specifying different caching directory than is used by default
148+
which could be after running manually removed.
149+
150+
The PRs that should be evaluated are specified by a YAML file passed to
151+
`--config-file` option. The YAML format is following:
152+
153+
```yaml
154+
---
155+
# PR numbers to compare
156+
- 315
157+
- 319
158+
# Optionally specification of PR number/id to evaluate in object format
159+
- id: 322
160+
# Additional options that will be used by `diffkemp compare` command when
161+
# comparing projects using PR version of DiffKemp.
162+
prCmpOpt:
163+
- "--enable-pattern=sequential-alu-ops"
164+
```
165+
166+
### Logger
167+
168+
For logging is used [pino Logger](https://github.com/pinojs/pino), the used
169+
logging level can be using `LOG_LEVEL` variable in `.env` file (`trace`, `info`,
170+
`error`, `debug`, ...).
171+
172+
### Links
173+
174+
- Bot created using [Probot](https://probot.github.io/docs/) framework.
175+
- Written in [TypeScript](https://www.typescriptlang.org/).
176+
- Uses [typescript-eslint](https://typescript-eslint.io/) as linter.
177+
- Uses [Prettier](https://prettier.io/) as formatter.
178+
- [Octokit REST API](https://octokit.github.io/rest.js/v22/) for creating API
179+
requests.
180+
- [GitHub webhooks docs](https://docs.github.com/en/webhooks/about-webhooks)
181+
for reacting on GitHub events.
182+
- [GitHub REST API docs](https://docs.github.com/en/rest).
183+
- [GitHub APP docs](https://docs.github.com/en/apps/overview).
184+
- [pm2](https://www.npmjs.com/package/pm2) for running the bot in the
185+
production.
186+
- [pino Logger](https://github.com/pinojs/pino).

0 commit comments

Comments
 (0)