Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .codegraph/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# CodeGraph data files
# These are local to each machine and should not be committed

# Database
*.db
*.db-wal
*.db-shm

# Cache
cache/

# Logs
*.log

# Hook markers
.dirty
35 changes: 35 additions & 0 deletions MEMORY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Dev Notes

## Local dev environment (preferred)

Use **`docker-compose-dev-mysql.yml`** for local dev — do not use `docker-compose.yml` or manual `go run` unless explicitly asked.

```bash
# Start full stack
docker compose -f docker-compose-dev-mysql.yml up -d --build

# Stop stack
docker compose -f docker-compose-dev-mysql.yml down
```

| Service | URL |
|-----------|-----|
| Config UI | http://localhost:4001 |
| DevLake API | http://localhost:8083 |
| Grafana | http://localhost:3001/grafana (login `admin` / `admin`) |
| MySQL | `127.0.0.1:3306` — user `merico`, password `merico`, database `lake` |

Grafana uses local dashboards from `grafana/dashboards/mysql/` (includes Board/Issue sections for `azuredevops_go`).

Before starting dev-mysql, stop any conflicting stack (`docker compose -f docker-compose.yml down`) and kill local `go run server/main.go` if running.

Requires `.env` at repo root with at least `ENCRYPTION_SECRET` (see `docker-compose.yml` for dev value).

DevLake container sets `AUTH_ENABLED=false` in compose so Config UI works without OIDC/SSO login.

## Plugins

- If blueprint trigger fails with `Plugin 'dora' doesn't exist` or `Plugin 'issue_trace' doesn't exist`, rebuild plugins with:
`DEVLAKE_PLUGINS=azuredevops_go,dora,issue_trace,org,refdiff make build-plugin`
- Dev-mysql runs DevLake in Docker (`devlake-local:latest`); for plugin changes on host, rebuild backend image or use bind-mount workflow as needed.
- Verify plugin loading via `GET http://localhost:8083/plugins`.
1 change: 1 addition & 0 deletions backend/Dockerfile.local
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ RUN apt-get update && apt-get install -y \
libssh2-1 \
libssl3 \
ca-certificates \
git \
&& rm -rf /var/lib/apt/lists/*

# Copy libgit2
Expand Down
58 changes: 49 additions & 9 deletions backend/plugins/azuredevops_go/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,56 @@ See the License for the specific language governing permissions and
limitations under the License.
-->

# Azure Devops Python Plugin
# Azure DevOps Go Plugin

This is a revamped version of the Python Azure DevOps Plugin, originally located at `../../python/plugins/azuredevops`.
The plugin is able to coexist with the Python version as both implementations come with their own `_raw` and `_tool` tables.
Go implementation of the Azure DevOps plugin for Apache DevLake. Collects repository (CODE/CODEREVIEW/CROSS), pipeline (CICD), and board/work item (TICKET) data.

**Read access** to the following Azure DevOps Scopes is required:
## Authentication

- Build
- Code
- Graph (collectAccounts task)
- Release
Create a Personal Access Token (PAT) with the following scopes:

Access to Service Connections has been removed as they usually contain sensitive security information.
| Scope | Purpose |
|-------|---------|
| **Code (read)** | Repositories, commits, pull requests |
| **Build (read)** | Pipelines and CI/CD runs |
| **Work Items (read)** (`vso.work`) | Boards, work items, iterations, changelogs |
| **Graph (read)** | User accounts (optional) |

When creating the PAT, select **All accessible organizations** if you need multi-org support.

## Data Scopes

This plugin uses **dual scopes** on a single connection:

| Scope type | API path | Domain entities |
|------------|----------|-----------------|
| **Repositories** | `/connections/:id/repos` | CODE, CODEREVIEW, CROSS, CICD |
| **Boards / Teams** | `/connections/:id/scopes` | TICKET |

In Config UI, add repositories and board/team scopes separately. Associate a Scope Config with the entities you need (e.g. enable TICKET for board scopes).

## Scope Config (TICKET)

Configure work item type mappings in the Scope Config transformation panel:

- **Requirement**: User Story, Product Backlog Item, Feature, Epic
- **Bug**: Bug
- **Incident**: Production Incident (maps to DORA incidents via `project_mapping`)
- **Story Points field**: defaults to `Microsoft.VSTS.Scheduling.StoryPoints`

Sprint changes in changelogs are normalized to `field_name = 'Sprint'` for Engineering Overview dashboards.

## Blueprint

When adding scopes to a project blueprint, DevLake resolves each scope ID against boards first, then repositories. Board scopes run the TICKET pipeline independently; repository scopes no longer create a stub board.

## Grafana

Two dashboards are provided under **Data Source Dashboard**:

| Dashboard | Metrics |
|-----------|---------|
| **Azure DevOps Repo** | PR contribution, PR handling, CI/CD pipeline runs |
| **Azure DevOps Board** | Issue throughput, delivery rate, lead time (TICKET entity) |

Use the **Board / Team** filter on the Board dashboard for `azuredevops_go` boards. Use the **Repo** filter on the Repo dashboard for repositories.
86 changes: 86 additions & 0 deletions backend/plugins/azuredevops_go/api/azuredevops/models.go
Original file line number Diff line number Diff line change
Expand Up @@ -130,3 +130,89 @@ type Repository struct {
IsDisabled bool `json:"isDisabled"`
IsInMaintenance bool `json:"isInMaintenance"`
}

type Team struct {
Id string `json:"id"`
Name string `json:"name"`
Description string `json:"description"`
Url string `json:"url"`
}

type WiqlResponse struct {
QueryType string `json:"queryType"`
QueryResult struct {
WorkItems []struct {
Id int `json:"id"`
Url string `json:"url"`
} `json:"workItems"`
} `json:"queryResult"`
WorkItems []struct {
Id int `json:"id"`
Url string `json:"url"`
} `json:"workItems"`
}

type WorkItemBatchRequest struct {
Ids []int `json:"ids"`
Fields []string `json:"fields,omitempty"`
Expand string `json:"$expand,omitempty"`
ErrorPolicy string `json:"errorPolicy,omitempty"`
}

type WorkItem struct {
Id int `json:"id"`
Rev int `json:"rev"`
Fields map[string]interface{} `json:"fields"`
Url string `json:"url"`
Relations []WorkItemRelation `json:"relations"`
}

type WorkItemRelation struct {
Rel string `json:"rel"`
Url string `json:"url"`
Attributes map[string]interface{} `json:"attributes"`
}

type TeamIteration struct {
Id string `json:"id"`
Name string `json:"name"`
Path string `json:"path"`
Attributes struct {
StartDate *time.Time `json:"startDate"`
FinishDate *time.Time `json:"finishDate"`
TimeFrame string `json:"timeFrame"`
} `json:"attributes"`
}

type WorkItemUpdate struct {
Id int `json:"id"`
Rev int `json:"rev"`
Fields struct {
SystemChangedDate struct {
NewValue string `json:"newValue"`
} `json:"System.ChangedDate"`
} `json:"fields"`
Relations *struct {
Added []WorkItemRelation `json:"added"`
} `json:"relations"`
}

type WorkItemUpdatesResponse struct {
Count int `json:"count"`
Value []WorkItemUpdate `json:"value"`
}

type IterationWorkItemsResponse struct {
WorkItemRelations []struct {
Rel string `json:"rel"`
Source struct {
Id int `json:"id"`
Url string `json:"url"`
} `json:"source"`
Target struct {
Id int `json:"id"`
Url string `json:"url"`
} `json:"target"`
} `json:"workItemRelations"`
}

Loading