Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 20 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,12 +72,12 @@ After setup, open Grafana at **http://localhost:3002** (admin / admin). DORA and

Four concepts to understand — then every command makes sense:

| Concept | What It Is |
|---------|-----------|
| **Connection** | An authenticated link to a data source (GitHub, Copilot, Jenkins). Each gets its own PAT/credentials. |
| **Scope** | *What* to collect — specific repos for GitHub, an org/enterprise for Copilot, jobs for Jenkins. |
| **Project** | Groups connections + scopes into a single view with DORA metrics enabled. |
| **Blueprint** | The sync schedule (cron). Created automatically with the project. |
| Concept | What It Is |
|---------|-----------|
| **Connection** | An authenticated link to a data source (GitHub, Copilot, Jenkins). Each gets its own PAT/credentials. |
| **Scope** | *What* to collect — specific repos for GitHub, an org/enterprise for Copilot, jobs for Jenkins. |
| **Project** | Groups connections + scopes into a single view with DORA metrics enabled. |
| **Blueprint** | The sync schedule (cron). Created automatically with the project. |

<!-- SCREENSHOT: Config UI project page showing connections → scopes → project hierarchy -->

Expand Down Expand Up @@ -117,12 +117,12 @@ The CLI will prompt you for your PAT. You can also pass `--token`, use an `--env
# GitHub (repos, PRs, workflows, deployments)
gh devlake configure connection add --plugin github --org my-org

# Copilot (usage metrics, seats, acceptance rates)
gh devlake configure connection add --plugin gh-copilot --org my-org

# Jenkins (jobs and build data for DORA)
gh devlake configure connection add --plugin jenkins --endpoint https://jenkins.example.com --username admin --token myapitoken
```
# Copilot (usage metrics, seats, acceptance rates)
gh devlake configure connection add --plugin gh-copilot --org my-org
# Jenkins (jobs and build data for DORA)
gh devlake configure connection add --plugin jenkins --endpoint https://jenkins.example.com --username admin --token myapitoken
```

The CLI tests each connection before saving. On success:

Expand All @@ -142,12 +142,12 @@ Tell DevLake which repos or orgs to collect from:
# GitHub — pick repos interactively, or pass --repos explicitly
gh devlake configure scope add --plugin github --org my-org

# Copilot — org-level metrics
gh devlake configure scope add --plugin gh-copilot --org my-org

# Jenkins — pick jobs interactively (or pass --jobs)
gh devlake configure scope add --plugin jenkins --org my-org
```
# Copilot — org-level metrics
gh devlake configure scope add --plugin gh-copilot --org my-org
# Jenkins — pick jobs interactively (or pass --jobs)
gh devlake configure scope add --plugin jenkins --org my-org
```

DORA patterns (deployment workflow, production environment, incident label) use sensible defaults. See [docs/configure-scope.md](docs/configure-scope.md) for overrides.

Expand Down Expand Up @@ -194,9 +194,10 @@ For the full guide, see [Day-2 Operations](docs/day-2.md).
| GitHub Copilot | ✅ Available | Usage metrics, seats, acceptance rates | `manage_billing:copilot`, `read:org` (+ `read:enterprise` for enterprise metrics) |
| Jenkins | ✅ Available | Jobs, builds, deployments (DORA) | Username + API token/password |
| Jira | ✅ Available | Boards, issues, sprints (change lead time, cycle time) | API token (permissions from user account) |
| Azure DevOps | 🔜 Coming soon | Repos, pipelines, deployments (DORA) | (TBD) |
| GitLab | ✅ Available | Repos, MRs, pipelines, deployments (DORA) | `read_api`, `read_repository` |
| Bitbucket Cloud | ✅ Available | Repos, PRs, commits | Bitbucket username + app password |
| SonarQube | ✅ Available | Code quality, coverage, code smells (quality gates) | API token (permissions from user account) |
| Azure DevOps | 🔜 Coming soon | Repos, pipelines, deployments (DORA) | (TBD) |

See [Token Handling](docs/token-handling.md) for env key names and multi-plugin `.devlake.env` examples.

Expand Down
79 changes: 79 additions & 0 deletions cmd/configure_scopes.go
Original file line number Diff line number Diff line change
Expand Up @@ -1021,3 +1021,82 @@ func putBitbucketScopes(client *devlake.Client, connID int, repos []*devlake.Bit
}
return client.PutScopes("bitbucket", connID, &devlake.ScopeBatchRequest{Data: data})
}

// scopeSonarQubeHandler is the ScopeHandler for the sonarqube plugin.
func scopeSonarQubeHandler(client *devlake.Client, connID int, org, enterprise string, opts *ScopeOpts) (*devlake.BlueprintConnection, error) {
fmt.Println("\n📋 Fetching SonarQube projects...")

// Aggregate all pages of remote scopes
var allChildren []devlake.RemoteScopeChild
pageToken := ""
for {
remoteScopes, err := client.ListRemoteScopes("sonarqube", connID, "", pageToken)
if err != nil {
return nil, fmt.Errorf("failed to list SonarQube projects: %w", err)
}
allChildren = append(allChildren, remoteScopes.Children...)
pageToken = remoteScopes.NextPageToken
if pageToken == "" {
break
}
}

// Extract projects from remote-scope response
var projectOptions []string
projectMap := make(map[string]*devlake.RemoteScopeChild)
for i := range allChildren {
child := &allChildren[i]
if child.Type == "scope" {
// Skip projects without a valid project key (child.ID)
if child.ID == "" {
continue
}
label := fmt.Sprintf("%s (key: %s)", child.Name, child.ID)
projectOptions = append(projectOptions, label)
projectMap[label] = child
}
}

if len(projectOptions) == 0 {
Comment on lines +1047 to +1060
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The selection list includes projects even when child.ID (the projectKey) is empty, but those selections are later skipped. This can lead to a confusing UX where a user selects items and then hits no valid projects to add. Consider filtering out child.ID == "" projects when building projectOptions (and not inserting them into projectMap), or fail early with a clear message that the server returned projects without keys.

Suggested change
for i := range allChildren {
child := &allChildren[i]
if child.Type == "scope" {
label := child.Name
if child.ID != "" {
label = fmt.Sprintf("%s (key: %s)", child.Name, child.ID)
}
projectOptions = append(projectOptions, label)
projectMap[label] = child
}
}
if len(projectOptions) == 0 {
missingKeyCount := 0
for i := range allChildren {
child := &allChildren[i]
if child.Type != "scope" {
continue
}
// Skip projects without a valid project key (child.ID)
if child.ID == "" {
missingKeyCount++
continue
}
label := fmt.Sprintf("%s (key: %s)", child.Name, child.ID)
projectOptions = append(projectOptions, label)
projectMap[label] = child
}
if len(projectOptions) == 0 {
if missingKeyCount > 0 {
return nil, fmt.Errorf("no SonarQube projects with valid project keys found for connection %d", connID)
}

Copilot uses AI. Check for mistakes.
return nil, fmt.Errorf("no SonarQube projects found for connection %d", connID)
}

fmt.Println()
selectedLabels := prompt.SelectMulti("Select SonarQube projects to track", projectOptions)
if len(selectedLabels) == 0 {
return nil, fmt.Errorf("at least one SonarQube project must be selected")
}

// Build scope data for PUT
fmt.Println("\n📝 Adding SonarQube project scopes...")
var scopeData []any
var blueprintScopes []devlake.BlueprintScope
for _, label := range selectedLabels {
child := projectMap[label]
scopeData = append(scopeData, devlake.SonarQubeProjectScope{
ConnectionID: connID,
ProjectKey: child.ID,
Name: child.Name,
})
blueprintScopes = append(blueprintScopes, devlake.BlueprintScope{
ScopeID: child.ID,
ScopeName: child.Name,
})
}

if len(scopeData) == 0 {
return nil, fmt.Errorf("no valid projects to add")
}

Comment on lines +1087 to +1090
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The len(scopeData) == 0 check here is unreachable: selectedLabels is guaranteed non-empty (checked at line 844), and the loop body unconditionally appends to scopeData for every label — there are no continue/skip branches unlike the Jira handler (which skips on empty or unparseable board IDs). Consider removing this dead check, or adding a comment noting it's a defensive guard for future-proofing.

Suggested change
if len(scopeData) == 0 {
return nil, fmt.Errorf("no valid projects to add")
}

Copilot uses AI. Check for mistakes.
err := client.PutScopes("sonarqube", connID, &devlake.ScopeBatchRequest{Data: scopeData})
if err != nil {
return nil, fmt.Errorf("failed to add SonarQube project scopes: %w", err)
}
fmt.Printf(" ✅ Added %d project scope(s)\n", len(scopeData))

return &devlake.BlueprintConnection{
PluginName: "sonarqube",
ConnectionID: connID,
Scopes: blueprintScopes,
}, nil
}
18 changes: 18 additions & 0 deletions cmd/connection_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,24 @@ var connectionRegistry = []*ConnectionDef{
ScopeIDField: "boardId",
HasRepoScopes: false,
},
{
Plugin: "sonarqube",
DisplayName: "SonarQube",
Available: true,
Endpoint: "", // user must provide (e.g., https://sonar.example.com/)
SupportsTest: true,
AuthMethod: "AccessToken",
RateLimitPerHour: 0, // uses default 4500
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RateLimitPerHour: 0 is ambiguous (it can mean “disabled”, “unlimited”, or “use default” depending on downstream logic). Since the comment states it should use the default (4500), prefer omitting the field (so the defaulting behavior is explicit), or set it to the actual intended default value to avoid misinterpretation.

Copilot uses AI. Check for mistakes.
// SonarQube uses API tokens; permissions come from the user account.
RequiredScopes: []string{},
ScopeHint: "",
TokenPrompt: "SonarQube token",
EnvVarNames: []string{"SONARQUBE_TOKEN", "SONAR_TOKEN"},
EnvFileKeys: []string{"SONARQUBE_TOKEN", "SONAR_TOKEN"},
ScopeFunc: scopeSonarQubeHandler,
ScopeIDField: "projectKey",
HasRepoScopes: false,
},
}

// AvailableConnections returns only available (non-coming-soon) connection defs.
Expand Down
67 changes: 66 additions & 1 deletion cmd/connection_types_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -664,4 +664,69 @@ func TestConnectionRegistry_Jenkins(t *testing.T) {
if !foundJobs {
t.Errorf("jenkins ScopeFlags should include jobs flag")
}
}
}
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing blank line between TestConnectionRegistry_Jenkins and TestConnectionRegistry_SonarQube. Every other top-level function in this file is separated by a blank line (e.g., lines 557-559). Add a blank line after the closing } on line 589.

Suggested change
}
}

Copilot uses AI. Check for mistakes.

// TestConnectionRegistry_SonarQube verifies the SonarQube plugin registry entry.
func TestConnectionRegistry_SonarQube(t *testing.T) {
def := FindConnectionDef("sonarqube")
if def == nil {
t.Fatal("sonarqube plugin not found in registry")
}

tests := []struct {
name string
got interface{}
want interface{}
}{
{"Plugin", def.Plugin, "sonarqube"},
{"DisplayName", def.DisplayName, "SonarQube"},
{"Available", def.Available, true},
{"Endpoint", def.Endpoint, ""},
{"SupportsTest", def.SupportsTest, true},
{"AuthMethod", def.AuthMethod, "AccessToken"},
{"ScopeIDField", def.ScopeIDField, "projectKey"},
{"HasRepoScopes", def.HasRepoScopes, false},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.got != tt.want {
t.Errorf("%s: got %v, want %v", tt.name, tt.got, tt.want)
}
})
}

if def.ScopeFunc == nil {
t.Error("ScopeFunc should not be nil")
}

// SonarQube uses API tokens, not OAuth/PAT scopes
if len(def.RequiredScopes) != 0 {
t.Errorf("RequiredScopes should be empty for SonarQube API tokens, got %v", def.RequiredScopes)
}
if def.ScopeHint != "" {
t.Errorf("ScopeHint should be empty for SonarQube API tokens, got %q", def.ScopeHint)
}

expectedEnvVars := []string{"SONARQUBE_TOKEN", "SONAR_TOKEN"}
if len(def.EnvVarNames) != len(expectedEnvVars) {
t.Errorf("EnvVarNames length: got %d, want %d", len(def.EnvVarNames), len(expectedEnvVars))
} else {
for i, v := range expectedEnvVars {
if def.EnvVarNames[i] != v {
t.Errorf("EnvVarNames[%d]: got %q, want %q", i, def.EnvVarNames[i], v)
}
}
}

expectedEnvFileKeys := []string{"SONARQUBE_TOKEN", "SONAR_TOKEN"}
if len(def.EnvFileKeys) != len(expectedEnvFileKeys) {
t.Errorf("EnvFileKeys length: got %d, want %d", len(def.EnvFileKeys), len(expectedEnvFileKeys))
} else {
for i, v := range expectedEnvFileKeys {
if def.EnvFileKeys[i] != v {
t.Errorf("EnvFileKeys[%d]: got %q, want %q", i, def.EnvFileKeys[i], v)
}
}
}
}
11 changes: 9 additions & 2 deletions internal/devlake/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,13 @@ type BitbucketRepoScope struct {
HTMLURL string `json:"htmlUrl,omitempty"`
}

// SonarQubeProjectScope represents a SonarQube project scope entry for PUT /scopes.
type SonarQubeProjectScope struct {
ConnectionID int `json:"connectionId"`
ProjectKey string `json:"projectKey"`
Name string `json:"name"`
}

// ScopeBatchRequest is the payload for PUT /scopes (batch upsert).
type ScopeBatchRequest struct {
Data []any `json:"data"`
Expand Down Expand Up @@ -172,7 +179,7 @@ type ScopeListResponse struct {

// RemoteScopeChild represents one item (group or scope) from the remote-scope API.
type RemoteScopeChild struct {
Type string `json:"type"` // "group" or "scope"
Type string `json:"type"` // "group" or "scope"
ID string `json:"id"`
ParentID string `json:"parentId"`
Name string `json:"name"`
Expand Down Expand Up @@ -244,4 +251,4 @@ type Pipeline struct {
Status string `json:"status"`
FinishedTasks int `json:"finishedTasks"`
TotalTasks int `json:"totalTasks"`
}
}