Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions eng/common/pipelines/templates/jobs/benchmark-job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
parameters:
- name: BenchmarkProject
type: string
default: 'tools/azsdk-cli/Azure.Sdk.Tools.Cli.Benchmarks'
- name: ScenarioName
type: string
default: ' '
- name: Tags
type: string
default: ' '
- name: Model
type: string
default: ' '
Comment on lines +7 to +13
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These parameters default to a single space (' ') and then rely on .Trim() in the script. Using an empty string default ('') avoids surprising whitespace behavior and removes the need for trimming to detect “unset” values (same applies to Tags/Model).

Suggested change
default: ' '
- name: Tags
type: string
default: ' '
- name: Model
type: string
default: ' '
default: ''
- name: Tags
type: string
default: ''
- name: Model
type: string
default: ''

Copilot uses AI. Check for mistakes.
- name: Parallelism
type: number
default: 5

jobs:
- job: Run_Benchmark
variables:
- template: /eng/pipelines/templates/variables/globals.yml
- template: /eng/pipelines/templates/variables/image.yml
- group: 'AzSDK_Eval_Variable_group'
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This template hard-codes a dependency on the Azure DevOps variable group AzSDK_Eval_Variable_group. That makes the job template non-portable and will fail in pipelines/projects where this group isn’t defined. Consider removing the variable group from the template and instead passing any required values in as parameters (or documenting that the consuming pipeline must include the group explicitly).

Copilot uses AI. Check for mistakes.
displayName: 'Run Benchmark'
pool:
name: $(LINUXPOOL)
image: $(LINUXVMIMAGE)
os: linux

steps:
- checkout: none

- template: /eng/common/pipelines/templates/steps/sparse-checkout.yml
parameters:
SkipCheckoutNone: true
Repositories:
- Name: $(Build.Repository.Name)
Commitish: $(Build.SourceVersion)
WorkingDirectory: $(System.DefaultWorkingDirectory)/$(Build.Repository.Name)
Paths:
- '${{ parameters.BenchmarkProject }}/**'
- 'eng/common/**'

- template: /eng/pipelines/templates/steps/install-dotnet.yml

- task: PowerShell@2
displayName: 'Run benchmarks'
condition: succeeded()
inputs:
targetType: 'inline'
workingDirectory: '$(System.DefaultWorkingDirectory)/$(Build.Repository.Name)/${{ parameters.BenchmarkProject }}'
pwsh: true
script: |
$cliArgs = @("run")

# Determine scenario selection: name, tags, or all
if ("${{ parameters.ScenarioName }}".Trim()) {
$cliArgs += "${{ parameters.ScenarioName }}".Trim()
} elseif ("${{ parameters.Tags }}".Trim()) {
$cliArgs += "--tags", "${{ parameters.Tags }}".Trim()
} else {
$cliArgs += "--all"
}

# Optional model override
if ("${{ parameters.Model }}".Trim()) {
$cliArgs += "--model", "${{ parameters.Model }}".Trim()
}

$cliArgs += "--parallel", "${{ parameters.Parallelism }}"
$cliArgs += "--cleanup", "always"

Write-Host "Running: dotnet run -- $($cliArgs -join ' ')"
dotnet run -- @cliArgs
env:
DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1
DOTNET_CLI_TELEMETRY_OPTOUT: 1
DOTNET_MULTILEVEL_LOOKUP: 0
COPILOT_GITHUB_TOKEN: $(azuresdk-copilot-github-pat)
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COPILOT_GITHUB_TOKEN is set from a specific secret variable name $(azuresdk-copilot-github-pat), but the template doesn’t define/require it as a parameter. This creates a hidden prerequisite and can break consumers that don’t have that variable configured. Prefer making the token an explicit (secret) parameter and only setting COPILOT_GITHUB_TOKEN when it’s non-empty.

Copilot uses AI. Check for mistakes.
Loading