Skip to content

Fix queue_duration metric to use queued_at instead of created_at#753

Open
Copilot wants to merge 6 commits intomainfrom
copilot/add-queued-at-timestamp-access
Open

Fix queue_duration metric to use queued_at instead of created_at#753
Copilot wants to merge 6 commits intomainfrom
copilot/add-queued-at-timestamp-access

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 16, 2026

GitHub support confirmed created_at is not the correct timestamp for measuring runner wait time — queued_at is when the job was actually placed in the runner queue. The previous formula (started_at - created_at) overstates queue duration.

Overview

Replace created_at with queued_at in the queue_duration calculation:

# Before
queue_duration = (job_info.started_at - job_info.created_at).total_seconds()

# After
queue_duration = (job_info.started_at - job_info.queued_at).total_seconds()

Rationale

Per GitHub support, queued_at is the timestamp when a job enters the runner queue. created_at precedes this and inflates the metric. The GitHub REST API (/repos/{owner}/{repo}/actions/runs/{run_id}/jobs) returns queued_at per job and is the documented way to measure runner wait time.

Juju Events Changes

None.

Module Changes

  • types_/github.py: Added queued_at: datetime to the JobInfo pydantic model.
  • platform/platform_provider.py: Added queued_at: datetime to the JobInfo dataclass.
  • github_client.py: _to_job_info() now extracts queued_at from the API response.
  • platform/github_provider.py: get_job_info() passes queued_at through to platform_provider.JobInfo.
  • metrics/github.py: queue_duration now computed as started_at - queued_at.

Library Changes

None.

Checklist

  • The charm style guide was applied.
  • The contributing guide was applied.
  • The changes are compliant with ISD054 - Managing Charm Complexity
  • The documentation for charmhub is updated.
  • The PR is tagged with appropriate label (urgent, trivial, complex).
  • The changelog is updated with changes that affects the users of the charm.
  • The application version number is updated in github-runner-manager/pyproject.toml.

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Co-authored-by: yanksyoon <37652070+yanksyoon@users.noreply.github.com>
Copilot AI changed the title [WIP] Add access to queued_at timestamps for scheduled events Fix queue_duration metric to use queued_at instead of created_at Mar 16, 2026
Copilot AI requested a review from yanksyoon March 16, 2026 05:38
yanksyoon and others added 2 commits March 23, 2026 07:14
The GitHub API may not always return queued_at for older jobs.
Changed to use .get() method and made queued_at optional in type
definitions. The metrics calculation now falls back to created_at
when queued_at is not available, maintaining backward compatibility.
@yanksyoon yanksyoon force-pushed the copilot/add-queued-at-timestamp-access branch from 1a9b8f6 to 0d564f9 Compare March 23, 2026 14:07
@yanksyoon yanksyoon marked this pull request as ready for review March 23, 2026 17:31
Copy link
Copy Markdown
Collaborator

@yhaliaw yhaliaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch.

One question to clarify things.

raise GithubMetricsError from exc

queue_duration = (job_info.started_at - job_info.created_at).total_seconds()
queued_or_created_at = job_info.queued_at or job_info.created_at
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a fail safe in case queued_at is missing?
Is it often that queued_at is missing?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, it's a failsafe - the field queued_at exists for job_info for all the jobs but is null from all the API responses that I've tested. I'm not entirely sure when it gets set to null and when it has a set value (the documentation does not exist), but this should set us back to the original behavior.

@yanksyoon
Copy link
Copy Markdown
Member

Quick GitHub API verification with list jobs & runs returns payload with queued_at field set to null. Enquired GitHub support again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants