Add support for Artifactory and Github Packages gem sources#1
Open
soulcutter wants to merge 8 commits into
Open
Add support for Artifactory and Github Packages gem sources#1soulcutter wants to merge 8 commits into
soulcutter wants to merge 8 commits into
Conversation
Parses Gemfile.lock to detect gem sources and queries appropriate APIs instead of hardcoding rubygems.org for all gems. Supports GitHub Packages via gh CLI when available, gracefully skipping private gems otherwise. Eliminates 404 errors when analyzing projects with gems from private registries or GitHub Packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Separated out the `gh`-based and rubygems-based implementations of Gemfile sources. This exposed a place where caching always used RubyGems, so that also got fixed in these changes. Mostly this behaves the same as before otherwise.
This was common behavior in every GemSource implementation, so exposed it in the `GemSource::Base` class api.
This also adds documentation in the README and some documentation on the general-purpose classes that are relevant to implementations of different gem sources.
When a Gemfile is explicitly passed, it is used to configure Bundler so that resolving the Gemfile and Gemfile.lock paths is always consistent with Bundler. When no Gemfile is explicitly passed, it uses Bundler's default behavior. Before these changes there was an inconsistency where the implicit Gemfile behavior was different (and wrong) compared to the explicit Gemfile behavior. This ensures it's both consistent and correct.
versions_sequence in Unsupported, GithubPackages, and Artifactory now report the problem and return [] instead of raising, so one bad source can't abort the report. Also fix two --releases crashes against Artifactory: - extract_version parsed the first hyphen-separated token from each matching artifact, so unrelated gems sharing a prefix (e.g. `datadog-ruby_core_source.gem` matching `datadog-*.gem`) produced invalid versions that blew up Gem::Version.new. Parsing now picks the longest leading substring accepted by Gem::Version.correct?. - VersionSequenceDelta.calculate returned nil - nil (TypeError) when a version wasn't in the sequence; it now returns 0. Reports::Console#put_line_summary rescues per-metric errors, matching Reports::Base#to_h, so a single bad gem can't abort the line summary. Expose GemSource::Base#problem_reporter= for IO injection.
soulcutter
force-pushed
the
artifactory-support
branch
from
b161ed3 to
fa8b28b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
This tool assumes that https://rubygems.org is always the gem source. As a result gems that are hosted on a different registry that are not in rubygems.org - such as private gems - will not be counted by the tool.
What this does
Based on @bryan-ash 's work in jaredbeck#55 to support Github Packages, this refactors the internal api towards being extensible to other gem sources, and then uses that to add support for Artifactory using the private API token configured in bundler.
GemSource Architecture
Most changes are on the lefthand side of this diagram where the data is supplied for the pre-existing reporting calculations.
flowchart LR URL[/"source_url<br/>(from Gemfile.lock)"/] Factory{{"GemSource.for(source_url)"}} subgraph Adapters["GemSource adapters"] RG[Rubygems] AF[Artifactory] GP[GithubPackages] UN[Unsupported] end Gem["Models::Gem"] Report["Reports::Base<br/>(Console / JSON)"] URL --> Factory Factory -- rubygems.org --> RG Factory -- "*.jfrog.io" --> AF Factory -- rubygems.pkg.github.com --> GP Factory -- other --> UN Adapters -- "release_date(name, version)<br/>versions_sequence(name)" --> Gem Gem -- "name<br/>libyears<br/>version_number_delta<br/>version_sequence_delta" --> ReportThis is a lot of new code
The refactor introduces a lot of new tests for new parts of the API, and the other big part is that the Artifactory implementation itself is a bit complicated. There's no getting around this being a significant architecture change of the internals.
Future
We should contribute this upstream to https://github.com/jaredbeck/libyear-bundler . This is fork intended to be temporary, but cannot predict if it will be accepted or when it might be considered. And maybe this isn't fully baked yet.