Add verbose modes. Add 1.11 support back. Various fixes. Add color#37
Open
IanButterworth wants to merge 9 commits into
Open
Add verbose modes. Add 1.11 support back. Various fixes. Add color#37IanButterworth wants to merge 9 commits into
IanButterworth wants to merge 9 commits into
Conversation
67e49a1 to
9117c7b
Compare
IanButterworth
commented
Jul 9, 2025
9117c7b to
a59a8bc
Compare
74051cd to
7aa42e4
Compare
Owner
|
Do you think this might be better as a filter rather than a printer? It seems you can do more if you pass back the subset of methods rather than just printing them. |
Collaborator
Author
|
I think there's uses for both. With "verbose" stuff it's kind of useful to just dump things into a text file etc. |
7aa42e4 to
00998ee
Compare
Add `verbose` modes to `info_cachefile` for controlling how much detail is
printed when displaying a `PkgCacheInfo`:
- `:none` - summary counts only (default)
- `:internal` - list internal methods and their specializations
- `:external` - list external methods and new specializations
- `:all` - list everything
Method, specialization, and external-method lists are sorted for stable
output, and external entries are filtered to those whose defining module
is not part of the package being inspected.
Internal-method counting is reworked to use `Core.methodtable` (renamed
from `Core.GlobalMethods` in Julia 1.12), with a fallback for older
nightlies.
Package lookup in `info_cachefile(pkg::PkgId, ...)` now uses
`Base.locate_package_load_spec` when available so the cache's recorded
syntax version is honored; this fixes spurious "all cache files are
stale" errors on Julia 1.13+ when the active manifest was resolved with
a different Julia version than the running one.
`parse_cache_header`'s return tuple changed shape on master (a new
`syntax_version` element was appended and `prefs`/`prefs_hash` was
replaced with `prefs_blob`), shifting `clone_targets` from index 7 to
index 6. Locate `clone_targets` by type (`Vector{UInt8}`) so the code
works on 1.12, 1.13, and nightly without version branching.
Set up `Project.toml` as a workspace with `test` and `docs` members
(adding `test/Project.toml`), and expand CI to cover 1.12, 1.12-nightly,
1.13-nightly, and nightly.
Co-authored-by: GitHub Copilot <copilot@github.com>
00998ee to
833288d
Compare
Extend the verbose-modes PR to cover Julia 1.11 as well as 1.12, 1.13 and nightly: - `_unpack_restored_sv` learns the 8-element 1.11 layout `(modules, init_order, extext_methods, new_ext_cis, method_roots_list, ext_targets, edges, cachesizes)`. - Guard `Core.GlobalMethods` / `Core.methodtable` access so 1.11 (which exposes neither publicly) falls through to the `info.internal_methods` source and otherwise reports no per-module counts rather than erroring at top level. - Gate the `ocachefile_from_cachefile` fallback on a non-empty `clone_targets`, matching upstream behavior when no pkgimage `.dylib` is associated with the `.ji` cache. - CI matrix: add `'1.11'`; keep `'1.12'`, `'1.13-nightly'`, `'nightly'`. - Compat: lower `julia` to `"1.11"`. - `docs/make.jl`: set `checkdocs=:exports` so internal helpers (`_unpack_restored_sv`, `_split_code_instances`, `show_verbose_*`) do not break the docs build. - `.gitignore`: exclude per-version root Manifests and `test/Manifest.toml`. Tested with `Pkg.test()` on 1.11, 1.12, 1.13.0-rc1 and 1.14.0-DEV; all 9 tests pass on every version. Co-authored-by: GitHub Copilot (Claude Opus 4.7) <noreply@github.com> Written with the assistance of generative AI (GitHub Copilot, Claude Opus 4.7).
88991e8 to
569beb8
Compare
`show_verbose_internal_methods` iterated `names(mod; all=true)` and called `methods(obj)` for each binding. When multiple gensym-aliased bindings referred to the same underlying function (e.g. `var"#15#val"` aliasing `normalize_hue`), each alias produced its own section, listing the same methods again under an unhelpful binding name. The summary count was correct but the per-function breakdown could repeat methods many times. Collect methods into a `Method`-keyed set first, group by the method's function-owner symbol, then render. Each Method is now listed exactly once, under its true function name. Co-authored-by: GitHub Copilot (Claude Opus 4.7) <noreply@github.com> Written with the assistance of generative AI (GitHub Copilot, Claude Opus 4.7).
Replace plain `print`/`println` of headers, counts, module names and function-name groupings with `printstyled` calls so the rendered `PkgCacheInfo` and `PkgCacheSizes` summaries highlight structure in color-capable terminals. `printstyled` is a no-op for non-color IO, so `sprint(show, info)` and other colorless contexts are unchanged. Render `Method` objects via their own color-aware `show` (preserving source-location highlighting) instead of `print`-stringifying them, and propagate the caller's `IOContext` into `Base.show_tuple_as_call` so specialization signatures pick up color too. Drop the buffered `IOContext(method_buffer, stdout)` indirection in the external-methods helper — it inherited `stdout`'s color regardless of the caller's IO. Co-authored-by: Claude <noreply@anthropic.com>
Loading the same pkgimage twice in one session via `jl_restore_package_image_from_file` / `jl_restore_incremental` corrupts the runtime and segfaults. Track previously inspected cache files in a module-local dict keyed by the realpath; on a repeat call, emit a warning and return the prior `PkgCacheInfo` (rebuilt with the freshly requested `verbose` setting) instead of re-invoking the loader. Co-authored-by: Claude <noreply@anthropic.com>
Apply review cleanup cluster:
- Tighten `PkgCacheInfo` element types from `Vector{Any}` to `Vector{Method}` /
`Vector{Core.CodeInstance}` for `external_methods`, `internal_methods`,
`internal_method_specializations`, and `new_specializations`. These fields
were already populated with the concrete types; the looser typing only cost
inference quality.
- Switch the public `info_cachefile` entry points to accept `verbose` as a
keyword argument so the docstring's promise (`; verbose=...`) is actually
honored for every signature.
- Make `count_module_specializations` defensive against non-`Method`-backed
`CodeInstance`s, matching the guards used elsewhere.
- Replace `return ArgumentError(...)` with `throw(ArgumentError(...))` on the
invalid-header / bad-CRC paths so callers see a real error.
- Extract `_truncate_for` to remove the duplicated `Base.rtruncate` block.
- Fix `!!! warn` -> `!!! warning` so Documenter renders the admonition.
- Drop the dead `moduleof(::Module)` method.
- Rename a `methods` loop variable that shadowed `Base.methods`.
- Remove a stray double blank line.
Co-authored-by: Claude <noreply@anthropic.com>
- Drop unused `assert_havelock` / `Core.SimpleVector` imports.
- Fix `auxillary` → `auxiliary` and "gets returns by" typos in
`PkgCacheSizes` docstrings.
- Collapse stale defensive `isa(_, Core.CodeInstance)` chains in the
three verbose/show helpers now that the corresponding fields are
typed `Vector{Core.CodeInstance}`. Drop the redundant
`mod ∉ info.modules` re-filter in `show_verbose_external_methods`
(`_split_code_instances` already partitions by module).
- Move the `_INFO_CACHE` lookup into the path-form
`info_cachefile(pkg, path; verbose)` entry, keyed on the resolved
load path. Repeat calls now short-circuit before opening the
cachefile, validating CRC, or pulling deps via
`_tryrequire_from_serialized`. Factored out `_with_verbose` /
`_cache_key` helpers; the inner 6-arg method keeps a defensive
lookup as a safety net.
Co-authored-by: Claude <noreply@anthropic.com>
- Rename `_n`/`_mod`/`_hdr` → `_count`/`_modname`/`_header` for self-documenting call sites. - Move the test-only zero-arg `PkgCacheSizes()` and two-arg `PkgCacheInfo(cachefile, modules)` constructors out of the source module and into `test/runtests.jl` as `empty_cachesizes()` / `empty_info(...)` test helpers. - Scope `Pkg.precompile()` in the test runner to `"Colors"` (the only package actually inspected by the tests). Co-authored-by: Claude <noreply@anthropic.com>
d264d5b to
a76aba3
Compare
On Julia post-#58131 (single global jl_method_table), info.external_methods contains *all* worklist-defined methods, not only those extending externally- owned functions. The summary line previously printed "N external methods" with the same count as "N internal methods", which was confusing and redundant. Changes: - Replace the "N external methods" summary line with "N methods extending external functions", computed via the new extending_external_methods helper, so the count reflects only the true external-extension subset. - Fix show_verbose_external_methods to filter through extending_external_methods instead of iterating all of info.external_methods. - Add an unexported extending_external_methods(info) helper. It filters info.external_methods to methods whose function-type owning module is outside the pkgimage's worklist modules. - Rewrite the docstrings for external_methods and internal_methods fields to accurately describe their post-#58131 semantics (both contain all worklist methods; they differ only in object form and load-time role). - Add four tests for extending_external_methods.
a76aba3 to
9f27d68
Compare
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This became a bit of a Claude take the wheel and fix everything...
Closes #41
Add verbose modes (and broader Julia compatibility)
Adds opt-in verbose output to
info_cachefilefor inspecting pkgimagecontents in detail, refreshes compatibility with current Julia versions
(1.11–nightly), and cleans up the surrounding code.
New
verbosekeyword oninfo_cachefileinfo_cachefile(pkg; verbose=:none)now controls how much detail isprinted when a
PkgCacheInfois shown::none:internal:external:all:internaland:externalMethod, specialization, and external-method lists are sorted for stable
output. External entries are filtered to those whose defining module is
not part of the package being inspected, so the section actually shows
extension of upstream functions rather than re-listing the package's
own methods.
Internal-method counting is reworked to walk
Core.methodtable(renamedfrom
Core.GlobalMethodsin 1.12) with a fallback toinfo.internal_methodsfor older versions where the API isn't public.Verbose rendering is colorized: headers, counts, module names, and the
function-name groupings are printed via
printstyledso the outputhighlights structure in color-capable terminals while remaining a no-op
under
sprintand other colorlessIOs.Methodobjects are renderedthrough their own color-aware
show(preserving source-locationhighlighting), and the caller's
IOContextis propagated throughBase.show_tuple_as_callso specialization signatures pick up colortoo.
Julia 1.11 / 1.12 / 1.13 / nightly compatibility
The cache-file plumbing has shifted shape across recent Julia versions;
this PR teaches the inspector to handle each:
(modules, init_order, extext_methods, new_ext_cis, method_roots_list, ext_targets, edges, cachesizes)), 1.12/1.13, and master layouts are all unpackedby a single
_unpack_restored_svhelper.parse_cache_header's return tuple grew asyntax_versionelement onmaster and replaced
prefs/prefs_hashwithprefs_blob, shiftingclone_targetsfrom index 7 to index 6. The PR locatesclone_targetsby type (Vector{UInt8}) so the code works on 1.12,1.13, and nightly without version branching.
Core.GlobalMethods/Core.methodtableaccess is guarded so 1.11(which exposes neither publicly) falls through to
info.internal_methodsinstead of erroring at top level.ocachefile_from_cachefileis only consulted whenclone_targetsisnon-empty, matching upstream behavior when no pkgimage
.dylibisassociated with a
.ji.info_cachefile(pkg::PkgId, …)usesBase.locate_package_load_specwhen available so the cache'srecorded syntax version is honored. This fixes spurious "all cache
files are stale" errors on Julia 1.13+ when the active manifest was
resolved with a different Julia version than the running one.
CI matrix expanded to
'1.11','1.12','1.13-nightly','nightly'.Project.tomlis now a workspace withtest/anddocs/members(adds
test/Project.toml), and thejuliacompat is lowered to"1.11".Repeat-call safety
Loading the same pkgimage twice in one session via
jl_restore_package_image_from_file/jl_restore_incrementalcorrupts the runtime and segfaults.
info_cachefilenow caches resultsin a module-local
_INFO_CACHEkeyed by the realpath of the cachefile; on a repeat call it emits a warning and returns the prior
PkgCacheInfo(re-wrapped with the freshly requestedverbosesetting) instead of re-invoking the loader. The fast-path runs before
any cache-file open, CRC check, or
_tryrequire_from_serializedcall.Code-quality cleanup
Folded into the same PR while the file was open:
PkgCacheInfofields move fromVector{Any}toconcrete element types (
Vector{Method}forexternal_methodsandinternal_methods,Vector{Core.CodeInstance}forinternal_method_specializationsandnew_specializations). Thefields were already populated with those concrete types; the loose
typing only cost inference quality.
info_cachefileentry point nowaccepts
verboseas a keyword argument, matching the docstring.count_module_specializationsand the verboserenderers gracefully skip non-
Method-backedCodeInstances.return ArgumentError(...)on the invalid-header /bad-CRC paths is now
throw(ArgumentError(...))so callers actuallysee a failure.
Method, not binding name.show_verbose_internal_methodsused to iterate
names(mod; all=true)and callmethods(obj)perbinding, so gensym aliases (e.g.
var"#15#val"aliasingnormalize_hue) duplicated whole sections. It now collects into aMethod-keyed set first and groups under the true function-ownersymbol, so each
Methodis listed exactly once.assert_havelockimport, the stub
moduleof(::Module)method, redundantisa(_, Core.CodeInstance)chains, and a double blank line. Fixesauxillary→auxiliaryand!!! warn→!!! warningso theDocumenter admonition renders. Sets
checkdocs=:exportsso internalhelpers don't break the docs build.
_n/_mod/_hdr→_count/_modname/_headerfor self-documenting call sites. The test-onlyzero-arg
PkgCacheSizes()and two-argPkgCacheInfo(cachefile, modules)constructors moved out of thesource module into
test/runtests.jlasempty_cachesizes()/empty_info(...).Pkg.precompile()in the test runner is scopedto
"Colors"(the only inspected package)..gitignore. Excludes per-version root Manifests andtest/Manifest.toml.Test plan
Pkg.test()passes on Julia 1.11, 1.12, 1.13.0-rc1, and1.14.0-DEV (9/9 tests on every version). Verbose rendering was
spot-checked against the Pkg, REPL, Dates, and Colors pkgimages on
1.14.0-DEV.
Commits
Add verbose modes and Julia 1.12/1.13/nightly compatibilitySupport Julia 1.11show_verbose_internal_methods: dedupe by Method, not binding nameshow: colorize output and use color-aware Method renderinginfo_cachefile: cache result to avoid double-load segfaultcleanup: tighten types, kwarg-ify verbose, defensive helperscleanup: prune dead code, tighten cache fast-path, fix typoscleanup: rename color helpers, move test-only ctors, scope precompile