Skip to content

Evaluator: transplant_instance_variables copies Minitest-internal ivars into class-scoped bindings #11

Description

@jcouball

Summary

Evaluator#transplant_instance_variables copies every instance variable from the spec instance — including Minitest-internal ones such as @__name__, @assertions, @__io__, and @time — into the class-scoped binding used to evaluate example code. Any example code that reads or writes those variable names gets or overwrites Minitest's own state, producing silent assertion count corruption or subtler test-isolation bugs.

Location

lib/yard_example_test/example/evaluator.rbtransplant_instance_variables

def transplant_instance_variables(ctx)
  @instance_variables.each do |ivar, value|
    local = "__yard_example_test__#{ivar.to_s.delete('@')}"
    ctx.local_variable_set(local, value)
    ctx.eval("#{ivar} = #{local}")
  end
end

@instance_variables is built from instance_variable_hash on the spec instance:

def instance_variable_hash
  instance_variables.to_h do |ivar|
    [ivar, instance_variable_get(ivar)]
  end
end

Minitest::Test sets several instance variables on itself before running each test. instance_variables enumerates all of them, so the snapshot includes at minimum:

Variable Purpose
@__name__ The test method name
@assertions Running assertion count
@__io__ I/O buffer for test output
@time Elapsed time accumulator

Consequences

  1. Assertion count corruption — if example code sets @assertions (e.g. a class method called assertions= that happens to write to @assertions in the binding), the counter is silently reset.
  2. Unexpected failures — if example code reads @__name__ expecting it to come from user-defined setup, it gets a Minitest method name instead.
  3. Any before-hook variable that collides with a Minitest internal name is silently overwritten by the snapshot taken after the hook runs — and then transplanted, potentially shadowing the internal value.

Proposed solution

Filter the snapshot to include only variables that were explicitly set by user-defined before hooks, not the full instance_variables set. One approach: snapshot instance variables before running hooks, then diff:

# in Example (spec instance method)
def instance_variable_hash
  minitest_ivars = Set[:@__name__, :@assertions, :@__io__, :@time]
  instance_variables.reject { |ivar| minitest_ivars.include?(ivar) }.to_h do |ivar|
    [ivar, instance_variable_get(ivar)]
  end
end

Alternatively, snapshot instance_variables before and after calling each before hook and only propagate the difference.

Acceptance criteria

  • Minitest-internal instance variables (@__name__, @assertions, etc.) are not transplanted into class-scoped bindings
  • Instance variables set by user before hooks are still available in evaluated code
  • The assertion counter reported by Minitest is not affected by example evaluation
  • All existing cucumber scenarios continue to pass

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions