Library prompt initialization order issue: "no agent pool" error during AgentPool.__init__

[Leoyzen]

Labels: bug, initialization, library-prompt

---

Description

When using type: library in system_prompt configuration, agentpool initialization fails with:

ValueError: Cannot resolve library prompt 'expert_analyst': no agent pool

This occurs during AgentPool.__init__ when creating agents from manifest configuration.

Reproduction Steps

1. Create an agentpool configuration with a library prompt reference:

agents:
  assistant:
    type: native
    system_prompt:
      - type: static
        content: |
          You are a helpful assistant.
      - type: library
        reference: expert_analyst  # ← library prompt reference
    # ... other config ...

2. Run agentpool:

uv run agentpool serve-opencode packages/xeno-agent/config/agentpool.yaml --port 7162

3. Error occurs during initialization:

ValueError: Cannot resolve library prompt 'expert_analyst': no agent pool

Error Details

Full traceback:

File "packages/agentpool/src/agentpool/agents/native_agent/agent.py", line 370, in from_config
    raise ValueError(msg)
ValueError: Cannot resolve library prompt 'expert_analyst': no agent pool

The error is triggered at agent.py:368:

case LibraryPromptConfig(reference=reference):
    if not agent_pool:  # ← agent_pool is None
        msg = f"Cannot resolve library prompt {reference!r}: no agent pool"
        raise ValueError(msg)

Root Cause Analysis

The issue stems from duplicate parsing of library prompts and timing/ordering problems in the initialization flow:

Initialization Flow

1. AgentPool.init (pool.py:74-165):

   # Line 142: Initialize prompt_manager
   self.prompt_manager = PromptManager(self.manifest.prompts)
   
   # Line 155: Create agents
   for name, config in self.manifest.agents.items():
       agent = self._create_agent_from_config(name, config, deps_type=shared_deps_type)
       agent.agent_pool = self  # ← agent_pool set AFTER agent creation
       self.register(name, agent)

2. _create_agent_from_config (pool.py:840-929):

   # Line 867-874: Parse LibraryPromptConfig
   case LibraryPromptConfig(reference=reference):
       try:
           content = self.prompt_manager.get.sync(reference)
           sys_prompts.append(content)
       except Exception as e:
           msg = f"Failed to load library prompt {reference!r} for agent {name}"
           logger.exception(msg)
           raise ValueError(msg) from e
   
   # Line 903-929: Call Agent() with system_prompt
   return Agent(
       model=model,
       model_settings=model_settings,
       system_prompt=sys_prompts,  # ← Should be resolved strings
       name=name,
       ...
       agent_pool=self,  # ← agent_pool IS passed
   )

3. Agent.from_config (agent.py:307-406):

   # Line 367-377: ALSO parses LibraryPromptConfig
   case LibraryPromptConfig(reference=reference):
       if not agent_pool:  # ← This check triggers error
           msg = f"Cannot resolve library prompt {reference!r}: no agent pool"
           raise ValueError(msg)
       content = agent_pool.prompt_manager.get.sync(reference)
       sys_prompts.append(content)

The Problem

Duplicate parsing logic exists in two places:

• _create_agent_from_config (pool.py:867-874) - uses self.prompt_manager
• Agent.from_config (agent.py:367-377) - uses agent_pool.prompt_manager

When _create_agent_from_config successfully resolves library prompts and passes them as a list[str] to Agent(), Agent.from_config parsing should not execute. However, if the system_prompt passed to Agent() contains LibraryPromptConfig objects (unresolved), then Agent.from_config will attempt to parse them, potentially hitting the "no agent pool" error if agent_pool is not properly set.

Potential Scenarios

1. Initialization order issue: If agent_pool is set after agent is created, and Agent.from_config is invoked during initialization with unresolved library prompts, error occurs.

2. Configuration object leakage: If _create_agent_from_config does not fully resolve all LibraryPromptConfig objects before passing to Agent(), unresolved objects leak into Agent.from_config.

Expected Behavior

Library prompts should be resolved successfully during AgentPool.__init__, and agents should be created without "no agent pool" errors.

Suggested Fix Options

Option 1: Ensure Full Resolution in _create_agent_from_config

Modify _create_agent_from_config to guarantee all library prompts are fully resolved before passing to Agent(), and ensure error handling is comprehensive.

Option 2: Improve Error Messages in Agent.from_config

When agent_pool is None, provide a clearer error message explaining that agents with library prompts should be created through AgentPool, not directly via Agent.from_config.

Option 3: Remove Duplicate Logic

Remove the library prompt parsing from _create_agent_from_config and handle it uniformly in Agent.from_config, ensuring agent_pool is always available when needed. However, this would require careful initialization ordering to guarantee agent_pool is set before any resolution is needed.

Additional Context

• Library prompts are defined in the YAML configuration with type: library and reference: <prompt_name>
• The manifest prompts are loaded via PromptManager(self.manifest.prompts) in AgentPool.__init__
• The issue occurs specifically during AgentPool.__init__ initialization phase

[phil65]

thanks. should be partially fixed by 82980c5, but I think theres another one. will take a look soon.

[Leoyzen]

Thanks for the explanation regarding the dependencies and newer language features.
To address your question about why we're staying at an older Python version: we're operating a production-grade system with a substantial stack of AI-related dependencies. The challenge is that many of these packages have version ceilings that don't go beyond Python 3.12, and attempting to run them on Python >=3.12 results in either runtime errors or missing wheel packages.
This isn't about preferring older versions—it's about the hard constraints imposed by our dependency ecosystem. Upgrading Python isn't "super easy" when it requires replacing or finding alternatives for multiple critical packages.
My suggestion isn't necessarily to go all the way back to 3.10, but rather to consider a more reasonable minimum version that balances:

• The ability to use modern Python features you need
• Compatibility with the broader AI/ML ecosystem
• Adoption by production teams who have real dependency constraints
  If this project aims to be widely adopted—especially as a framework that others will integrate into their systems—maintaining a sensible minimum version guarantee becomes important for real-world usability.
  Perhaps Python 3.11 could be a reasonable compromise? Many AI/ML packages support 3.11 well, and it's significantly newer than the alternatives while still providing broader compatibility.

[phil65]

As said, this would require me to downgrade the version constraints of probably more than 10 packages. its not as easy as you might imagine. which packages are holding you back? i guess its easier to send upstream PRs to these packages and make themm compatibe with newer python versions.