Add Configurable LLM Provider & API Support

[KaranKumar2326]

Add Configurable LLM Provider & API Support

Summary

Introduce configurable LLM provider support so users can switch between different AI providers and models without modifying the core codebase.

Motivation

Currently, the project appears to be tightly coupled to a single provider/model setup. Adding a provider abstraction layer would improve flexibility, extensibility, and deployment options for different environments and use cases.

This would allow users to:

• Choose their preferred AI provider
• Experiment with different models
• Use self-hosted or custom endpoints
• Avoid vendor lock-in

Proposed Features

Multi-Provider Support

Support configurable providers such as:

• OpenAI
• Anthropic Claude
• Groq
• Google Gemini
• Ollama / local models (future enhancement)
• OpenAI-compatible APIs

Configurable Settings

Allow configuration through:

• Environment variables
• YAML/JSON/TOML config files
• CLI flags (optional)

Configurable Parameters

Users should be able to define:

• Provider name
• Model name
• API key
• Base URL
• Timeout/retry settings
• Temperature/max token settings

Example Configuration

llm:
  provider: openai
  model: gpt-4.1-mini
  api_key: ${OPENAI_API_KEY}
  base_url: https://api.openai.com/v1

Suggested Architecture

Introduce a common provider interface/adapter pattern.

type LLMProvider interface {
    GenerateResponse(prompt string) (string, error)
}

Each provider implementation can follow the same contract, making future integrations easier and keeping the architecture modular.

Benefits

• Cleaner separation of concerns
• Easier future integrations
• Better developer experience
• Improved extensibility
• More deployment flexibility
• Easier testing and mocking

Additional Notes

Supporting OpenAI-compatible APIs would automatically enable compatibility with many self-hosted and third-party providers.