some suggestions + questions

[Greatz08]

Suggestion :

1. Set about/description for this project and also set some tags for this project related to cybersec so that cybersec people can easily find this project ( cant depend 100% only on google spider crawlers to always show reliable result :-)) )

2. Highlight Litellm in a better way and make people realize that THEY CAN USE 100+ LLM's supported by litellm easily with this project which includes proprietary models which team already mentioned but team didnt mentioned that with litellm people can also use open weights model supported by llama.cpp/ollama (open source community) which are also great at coding and many people might prefer them due to less cost, faster speed, privacy etc (running through powerful gpu or through groq,sambanova like services)

Question:

For a project that contains 100+ code files, with large codebases and many libraries, what would be the process of identifying and fixing vulnerabilities? Specifically:

Scope of Vulnerability Detection:

    Will the process only detect vulnerable third-party packages (e.g., using vulnerability databases)?

    Or will it also inspect and analyze the full codebase to detect bad implementations, insecure logic, or insecure coding patterns?

    Or will it do both?

Fixing Vulnerabilities:

    Will the system/tool automatically suggest or apply fixes for the vulnerabilities it finds?

    Or will it only report the issues for manual intervention?

Use of Retrieval-Augmented Generation (RAG):

    Will it use RAG or similar techniques (e.g., chunking and retrieval from long code files) to understand and reason about large codebases?

    If yes, how does RAG contribute to analyzing and fixing code vulnerabilities in this context?

Additional Capabilities:

    Are there any other techniques or steps involved in the process (e.g., static analysis, dynamic analysis, AI-based reasoning, etc.)?

    Does the system take into account code context, data flow, and execution paths to detect complex vulnerabilities?

(Btw i did used llm to organize my questions to make it more easier to read and understand :-) )

[onestardao]

interesting points. i think part of the tension here is that people often blur “what the tool promises” vs “what the workflow actually supports.” description/tags fix discoverability, but the deeper problem is expectation management.

on the vuln detection side, the hard bit isn’t just spotting issues — it’s when you start reasoning over large codebases. that’s where hallucination creeps in, or the system feels brittle. in my experience, the missing safeguard is a structured checklist to test claims systematically. we use a 16-problem map internally to pressure-test pipelines (things like drift, chunking misalignments, false positives in retrieval). it makes gaps show up fast.

curious if you’ve thought about evaluating your setup against a fixed set of failure modes instead of ad-hoc testing? it’s been a game changer for us in separating “nice demo” from “actually production-ready.”