Skip to content

Mock API Generator #19

Description

@ms-shashank

Current file: app/src/lib/tools/mock-api-generator.ts
Current model: qwen-3-coder-30b
Current approach: Single prompt asking LLM to generate mock records. No schema parsing, no constraint enforcement, no JSON validation.

Problems with current approach:

  • Does not reliably respect type constraints from the schema.
  • Edge cases (nulls, boundary values) are inconsistently generated.
  • Output JSON is sometimes malformed.
  • Generated data may not be internally consistent (e.g., foreign keys that do not reference valid primary keys).

Upgrade plan:

Step Agent Action
1 Schema Parser Programmatic: Parse JSON schema to extract field names, types, constraints (min/max, enum values, required, nullable).
2 Data Planner Plan data generation strategy: define realistic value distributions, identify edge cases to include, plan referential integrity.
3 Data Generator Generate mock records following the plan and schema constraints.
4 JSON Validator Programmatic: Validate output is well-formed JSON. Validate each record against original schema. Check referential integrity.
5 Repair Agent If validation fails, fix specific records that failed. Max 2 retries.
  • You are free to enhance the agents stacks in the above plan layout, the above one is just for reference. You can enhance more if needed.

Model suggestions to start with:

  • Step 2: Try deepseek-v3.2 for planning (lightweight reasoning).
  • Steps 3 and 5: Try qwen-3-coder-30b for data generation. Also try llama-3.3-70b or minimax-m2.5 and compare which produces more realistic data.

Model Selection Guidance

  • You are free to pick any model from the Oxlo catalog based on your own testing and evaluation.
  • The Models suggestions above, not mandates. Try them first, and if they do not meet the accuracy target, experiment with alternatives.

Compare against: GPT 5.3 Thinking & Claude Sonnet 4.6.

Acceptance criteria:

  • Output JSON must be valid in 100% of cases.
  • Schema constraint compliance at 95%+.
  • Edge cases (null, boundary, empty) present in every generated dataset.
  • Overall quality matches or exceeds GPT 5.3 Thinking/Claude Sonnet 4.6 on test cases.
  • Overall accuracy at 80%+.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions