feat: Add unified batch processing API

[bugproof]

• I have looked for existing issues (including closed) about this

Feature Request

e.g. https://platform.claude.com/docs/en/build-with-claude/batch-processing
https://ai.google.dev/gemini-api/docs/batch-api
https://platform.openai.com/docs/guides/batch

Motivation

To send asynchronous groups of requests with 50% lower costs, a separate pool of significantly higher rate limits, and a clear 24-hour turnaround time. The service is ideal for processing jobs that don't require immediate responses.

Proposal

Not as of now. I saw there is already JSONL decoder here so might be a starting point

rig/rig/rig-core/src/providers/anthropic/decoders/jsonl.rs

Line 1 in a9be758

//! JSONL is currently not used, it might be used when Anthropic batches beta feature is used.

Alternatives

No

[linear]

RIG-1134

[joshua-mo-143]

Looks good to me, didn't realise they'd added this already.

We should agree on what the initial design should look like first before implementing anything - do you have any ideas?

[bugproof]

Not yet but a similar abstraction exists in popular .NET project https://github.com/lofcz/LLMTornado so it might be a good source of inspiration for this implementation https://github.com/lofcz/LLMTornado/tree/d73c37ea10dd71e5506ab895bfda3f694b2a7b5a/src/LlmTornado/Batch

[joshua-mo-143]

Not yet but a similar abstraction exists in popular .NET project https://github.com/lofcz/LLMTornado so it might be a good source of inspiration for this implementation https://github.com/lofcz/LLMTornado/tree/d73c37ea10dd71e5506ab895bfda3f694b2a7b5a/src/LlmTornado/Batch

I guess the next question would be whether or not we need to support the Files API since OpenAI explicitly requires that a batch must be created using a file in their Files API.

I guess not for now, but if we support Batch then it's highly likely that the Files API will be useful to support

[joshua-mo-143]

Related: #1278

[lofcz]

Hey, just found this thread randomly - LLMTornado maintainer here. If you'd like to discuss anything about design of shared abstractions, feel free to reach out, discord: lofcz. I've been looking for something similar to LLMTornado for Rust (working on some Tauri apps at the moment) and Rig looks great! You all are doing a great work. ❤️