Skip to content

Commit 9090bca

Browse files
localai-botlocalai-botclaude
authored
feat: Add documentation for undocumented API endpoints (mudler#8852)
* feat: add documentation for undocumented API endpoints Creates comprehensive documentation for 8 previously undocumented endpoints: - Voice Activity Detection (/v1/vad) - Video Generation (/video) - Sound Generation (/v1/sound-generation) - Backend Monitor (/backend/monitor, /backend/shutdown) - Token Metrics (/tokenMetrics) - P2P endpoints (/api/p2p/* - 5 sub-endpoints) - System Info (/system, /version) Each documentation file includes HTTP method, request/response schemas, curl examples, sample JSON responses, and error codes. * docs: remove token-metrics endpoint documentation per review feedback The token-metrics endpoint is not wired into the HTTP router and should not be documented per reviewer request. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: move system-info documentation to reference section Per review feedback, system-info endpoint docs are better suited for the reference section rather than features. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: localai-bot <localai-bot@noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent ec8f2d7 commit 9090bca

7 files changed

Lines changed: 665 additions & 0 deletions

File tree

‎docs/content/features/_index.en.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,10 @@ LocalAI provides a comprehensive set of features for running AI models locally.
1414
- **[Text Generation](text-generation/)** - Generate text with GPT-compatible models using various backends
1515
- **[Image Generation](image-generation/)** - Create images with Stable Diffusion and other diffusion models
1616
- **[Audio Processing](audio-to-text/)** - Transcribe audio to text and generate speech from text
17+
- **[Text to Audio](text-to-audio/)** - Generate speech from text with TTS models
18+
- **[Sound Generation](sound-generation/)** - Generate music and sound effects from text descriptions
19+
- **[Voice Activity Detection](voice-activity-detection/)** - Detect speech segments in audio data
20+
- **[Video Generation](video-generation/)** - Generate videos from text prompts and reference images
1721
- **[Embeddings](embeddings/)** - Generate vector embeddings for semantic search and RAG applications
1822
- **[GPT Vision](gpt-vision/)** - Analyze and understand images with vision-language models
1923

@@ -24,6 +28,7 @@ LocalAI provides a comprehensive set of features for running AI models locally.
2428
- **[Constrained Grammars](constrained_grammars/)** - Control model output format with BNF grammars
2529
- **[GPU Acceleration](GPU-acceleration/)** - Optimize performance with GPU support
2630
- **[Distributed Inference](distributed_inferencing/)** - Scale inference across multiple nodes
31+
- **[P2P API](p2p/)** - Monitor and manage P2P worker and federated nodes
2732
- **[Model Context Protocol (MCP)](mcp/)** - Enable agentic capabilities with MCP integration
2833
- **[Agents](agents/)** - Autonomous AI agents with tools, knowledge base, and skills
2934

@@ -34,6 +39,7 @@ LocalAI provides a comprehensive set of features for running AI models locally.
3439
- **[Stores](stores/)** - Vector similarity search for embeddings
3540
- **[Model Gallery](model-gallery/)** - Browse and install pre-configured models
3641
- **[Backends](backends/)** - Learn about available backends and how to manage them
42+
- **[Backend Monitor](backend-monitor/)** - Monitor backend status and resource usage
3743
- **[Runtime Settings](runtime-settings/)** - Configure application settings via web UI without restarting
3844

3945
## Getting Started

‎docs/content/features/backend-monitor.md‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
+++
2+
disableToc = false
3+
title = "Backend Monitor"
4+
weight = 20
5+
url = "/features/backend-monitor/"
6+
+++
7+
8+
LocalAI provides endpoints to monitor and manage running backends. The `/backend/monitor` endpoint reports the status and resource usage of loaded models, and `/backend/shutdown` allows stopping a model's backend process.
9+
10+
## Monitor API
11+
12+
- **Method:** `GET`
13+
- **Endpoints:** `/backend/monitor`, `/v1/backend/monitor`
14+
15+
### Request
16+
17+
The request body is JSON:
18+
19+
| Parameter | Type | Required | Description |
20+
|-----------|----------|----------|--------------------------------|
21+
| `model` | `string` | Yes | Name of the model to monitor |
22+
23+
### Response
24+
25+
Returns a JSON object with the backend status:
26+
27+
| Field | Type | Description |
28+
|----------------------|----------|-------------------------------------------------------|
29+
| `state` | `int` | Backend state: `0` = uninitialized, `1` = busy, `2` = ready, `-1` = error |
30+
| `memory` | `object` | Memory usage information |
31+
| `memory.total` | `uint64` | Total memory usage in bytes |
32+
| `memory.breakdown` | `object` | Per-component memory breakdown (key-value pairs) |
33+
34+
If the gRPC status call fails, the endpoint falls back to local process metrics:
35+
36+
| Field | Type | Description |
37+
|------------------|---------|--------------------------------|
38+
| `memory_info` | `object`| Process memory info (RSS, VMS) |
39+
| `memory_percent` | `float` | Memory usage percentage |
40+
| `cpu_percent` | `float` | CPU usage percentage |
41+
42+
### Usage
43+
44+
```bash
45+
curl http://localhost:8080/backend/monitor \
46+
-H "Content-Type: application/json" \
47+
-d '{"model": "my-model"}'
48+
```
49+
50+
### Example response
51+
52+
```json
53+
{
54+
"state": 2,
55+
"memory": {
56+
"total": 1073741824,
57+
"breakdown": {
58+
"weights": 536870912,
59+
"kv_cache": 268435456
60+
}
61+
}
62+
}
63+
```
64+
65+
## Shutdown API
66+
67+
- **Method:** `POST`
68+
- **Endpoints:** `/backend/shutdown`, `/v1/backend/shutdown`
69+
70+
### Request
71+
72+
| Parameter | Type | Required | Description |
73+
|-----------|----------|----------|---------------------------------|
74+
| `model` | `string` | Yes | Name of the model to shut down |
75+
76+
### Usage
77+
78+
```bash
79+
curl -X POST http://localhost:8080/backend/shutdown \
80+
-H "Content-Type: application/json" \
81+
-d '{"model": "my-model"}'
82+
```
83+
84+
### Response
85+
86+
Returns `200 OK` with the shutdown confirmation message on success.
87+
88+
## Error Responses
89+
90+
| Status Code | Description |
91+
|-------------|------------------------------------------------|
92+
| 400 | Invalid or missing model name |
93+
| 500 | Backend error or model not loaded |

‎docs/content/features/p2p.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
+++
2+
disableToc = false
3+
title = "P2P API"
4+
weight = 22
5+
url = "/features/p2p/"
6+
+++
7+
8+
LocalAI supports peer-to-peer (P2P) networking for distributed inference. The P2P API endpoints allow you to monitor connected worker and federated nodes, retrieve the P2P network token, and get cluster statistics.
9+
10+
For an overview of distributed inference setup, see [Distributed Inference](/features/distributed_inferencing/).
11+
12+
## Endpoints
13+
14+
### List all P2P nodes
15+
16+
- **Method:** `GET`
17+
- **Endpoint:** `/api/p2p`
18+
19+
Returns all worker and federated nodes in the P2P network.
20+
21+
#### Response
22+
23+
| Field | Type | Description |
24+
|--------------------|---------|--------------------------------------|
25+
| `nodes` | `array` | List of worker nodes |
26+
| `federated_nodes` | `array` | List of federated nodes |
27+
28+
Each node object:
29+
30+
| Field | Type | Description |
31+
|------------------|----------|------------------------------------------|
32+
| `Name` | `string` | Node name |
33+
| `ID` | `string` | Unique node identifier |
34+
| `TunnelAddress` | `string` | Network tunnel address |
35+
| `ServiceID` | `string` | Service identifier |
36+
| `LastSeen` | `string` | ISO 8601 timestamp of last heartbeat |
37+
38+
#### Usage
39+
40+
```bash
41+
curl http://localhost:8080/api/p2p
42+
```
43+
44+
#### Example response
45+
46+
```json
47+
{
48+
"nodes": [
49+
{
50+
"Name": "worker-1",
51+
"ID": "abc123",
52+
"TunnelAddress": "192.168.1.10:9090",
53+
"ServiceID": "worker",
54+
"LastSeen": "2025-01-15T10:30:00Z"
55+
}
56+
],
57+
"federated_nodes": [
58+
{
59+
"Name": "federation-1",
60+
"ID": "def456",
61+
"TunnelAddress": "192.168.1.20:9090",
62+
"ServiceID": "federated",
63+
"LastSeen": "2025-01-15T10:30:05Z"
64+
}
65+
]
66+
}
67+
```
68+
69+
---
70+
71+
### Get P2P token
72+
73+
- **Method:** `GET`
74+
- **Endpoint:** `/api/p2p/token`
75+
76+
Returns the P2P network token used for node authentication.
77+
78+
#### Usage
79+
80+
```bash
81+
curl http://localhost:8080/api/p2p/token
82+
```
83+
84+
#### Response
85+
86+
Returns the token as a plain text string.
87+
88+
---
89+
90+
### List worker nodes
91+
92+
- **Method:** `GET`
93+
- **Endpoint:** `/api/p2p/workers`
94+
95+
Returns worker nodes with online status.
96+
97+
#### Response
98+
99+
| Field | Type | Description |
100+
|--------------------------|----------|--------------------------------------|
101+
| `nodes` | `array` | List of worker nodes |
102+
| `nodes[].name` | `string` | Node name |
103+
| `nodes[].id` | `string` | Unique node identifier |
104+
| `nodes[].tunnelAddress` | `string` | Network tunnel address |
105+
| `nodes[].serviceID` | `string` | Service identifier |
106+
| `nodes[].lastSeen` | `string` | Last heartbeat timestamp |
107+
| `nodes[].isOnline` | `bool` | Whether the node is currently online |
108+
109+
A node is considered online if it was last seen within the past 40 seconds.
110+
111+
#### Usage
112+
113+
```bash
114+
curl http://localhost:8080/api/p2p/workers
115+
```
116+
117+
---
118+
119+
### List federated nodes
120+
121+
- **Method:** `GET`
122+
- **Endpoint:** `/api/p2p/federation`
123+
124+
Returns federated nodes with online status. Same response format as `/api/p2p/workers`.
125+
126+
#### Usage
127+
128+
```bash
129+
curl http://localhost:8080/api/p2p/federation
130+
```
131+
132+
---
133+
134+
### Get P2P statistics
135+
136+
- **Method:** `GET`
137+
- **Endpoint:** `/api/p2p/stats`
138+
139+
Returns aggregate statistics about the P2P cluster.
140+
141+
#### Response
142+
143+
| Field | Type | Description |
144+
|--------------------|----------|-----------------------------------|
145+
| `workers.online` | `int` | Number of online worker nodes |
146+
| `workers.total` | `int` | Total worker nodes |
147+
| `federated.online` | `int` | Number of online federated nodes |
148+
| `federated.total` | `int` | Total federated nodes |
149+
150+
#### Usage
151+
152+
```bash
153+
curl http://localhost:8080/api/p2p/stats
154+
```
155+
156+
#### Example response
157+
158+
```json
159+
{
160+
"workers": {
161+
"online": 3,
162+
"total": 5
163+
},
164+
"federated": {
165+
"online": 2,
166+
"total": 2
167+
}
168+
}
169+
```
170+
171+
## Error Responses
172+
173+
| Status Code | Description |
174+
|-------------|---------------------------------------------|
175+
| 500 | P2P subsystem not available or internal error |

0 commit comments

Comments
 (0)