Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@

# Documentation
*.md @czero-cc
/docs/ @czero-cc

# Core functionality
/src/core/ @czero-cc
Expand Down
10 changes: 6 additions & 4 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,12 @@ Fixes #(issue number)
## 🧪 Testing

<!-- How did you test your changes? -->
- [ ] Ran `uv run pytest`
- [ ] Tested intelligent scraper with: `scapo scrape run --sources [source] --limit 5`
- [ ] Verified LLM processing worked correctly
- [ ] Checked that only AI/ML content was processed
- [ ] Tested service discovery: `uv run scapo scrape discover --update`
- [ ] Tested targeted scraping: `uv run scapo scrape targeted --service "[service]" --limit 10`
- [ ] Tested batch processing: `uv run scapo scrape batch --category [category] --limit 10 --batch-size 2`
- [ ] Verified LLM extraction quality (checked generated files in `models/`)
- [ ] Tested TUI explorer: `uv run scapo tui`
- [ ] For OpenRouter users: Updated context cache with `uv run scapo update-context`
- [ ] Other testing (please describe):

## 📸 Screenshots (if applicable)
Expand Down
30 changes: 23 additions & 7 deletions QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ cp .env.example .env
LLM_PROVIDER=openrouter
OPENROUTER_API_KEY=sk-or-v1-your-key-here # Get from openrouter.ai
OPENROUTER_MODEL=your_model

# IMPORTANT: Update model context cache for accurate batching
scapo update-context # Without this, defaults to 4096 tokens (poor performance!)
```

#### Option B: Ollama (Local)
Expand Down Expand Up @@ -117,27 +120,27 @@ scapo models search "copilot" # Search for specific models
cat models/audio/eleven-labs/cost_optimization.md
```

### 5. (Optional) Use with Claude Desktop
### 5. (Optional) MCP Server - Query Your Extracted Tips

**Important:** The MCP server only reads tips you've already extracted. Run scrapers first (Steps 3-4) to populate models/ folder!

Add SCAPO as an MCP server to query your extracted tips (from models/ folder) directly in Claude:
Add SCAPO as an MCP server to query your extracted tips directly in MCP-compatible clients:

```json
// Add to claude_desktop_config.json
// Add to config.json
{
"mcpServers": {
"scapo": {
"command": "npx",
"args": ["@scapo/mcp-server"],
"args": ["@arahangua/scapo-mcp-server"],
"env": {
"SCAPO_MODELS_PATH": "path/to/scapo/models"
"SCAPO_MODELS_PATH": "/path/to/scapo/models"
}
}
}
}
```

Then ask Claude: "Get best practices for Midjourney" - no Python needed!

## 📊 Understanding the Output

SCAPO creates organized documentation:
Expand All @@ -151,6 +154,19 @@ models/
│ └── parameters.json # Recommended settings
```

## ⚙️ Utility Commands

```bash
# Update OpenRouter model context cache (for accurate batching)
scapo update-context # Updates if >24h old
scapo update-context -f # Force update

# View extracted tips
scapo tui # Interactive TUI explorer
scapo models list # List all extracted models
scapo models search "copilot" # Search for specific models
```

## ⚙️ The --limit flag

```bash
Expand Down
28 changes: 17 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,13 @@ cp .env.example .env
Get your API key from [openrouter.ai](https://openrouter.ai/)
* you can also use local LLMs (Ollama, LMstudio). Check [QUICKSTART.md](./QUICKSTART.md)

#### Important: Update Model Context Cache (OpenRouter users)
```bash
# REQUIRED for optimal performance - fetches accurate token limits
scapo update-context # Creates cache for faster processing
```
Without this, SCAPO defaults to 4096 tokens (severely limiting batch efficiency)


### 3. Start Extracting Optimization Tips

Expand Down Expand Up @@ -274,28 +281,24 @@ MAX_POSTS_PER_SCRAPE=100 # Limit per source
```
Hand-wavy breakdown: With 5 posts, extraction success ~20%. With 20+ posts, success jumps to ~80%.

## 🤖 MCP Server for Claude Desktop
## 🤖 MCP Server (Optional Reader)

Query your extracted tips directly in Claude (reads from models/ folder - run scrapers first!):
**Note:** The MCP server is a reader that queries your already-extracted tips. You must run SCAPO scrapers first to populate the models/ folder!

```json
// Add to %APPDATA%\Claude\claude_desktop_config.json (Windows)
// or ~/Library/Application Support/Claude/claude_desktop_config.json (macOS)
// Add to your client's mcp config.json
{
"mcpServers": {
"scapo": {
"command": "npx",
"args": ["@scapo/mcp-server"],
"args": ["@arahangua/scapo-mcp-server"],
"env": {
"SCAPO_MODELS_PATH": "C:\\path\\to\\scapo\\models" // Your models folder
}
}
}
}
```

Then ask Claude: "Get me best practices for GitHub Copilot" or "What models are good for coding?"

See [mcp/README.md](mcp/README.md) for full setup and available commands.

## 🎨 Interactive TUI
Expand Down Expand Up @@ -361,10 +364,8 @@ Help us build the community knowledge base for AI service optimization!

## 📚 Documentation

- [Configuration Guide](docs/CONFIGURATION.md)
- [Quick Start Guide](QUICKSTART.md)
- [Contributing Guide](CONTRIBUTING.md)
- [Add New Source Tutorial (legacy method)](docs/ADD_NEW_SOURCE.md)

## 📜 License

Expand All @@ -378,7 +379,12 @@ Built as part of the CZero Engine project to improve AI application development.
- [OpenRouter](https://openrouter.ai/) for accessible AI APIs
- Coffee ☕ for making this possible
- [Ollama](https://ollama.com/) and [LMstudio](https://lmstudio.ai/) for awesome local LLM experience
- [Awesome Generative AI](https://github.com/steven2358/awesome-generative-ai) & [Awesome AI Tools](https://github.com/mahseema/awesome-ai-tools) for service discovery
- Service discovery powered by awesome lists:
- [steven2358/awesome-generative-ai](https://github.com/steven2358/awesome-generative-ai)
- [mahseema/awesome-ai-tools](https://github.com/mahseema/awesome-ai-tools)
- [filipecalegario/awesome-generative-ai](https://github.com/filipecalegario/awesome-generative-ai)
- [aishwaryanr/awesome-generative-ai-guide](https://github.com/aishwaryanr/awesome-generative-ai-guide)
- [eudk/awesome-ai-tools](https://github.com/eudk/awesome-ai-tools)
- All opensource contributors in this space

---
Expand Down
161 changes: 0 additions & 161 deletions docs/ADD_NEW_SOURCE.md

This file was deleted.

Loading