diff --git a/PROMPTS_GUIDE.md b/PROMPTS_GUIDE.md new file mode 100644 index 0000000..f3a70eb --- /dev/null +++ b/PROMPTS_GUIDE.md @@ -0,0 +1,303 @@ +MCP Prompts Guide +----- + +This document provides detailed documentation for the 5 MCP prompts in the Parseable MCP server. + +# What are MCP Prompts? + +MCP Prompts are pre-built, multi-step workflows that guide AI agents through common tasks. +Instead of calling individual tools manually, prompts provide a structured approach to accomplishing complex operations. + +When an AI agent receives a prompt, it gets detailed instructions on: +- Which tools to call +- In what order +- What to analyze +- What to report back + +When using prompt, we can use natural language to ask questions related to our data: +- "Check the health of the otellogs stream" +- "Find errors in network_logstream from yesterday" +- "Compare otellogs and monitor_logstream" +- "Investigate the severity_text field" +- "Find anomalies in the last 24 hours" + +--- +# Available Prompts +Agents can use the following pre-built prompts to guide their analysis: + +## 1. analyze-errors + +**Purpose:** Find and analyze error logs in a data stream. + +**When to use:** +- Investigating system errors or failures +- Understanding error patterns over time +- Troubleshooting production issues + +**Required Arguments:** +- `streamName` - Name of the data stream to analyze +- `startTime` - Start time in ISO 8601 format (e.g., "2026-02-13T00:00:00Z") +- `endTime` - End time in ISO 8601 format + +**Optional Arguments:** +- `errorField` - Specific field to check for errors (default: "body") + +**What it does:** +1. Gets the stream schema to understand available fields +2. Queries for errors using pattern matching (error, exception, failed) +3. Analyzes error distribution over time +4. Identifies common error patterns and messages +5. Provides recommendations for investigation + +**Usage:** +``` +"Find errors in the otellogs stream from yesterday" +"Analyze errors in network_logstream from the last 24 hours" +``` + +--- +## 2. stream-health-check + +**Purpose:** Perform a comprehensive health assessment of a data stream. + +**When to use:** +- Regular stream monitoring +- Checking if data ingestion is working +- Verifying stream configuration +- Identifying potential issues + +**Required Arguments:** +- `streamName` - Name of the data stream to check + +**What it does:** +1. Gets stream information (creation time, event timestamps) +2. Gets stream statistics (ingestion rates, storage) +3. Gets stream schema +4. Analyzes: + - Stream activity status (active/inactive) + - Data freshness (time since last event) + - Ingestion rates (events per day/hour) + - Storage efficiency +5. Reports warnings or concerns +6. Provides optimization recommendations + +**Usage:** +``` +"Check the health of the otellogs stream" +"Is the network_logstream receiving data?" +``` + +--- +## 3. investigate-field + +**Purpose:** Deep dive into a specific field's values and patterns. + +**When to use:** +- Understanding field distributions +- Data quality analysis +- Finding most common values +- Checking for nulls or empty values + +**Required Arguments:** +- `streamName` - Name of the data stream +- `fieldName` - Name of the field to investigate +- `startTime` - Start time in ISO 8601 format +- `endTime` - End time in ISO 8601 format + +**What it does:** +1. Verifies the field exists in the schema +2. Counts total occurrences and unique values +3. Gets top 20 most common values with counts +4. Checks for null/empty values +5. Calculates percentages +6. Provides data quality assessment +7. Recommends filters or queries + +**Usage:** +``` +"Investigate the severity_text field in otellogs" +"Show me the distribution of the status field" +"What are the most common values for service_name?" +``` + +--- +## 4. compare-streams + +**Purpose:** Compare metrics across multiple data streams. + +**When to use:** +- Comparing production vs staging streams +- Understanding which streams are most active +- Capacity planning +- Identifying underutilized streams + +**Required Arguments:** +- `stream1` - First data stream name +- `stream2` - Second data stream name + +**Optional Arguments:** +- `stream3` - Optional third data stream name + +**What it does:** +1. Gets info and stats for each stream +2. Compares: + - Total events (lifetime count) + - Storage usage + - Ingestion rates (events per day) + - Data freshness (last event time) + - Activity levels +3. Creates comparison table +4. Identifies which stream is most active +5. Highlights concerning trends +6. Provides recommendations + +**Usage:** +``` +"Compare otellogs and network_logstream" +"Which stream is more active: prod-logs or staging-logs?" +"Compare the top 3 streams by size" +``` + +--- +## 5. find-anomalies + +**Purpose:** Detect unusual patterns, spikes, or drops in event volumes. + +**When to use:** +- Detecting system anomalies +- Finding traffic spikes +- Identifying data ingestion issues +- Monitoring for unusual activity + +**Required Arguments:** +- `streamName` - Name of the data stream to analyze +- `startTime` - Start time in ISO 8601 format +- `endTime` - End time in ISO 8601 format + +**Optional Arguments:** +- `groupBy` - Time grouping: "hour" or "day" (default: "hour") + +**What it does:** +1. Queries event counts are grouped by time (hour or day) +2. Calculates average event count (baseline) +3. Identifies spikes (significantly higher counts) +4. Identifies drops (significantly lower counts or gaps) +5. Investigates anomalous periods +6. Provides timeline with highlighted anomalies +7. Suggests potential causes +8. Recommends follow-up actions + +**Usage:** +``` +"Find anomalies in otellogs from the last week" +"Are there any spikes in network_logstream today?" +"Detect unusual patterns in the last 24 hours" +``` + +--- +# Using Prompts with AI Agents + +## In AI Agent Conversations + +When using MCP-compatible clients, you can reference prompts using natural language: + +**Health Checks:** +- "Check the health of otellogs" +- "Is network_logstream working?" +- "Run a health check on all my streams" + +**Error Analysis:** +- "Find errors in the last hour" +- "Analyze errors in otellogs from yesterday" +- "Show me all failed requests" + +**Field Investigation:** +- "What are the common values for severity_text?" +- "Investigate the user_id field" +- "Show me the distribution of status codes" + +**Stream Comparison:** +- "Compare otellogs and network_logstream" +- "Which stream has more data?" +- "Compare all production streams" + +**Anomaly Detection:** +- "Find any unusual activity" +- "Are there spikes in the last 24 hours?" +- "Detect anomalies this week" + + +## Testing Prompts + +To test prompts, see [TESTING.md](TESTING.md) for the complete testing guide using the stdio test script. + +--- +# Best Practices + +## Time Ranges + +- Start with shorter time ranges (1-24 hours) for faster results +- Expand to longer ranges if needed +- Use ISO 8601 format: `2026-02-14T00:00:00Z` + +## Field Names + +- Use `get_data_stream_schema` first to see available fields +- Field names are case-sensitive and fields with special characters like `.` or `-` must be quoted with backticks. +- Common fields: `body`, `severity_text`, `p_timestamp` + +## Stream Names + +- Use exact stream names (case-sensitive) +- List streams with `get_data_streams` tool if unsure +- Stream names in prompts must match exactly + +## Performance + +- **analyze-errors**: Limit to 100-1000 results +- **investigate-field**: Works best with shorter time ranges +- **find-anomalies**: Use `groupBy=day` for long time ranges +- **compare-streams**: Compare 2-3 streams at a time + +--- +# Troubleshooting + +**"Stream not found"** +- Verify stream name with `get_data_streams` tool +- Check spelling and case sensitivity + +**"No data returned"** +- Verify time range contains data +- Check stream has events in that period +- Use `get_data_stream_info` to see first/last event times + +**"Field not found"** +- Use `get_data_stream_schema` to see available fields +- Check field name spelling and case + +**Slow responses** +- Reduce time range +- Use hourly grouping instead of minute-level queries +- For anomalies, use `groupBy=day` for long ranges + +--- +# Next Steps + +1. Review [TESTING.md](TESTING.md) to test prompts +2. Configure Claude Desktop or another MCP client +3. Try prompts with natural language +4. Customize prompts in `prompts/prompts.go` for your needs +5. Create new prompts for your specific use cases + +--- +# Customizing Prompts + +Prompts are defined in `prompts/prompts.go`. You can: + +- Modify SQL queries in existing prompts +- Change default values (e.g., error patterns, field names) +- Add new arguments +- Create entirely new prompts + +Each prompt is a function that returns instructions for the AI agent. The agent then uses the MCP tools to execute those instructions. + diff --git a/README.md b/README.md index f36bfd4..57f78dc 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,19 @@ mcp-parseable-server - A Parseable MCP Server ---------------------- -> This project is currently in early development. Feedback and contributions are welcome. -> +> This project is currently in early development with focus on log data. +> Feedback and contributions are welcome. + > Testing has been done using: > - vscode with Github Copilot as the agent. -> - opencode agent CLI tool - works from v0.2.0 +> - opencode agent CLI tool +[TOC] +--- # Overview This project provides an MCP (Message Context Protocol) server for [Parseable](https://github.com/parseablehq/parseable), enabling AI agents and tools to interact with Parseable data streams (logs, metrics, traces) using natural language and structured tool calls. +--- # Features - List available data streams in Parseable - Query data streams using SQL @@ -19,48 +23,22 @@ This project provides an MCP (Message Context Protocol) server for [Parseable](h - Environment variable and flag-based configuration - The mcp server returns responses in json where the payload is both in text and structured format. -> In Parseable dataset and data stream names are used interchangeably, as Parseable's datasets are essentially -> named data streams. In all tool description we try to use the term data stream to avoid confusion with the term dataset which can have different + +> In Parseable dataset and data stream names are used interchangeably, as Parseable's datasets are essentially +> named data streams. In all tool description we try to use the term data stream to avoid confusion with the term dataset which can have different > meanings in other contexts. -# Testing -To test the server you can use the [mcp-cli](https://github.com/philschmid/mcp-cli) -```shell -mcp-cli call parseable get_roles -``` -Returns -```json -{ - "content": [ - { - "type": "text", - "text": "{\"admins\":[{\"privilege\":\"admin\"}],\"network_role\":[{\"privilege\":\"reader\",\"resource\":{\"stream\":\"network_logstream\"}}],\"otel_gateway\":[{\"privilege\":\"editor\"}]}" - } - ], - "structuredContent": { - "admins": [ - { - "privilege": "admin" - } - ], - "network_role": [ - { - "privilege": "reader", - "resource": { - "stream": "network_logstream" - } - } - ], - "otel_gateway": [ - { - "privilege": "editor" - } - ] - } -} +--- +# Limitations +## Authentication +The mcp server does not implement any authentication mechanisms. If this is needed, use a reverse proxy like nginx or envoy +in front of the mcp server to add authentication and authorization. -``` +## MCP session management +The mcp server does not implement any session management since all tool calls are stateless. This may change in the +future. +--- # Building Ensure you have Go 1.20+ installed. @@ -72,6 +50,7 @@ cd mcp-parseable-server go build -o mcp-parseable-server ./cmd/mcp_parseable_server ``` +--- # Running ## HTTP Mode (default) @@ -90,6 +69,7 @@ The MCP server will listen on `http://localhost:9034/mcp` for agent/tool request This mode is used for CLI or agent-to-agent workflows. +--- # Configuration You can configure the Parseable connection using environment variables or flags: @@ -105,8 +85,18 @@ Example: ```sh PARSEABLE_URL="http://your-parseable-host:8000" PARSEABLE_USER="admin" PARSEABLE_PASS="admin" ./mcp-parseable-server ``` +--- +# Production deployment +For production deployment use a reverse proxy like nginx or envoy in front of the mcp server that manages +authentication, authorization, and tls termination. -# MCP Tools +--- +# Testing + +See [TESTING.md](TESTING.md) for a complete testing guide. + +--- +# MCP Tools Reference ## 1. `query_data_stream` Execute a SQL query against a data stream. @@ -115,14 +105,14 @@ Execute a SQL query against a data stream. - `streamName`: Name of the data stream - `startTime`: ISO 8601 start time (e.g. 2026-01-01T00:00:00+00:00) - `endTime`: ISO 8601 end time -- **Returns:** Query result +- **Returns:** Query result and the row count returned ## 2. `get_data_streams` List all available data streams in Parseable. -- **Returns:** Array of stream names +- **Returns:** Array of stream objects with count ## 3. `get_data_stream_schema` -Get the schema for a specific data stream. +Get the fields schema for a specific data stream. - **Inputs:** - `stream`: Name of the data stream - **Returns:** Schema fields and types @@ -139,79 +129,84 @@ Get info for a data stream. - `streamName`: Name of the data stream - **Returns:** Info object (see tool description for details) -## 6. `get_parseable_about` +## 6. `get_about` Get Parseable about info. - **Returns:** About object (see tool description for details) -## 7. `get_parseable_roles` +## 7. `get_roles` Get Parseable roles. - **Returns:** Roles object (see tool description for details) -# Example: Querying with curl +## 8. `get_users` +Get all configured users. +- **Returns:** Users array with count -1. **List streams:** -```sh -curl -X POST http://localhost:8080/mcp \ - -H "Content-Type: application/json" \ - -d '{ - "jsonrpc": "2.0", - "id": 1, - "method": "callTool", - "params": { "tool": "list_data_streams", "arguments": {} } - }' -``` +--- +# MCP Prompts Reference -2. **Query a stream:** -```sh -curl -X POST http://localhost:8080/mcp \ - -H "Content-Type: application/json" \ - -d '{ - "jsonrpc": "2.0", - "id": 2, - "method": "callTool", - "params": { - "tool": "query_data_stream", - "arguments": { - "query": "SELECT COUNT(*) FROM log WHERE body ILIKE '%clamd%'", - "streamName": "otellogs", - "startTime": "2026-01-01T00:00:00+00:00", - "endTime": "2026-01-06T00:00:00+00:00" - } - } - }' -``` +The server provides 5 pre-built prompts for common workflows: -3. **Get schema for a stream:** -```sh -curl -X POST http://localhost:8080/mcp \ - -H "Content-Type: application/json" \ - -d '{ - "jsonrpc": "2.0", - "id": 3, - "method": "callTool", - "params": { - "tool": "get_data_stream_schema", - "arguments": { - "stream": "otellogs" - } - } - }' -``` +1. **analyze-errors** - Find and analyze error logs +2. **stream-health-check** - Perform comprehensive health assessment +3. **investigate-field** - Deep dive into field values and distributions +4. **compare-streams** - Compare metrics across multiple streams +5. **find-anomalies** - Detect unusual patterns and anomalies + +For detailed documentation and examples, see [PROMPTS_GUIDE.md](PROMPTS_GUIDE.md). +--- # Tool Discovery -Agents can discover all available tools and their input/output schemas via the MCP protocol. Each tool description includes details about returned fields and their meanings. +Agents can discover all available tools and their input/output schemas via the MCP protocol. +Each tool description includes details about returned fields and their meanings. +--- # Extending -To add new tools, create a new file in `tools/`, implement the registration function, and add it to `RegisterParseableTools` in `tools/register.go`. +To add new tools, create a new file in `tools/`, implement the registration function, and add it to +`RegisterParseableTools` in `tools/register.go`. +--- +# Troubleshooting + +## PARSEABLE_URL connection issues +Verify connection to Parseable: +```bash +curl -u admin: http://localhost:8000/api/v1/about +``` +## Agent times out on queries +- Reduce query time range +- Add LIMIT clauses to SQL queries +- Check if Parseable is responsive + +## Agent can't find streams +- Verify streams exist: ask agent to "list all streams" +- Check stream names are exact matches (case-sensitive) +- Verify Parseable has data in the streams + +## Empty prompt responses +Check that: +- Stream name exists and has data +- Time range includes actual events +- PARSEABLE_URL, USER, PASS are correct + +## Agent returns incomplete data +- Check that time range contains data +- Verify PARSEABLE_USER has permissions for the streams +- Use agent's troubleshooting capabilities to debug + +## MCP server not starting +Check: +- Port 9034 is available (for HTTP mode) +- Environment variables are set +- Go version is 1.20+ (`go version`) + +--- # License This work is licensed under the GNU GENERAL PUBLIC LICENSE Version 3. +--- # Todo -- No prompts or resources are currently included for agent usage. +- No resources are currently included for agent usage (prompts are implemented). - No tools to understand the Parseable setups. It can by using the `about` tool understand if cluster or standalone, but nothing about the configuration. -- No authentication mechanisms implemented. -- ..... \ No newline at end of file diff --git a/TESTING.md b/TESTING.md new file mode 100644 index 0000000..cb0bd57 --- /dev/null +++ b/TESTING.md @@ -0,0 +1,226 @@ +Testing MCP server +------------- +This guide shows you how to test the MCP tools and MCP prompts in your Parseable server. + +# Testing Tools with mcp-cli + +We use `mcp-cli` from https://github.com/philschmid/mcp-cli to test the MCP tools. + +```bash +# Install mcp-cli (philschmid/mcp-cli) +# Follow the install instructions in the repo: +# https://github.com/philschmid/mcp-cli +``` + +You can use `mcp-cli` to test the MCP tools: + +```bash +# Test get_roles tool +mcp-cli call parseable get_roles + +# Test get_data_streams tool +mcp-cli call parseable get_data_streams + +# Test query_data_stream tool +mcp-cli call parseable query_data_stream \ +'{"streamName": "a_logstream", "query": "select hostname, count(*) as count from a_logstream group by hostname limit 5", "startTime": "2026-02-12T17:00:00Z", "endTime":"2026-02-12T19:00:00Z"}' + +# Test get_data_stream_schema tool +mcp-cli call parseable get_data_stream_schema \ + '{"streamName": "a_logstream"}' +``` + +Example output from `mcp-cli call parseable get_roles`: +```json +{ + "content": [ + { + "type": "text", + "text": "{\"admins\":[{\"privilege\":\"admin\"}],\"network_role\":[{\"privilege\":\"reader\",\"resource\":{\"stream\":\"network_logstream\"}}]}" + } + ], + "structuredContent": { + "admins": [ + { + "privilege": "admin" + } + ], + "network_role": [ + { + "privilege": "reader", + "resource": { + "stream": "network_logstream" + } + } + ] + } +} +``` +--- +# Testing Prompts + +The following sections show how to test MCP prompts. When testing prompts with the test script, the server runs +in stdio mode, which allows you to see the full prompt instructions and responses directly in the terminal. + +## 1. Build the server +```bash +cd /home/andersh/github/mcp-parseable-server +go build -o mcp-parseable-server ./cmd/mcp_parseable_server +``` + +## 2. Run the test script +```bash +./examples/test-prompts-stdio.sh a_logstream 2026-02-01T00:00:00Z 2026-02-14T23:59:59Z +``` + +That's it! All 5 prompts will be tested in stdio mode. + +## Test Script Usage + +```bash +./examples/test-prompts-stdio.sh [stream-name] [start-time] [end-time] +``` + +**Arguments:** +- `stream-name` (optional): Name of your data stream (default: "otellogs") +- `start-time` (optional): ISO 8601 start time (default: "2026-02-13T00:00:00Z") +- `end-time` (optional): ISO 8601 end time (default: "2026-02-14T00:00:00Z") + +**Examples:** +```bash +# Use default stream and dates +./examples/test-prompts-stdio.sh + +# Test specific stream +./examples/test-prompts-stdio.sh pmeta + +# Test with custom date range +./examples/test-prompts-stdio.sh otellogs 2026-02-01T00:00:00Z 2026-02-14T23:59:59Z +``` + +## Expected Output + +``` +=== MCP Prompts Test Suite (stdio mode) === +Stream: network_logstream +Time Range: 2026-02-01T00:00:00Z to 2026-02-14T23:59:59Z + +[1/7] Initializing MCP connection... + ✓ Server initialized + +[2/7] Listing available prompts... + ✓ analyze-errors - Analyze error logs in a data stream... + ✓ stream-health-check - Perform a health check... + ✓ investigate-field - Investigate a specific field... + ✓ compare-streams - Compare metrics across streams... + ✓ find-anomalies - Find anomalies and unusual patterns... + +[3/7] Testing stream-health-check prompt... +You are performing a health check on the Parseable data stream "network_logstream". + +Follow these steps: +1. Get stream information using get_data_stream_info... +... + +=== All tests complete === +``` + +## Why stdio Mode for Testing? + +### HTTP Mode with SSE Transport + +The MCP server runs in HTTP mode using the StreamableHTTPServer, which uses Server-Sent Events (SSE) for transport. This requires session management, making simple curl testing complex. + +**How StreamableHTTPServer works:** +1. Client opens SSE connection: `GET /mcp` +2. Server sends session ID via SSE stream +3. Client includes session ID in subsequent POST requests +4. Server responds via SSE events + +This is handled automatically by MCP clients (Claude Desktop, etc.) but is complex to implement manually with curl/scripts. + +Note: `mcp-cli` does not support the MCP prompts feature. Use Claude Desktop or other MCP clients that support prompts +or use the stdio test script for testing. + +### stdio Mode Advantages + +- ✅ No session management needed +- ✅ Works with standard bash/jq +- ✅ Immediate results +- ✅ Perfect for CI/CD and testing + +## Understanding Prompt Responses + +**Important:** Prompts return **instructions** for AI agents, not actual data. + +Each prompt generates a detailed workflow that an AI agent (like Claude) would follow to complete the task. +The agent then uses the MCP tools to execute those instructions. + +Example response structure: +```json +{ + "result": { + "description": "Health check for otellogs", + "messages": [{ + "role": "user", + "content": { + "type": "text", + "text": "You are performing a health check...\n\nFollow these steps:\n1. Get stream information...\n2. Get stream statistics..." + } + }] + } +} +``` + +## Using Prompts with AI Agents + +### Claude Desktop + +Add to your Claude Desktop config: +- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json` +- Windows: `%APPDATA%\Claude\claude_desktop_config.json` +- Linux: `~/.config/Claude/claude_desktop_config.json` + +```json +{ + "mcpServers": { + "parseable": { + "command": "/path/to/mcp-parseable-server", + "args": ["--mode=stdio"], + "env": { + "PARSEABLE_URL": "http://localhost:8000", + "PARSEABLE_USER": "admin", + "PARSEABLE_PASS": "admin", + "LOG_LEVEL": "info" + } + } + } +} +``` + +### Other MCP Clients + +Any MCP-compatible client can use these prompts. The server works in both stdio and HTTP modes. +For HTTP mode, clients must properly handle SSE and session management. + + +## Manual Testing (Advanced) + +You can test manually by sending JSON-RPC via stdin: + +```bash +# List all prompts +{ + echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}' + echo '{"jsonrpc":"2.0","id":2,"method":"prompts/list","params":{}}' +} | ./mcp-parseable-server --mode=stdio +``` + +Test a specific prompt: +```bash +{ + echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}' + echo '{"jsonrpc":"2.0","id":2,"method":"prompts/get","params":{"name":"stream-health-check","arguments":{"streamName":"otellogs"}}}' +} | ./mcp-parseable-server --mode=stdio | tail -1 | jq -r '.result.messages[0].content.text' +``` + diff --git a/cmd/mcp_parseable_server/main.go b/cmd/mcp_parseable_server/main.go index 11df1a2..e31903d 100644 --- a/cmd/mcp_parseable_server/main.go +++ b/cmd/mcp_parseable_server/main.go @@ -7,6 +7,7 @@ import ( "github.com/mark3labs/mcp-go/server" + "mcp-pb/prompts" "mcp-pb/tools" ) @@ -98,6 +99,7 @@ Try not to second guess information - if you don't know something or lack inform ) tools.RegisterParseableTools(mcpServer) + prompts.RegisterParseablePrompts(mcpServer) if *mode == "stdio" { slog.Info("MCP server running in stdio mode", "parseable_url", tools.ParseableBaseURL) diff --git a/examples/test-prompts-stdio.sh b/examples/test-prompts-stdio.sh new file mode 100755 index 0000000..13eeb5f --- /dev/null +++ b/examples/test-prompts-stdio.sh @@ -0,0 +1,88 @@ +#!/bin/bash +# Test MCP Prompts using stdio mode +# Usage: ./test-prompts-stdio.sh [stream-name] [start-time] [end-time] + +set -e + +STREAM_NAME="${1:-otellogs}" +START_TIME="${2:-2026-02-13T00:00:00Z}" +END_TIME="${3:-2026-02-14T00:00:00Z}" + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +NC='\033[0m' # No Color + +echo -e "${GREEN}=== MCP Prompts Test Suite (stdio mode) ===${NC}" +echo "Stream: $STREAM_NAME" +echo "Time Range: $START_TIME to $END_TIME" +echo "" + +# Helper function to send JSON-RPC via stdio +send_rpc() { + local method="$1" + local params="$2" + + echo "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"$method\",\"params\":$params}" +} + +# Test 1: Initialize +echo -e "${YELLOW}[1/7] Initializing MCP connection...${NC}" +send_rpc "initialize" '{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}' | \ + ./mcp-parseable-server --mode=stdio | jq -r '.result.serverInfo.name' && echo -e "${GREEN} ✓ Server initialized${NC}" || echo -e "${RED} ✗ Failed${NC}" +echo "" + +# Test 2: List prompts +echo -e "${YELLOW}[2/7] Listing available prompts...${NC}" +{ + send_rpc "initialize" '{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}' + send_rpc "prompts/list" '{}' +} | ./mcp-parseable-server --mode=stdio | tail -1 | jq -r '.result.prompts[]? | " ✓ \(.name) - \(.description)"' +echo "" + +# Test 3: stream-health-check +echo -e "${YELLOW}[3/7] Testing stream-health-check prompt...${NC}" +{ + send_rpc "initialize" '{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}' + send_rpc "prompts/get" "{\"name\":\"stream-health-check\",\"arguments\":{\"streamName\":\"$STREAM_NAME\"}}" +} | ./mcp-parseable-server --mode=stdio | tail -1 | jq -r '.result.messages[0].content.text' | head -10 +echo "" + +# Test 4: analyze-errors +echo -e "${YELLOW}[4/7] Testing analyze-errors prompt...${NC}" +{ + send_rpc "initialize" '{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}' + send_rpc "prompts/get" "{\"name\":\"analyze-errors\",\"arguments\":{\"streamName\":\"$STREAM_NAME\",\"startTime\":\"$START_TIME\",\"endTime\":\"$END_TIME\"}}" +} | ./mcp-parseable-server --mode=stdio | tail -1 | jq -r '.result.messages[0].content.text' | head -10 +echo "" + +# Test 5: investigate-field +echo -e "${YELLOW}[5/7] Testing investigate-field prompt...${NC}" +{ + send_rpc "initialize" '{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}' + send_rpc "prompts/get" "{\"name\":\"investigate-field\",\"arguments\":{\"streamName\":\"$STREAM_NAME\",\"fieldName\":\"severity_text\",\"startTime\":\"$START_TIME\",\"endTime\":\"$END_TIME\"}}" +} | ./mcp-parseable-server --mode=stdio | tail -1 | jq -r '.result.messages[0].content.text' | head -10 +echo "" + +# Test 6: compare-streams +echo -e "${YELLOW}[6/7] Testing compare-streams prompt...${NC}" +{ + send_rpc "initialize" '{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}' + send_rpc "prompts/get" "{\"name\":\"compare-streams\",\"arguments\":{\"stream1\":\"otellogs\",\"stream2\":\"network_logstream\"}}" +} | ./mcp-parseable-server --mode=stdio | tail -1 | jq -r '.result.messages[0].content.text' | head -10 +echo "" + +# Test 7: find-anomalies +echo -e "${YELLOW}[7/7] Testing find-anomalies prompt...${NC}" +{ + send_rpc "initialize" '{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}' + send_rpc "prompts/get" "{\"name\":\"find-anomalies\",\"arguments\":{\"streamName\":\"$STREAM_NAME\",\"startTime\":\"$START_TIME\",\"endTime\":\"$END_TIME\",\"groupBy\":\"hour\"}}" +} | ./mcp-parseable-server --mode=stdio | tail -1 | jq -r '.result.messages[0].content.text' | head -10 +echo "" + +echo -e "${GREEN}=== All tests complete ===${NC}" +echo "" +echo "Note: The prompts return instructions for AI agents to follow." +echo "For HTTP mode testing, use an MCP-compatible client like Claude Desktop or mcp-cli." + diff --git a/prompts/prompts.go b/prompts/prompts.go new file mode 100644 index 0000000..d5edef9 --- /dev/null +++ b/prompts/prompts.go @@ -0,0 +1,307 @@ +package prompts + +import ( + "context" + + "github.com/mark3labs/mcp-go/mcp" + "github.com/mark3labs/mcp-go/server" +) + +// RegisterParseablePrompts registers all prompts with the MCP server +func RegisterParseablePrompts(mcpServer *server.MCPServer) { + registerAnalyzeErrorsPrompt(mcpServer) + registerStreamHealthCheckPrompt(mcpServer) + registerInvestigateFieldPrompt(mcpServer) + registerCompareStreamsPrompt(mcpServer) + registerFindAnomaliesPrompt(mcpServer) +} + +func registerAnalyzeErrorsPrompt(mcpServer *server.MCPServer) { + mcpServer.AddPrompt(mcp.NewPrompt( + "analyze-errors", + mcp.WithPromptDescription("Analyze error logs in a data stream over a time range. "+ + "Gets schema, queries for errors, and provides a summary with patterns and recommendations."), + mcp.WithArgument("streamName", mcp.RequiredArgument(), mcp.ArgumentDescription("Name of the data stream to analyze")), + mcp.WithArgument("startTime", mcp.RequiredArgument(), mcp.ArgumentDescription("Start time in ISO 8601 format (e.g., '2026-02-01T00:00:00Z')")), + mcp.WithArgument("endTime", mcp.RequiredArgument(), mcp.ArgumentDescription("End time in ISO 8601 format (e.g., '2026-02-13T23:59:59Z')")), + mcp.WithArgument("errorField", mcp.ArgumentDescription("Optional: specific field to check for errors (default: body)")), + ), func(ctx context.Context, request mcp.GetPromptRequest) (*mcp.GetPromptResult, error) { + args := request.Params.Arguments + streamName := args["streamName"] + startTime := args["startTime"] + endTime := args["endTime"] + errorField := args["errorField"] + if errorField == "" { + errorField = "body" + } + + promptText := `You are analyzing error logs in the Parseable data stream "` + streamName + `" from ` + startTime + ` to ` + endTime + `. + +Follow these steps: + +1. First, call get_data_stream_schema with streamName="` + streamName + `" to understand available fields. + +2. Query for errors using query_data_stream with: + - query: "SELECT * FROM ` + streamName + ` WHERE ` + errorField + ` ILIKE '%error%' OR ` + errorField + ` ILIKE '%exception%' OR ` + errorField + ` ILIKE '%failed%' ORDER BY p_timestamp DESC LIMIT 100" + - streamName: "` + streamName + `" + - startTime: "` + startTime + `" + - endTime: "` + endTime + `" + +3. Analyze the results and provide: + - Total number of errors found + - Common error patterns or messages + - Time distribution of errors (any spikes?) + - Affected components or services (if identifiable) + - Recommended next steps for investigation + +If no errors are found, confirm this and suggest checking different time ranges or search terms.` + + return mcp.NewGetPromptResult( + "Analyze errors in "+streamName, + []mcp.PromptMessage{ + mcp.NewPromptMessage(mcp.RoleUser, mcp.TextContent{ + Type: "text", + Text: promptText, + }), + }, + ), nil + }) +} + +func registerStreamHealthCheckPrompt(mcpServer *server.MCPServer) { + mcpServer.AddPrompt(mcp.NewPrompt( + "stream-health-check", + mcp.WithPromptDescription("Perform a health check on a data stream. "+ + "Analyzes ingestion rates, storage usage, and data freshness to identify potential issues."), + mcp.WithArgument("streamName", mcp.RequiredArgument(), mcp.ArgumentDescription("Name of the data stream to check")), + ), func(ctx context.Context, request mcp.GetPromptRequest) (*mcp.GetPromptResult, error) { + args := request.Params.Arguments + streamName := args["streamName"] + + promptText := `You are performing a health check on the Parseable data stream "` + streamName + `". + +Follow these steps: + +1. Get stream information using get_data_stream_info with streamName="` + streamName + `" + - Check createdAt, firstEventAt, and latestEventAt timestamps + - Verify if the stream is receiving recent data + +2. Get stream statistics using get_data_stream_stats with streamName="` + streamName + `" + - Review ingestion counts and sizes + - Compare lifetime vs current period metrics + - Check storage utilization + +3. Get stream schema using get_data_stream_schema with streamName="` + streamName + `" + - Verify schema structure is as expected + - Note important fields for querying + +4. Provide a health summary including: + - Stream status (active/inactive based on latestEventAt) + - Data ingestion rate (events per day/hour) + - Storage efficiency (compression ratio if detectable) + - Data freshness (time since last event) + - Any warnings or concerns (e.g., no recent data, rapid storage growth) + - Recommendations for optimization or investigation + +Present the findings in a clear, actionable format.` + + return mcp.NewGetPromptResult( + "Health check for "+streamName, + []mcp.PromptMessage{ + mcp.NewPromptMessage(mcp.RoleUser, mcp.TextContent{ + Type: "text", + Text: promptText, + }), + }, + ), nil + }) +} + +func registerInvestigateFieldPrompt(mcpServer *server.MCPServer) { + mcpServer.AddPrompt(mcp.NewPrompt( + "investigate-field", + mcp.WithPromptDescription("Investigate a specific field in a data stream. "+ + "Analyzes field values, distributions, and patterns over a time range."), + mcp.WithArgument("streamName", mcp.RequiredArgument(), mcp.ArgumentDescription("Name of the data stream")), + mcp.WithArgument("fieldName", mcp.RequiredArgument(), mcp.ArgumentDescription("Name of the field to investigate")), + mcp.WithArgument("startTime", mcp.RequiredArgument(), mcp.ArgumentDescription("Start time in ISO 8601 format")), + mcp.WithArgument("endTime", mcp.RequiredArgument(), mcp.ArgumentDescription("End time in ISO 8601 format")), + ), func(ctx context.Context, request mcp.GetPromptRequest) (*mcp.GetPromptResult, error) { + args := request.Params.Arguments + streamName := args["streamName"] + fieldName := args["fieldName"] + startTime := args["startTime"] + endTime := args["endTime"] + + promptText := `You are investigating the field "` + fieldName + `" in the Parseable data stream "` + streamName + `" from ` + startTime + ` to ` + endTime + `. + +Follow these steps: + +1. Verify the field exists using get_data_stream_schema with streamName="` + streamName + `" + - Confirm field name and data type + - If the field name include space, dot or special characters, escape them with backticks + +2. Get field statistics using query_data_stream: + - query: "SELECT COUNT(*) as total_count, COUNT(DISTINCT ` + fieldName + `) as unique_values FROM ` + streamName + `" + - This shows how many total records and unique values exist + +3. Get top values using query_data_stream: + - query: "SELECT ` + fieldName + `, COUNT(*) as count FROM ` + streamName + ` GROUP BY ` + fieldName + ` ORDER BY count DESC LIMIT 20" + - This shows the most common values + +4. Check for nulls/empty values using query_data_stream: + - query: "SELECT COUNT(*) FROM ` + streamName + ` WHERE ` + fieldName + ` IS NULL OR ` + fieldName + ` = ''" + +5. Analyze and report: + - Total occurrences and unique values + - Top 10 most common values with counts + - Percentage of null/empty values + - Any unusual patterns or outliers + - Data quality assessment + - Recommendations for queries or filters + +Present findings in a structured, easy-to-read format.` + + return mcp.NewGetPromptResult( + "Investigate field "+fieldName+" in "+streamName, + []mcp.PromptMessage{ + mcp.NewPromptMessage(mcp.RoleUser, mcp.TextContent{ + Type: "text", + Text: promptText, + }), + }, + ), nil + }) +} + +func registerCompareStreamsPrompt(mcpServer *server.MCPServer) { + mcpServer.AddPrompt(mcp.NewPrompt( + "compare-streams", + mcp.WithPromptDescription("Compare metrics across multiple data streams. "+ + "Useful for understanding relative activity, storage usage, and data patterns."), + mcp.WithArgument("stream1", mcp.RequiredArgument(), mcp.ArgumentDescription("First data stream name")), + mcp.WithArgument("stream2", mcp.RequiredArgument(), mcp.ArgumentDescription("Second data stream name")), + mcp.WithArgument("stream3", mcp.ArgumentDescription("Optional third data stream name")), + ), func(ctx context.Context, request mcp.GetPromptRequest) (*mcp.GetPromptResult, error) { + args := request.Params.Arguments + stream1 := args["stream1"] + stream2 := args["stream2"] + stream3 := args["stream3"] + + streams := `"` + stream1 + `" and "` + stream2 + `"` + if stream3 != "" { + streams = `"` + stream1 + `", "` + stream2 + `", and "` + stream3 + `"` + } + + promptText := `You are comparing the following Parseable data streams: ` + streams + `. + +Follow these steps for each stream: + +1. Get stream info using get_data_stream_info + - Note creation time and event timestamps + - Check stream type and telemetry type + +2. Get stream stats using get_data_stream_stats + - Record ingestion counts and sizes + - Note storage metrics + +3. Compare and analyze: + - Ingestion rates (events per day) + - Storage usage and growth + - Data freshness (time since last event) + - Ingestion format and storage format + - Activity level (active vs dormant) + +4. Present a comparison table with: + - Stream names + - Total events (lifetime_count) + - Storage size (human-readable) + - Average event size + - Events per day (calculated) + - Last event time + - Status (active/inactive) + +5. Provide insights: + - Which stream is most active? + - Which uses most storage? + - Any concerning trends (e.g., stream stopped receiving data)? + - Recommendations for stream management + +Format the comparison clearly for easy decision-making.` + + return mcp.NewGetPromptResult( + "Compare data streams", + []mcp.PromptMessage{ + mcp.NewPromptMessage(mcp.RoleUser, mcp.TextContent{ + Type: "text", + Text: promptText, + }), + }, + ), nil + }) +} + +func registerFindAnomaliesPrompt(mcpServer *server.MCPServer) { + mcpServer.AddPrompt(mcp.NewPrompt( + "find-anomalies", + mcp.WithPromptDescription("Find anomalies and unusual patterns in a data stream over time. "+ + "Looks for spikes, drops, and irregular patterns in event volumes."), + mcp.WithArgument("streamName", mcp.RequiredArgument(), mcp.ArgumentDescription("Name of the data stream to analyze")), + mcp.WithArgument("startTime", mcp.RequiredArgument(), mcp.ArgumentDescription("Start time in ISO 8601 format")), + mcp.WithArgument("endTime", mcp.RequiredArgument(), mcp.ArgumentDescription("End time in ISO 8601 format")), + mcp.WithArgument("groupBy", mcp.ArgumentDescription("Time grouping: 'hour' or 'day' (default: hour)")), + ), func(ctx context.Context, request mcp.GetPromptRequest) (*mcp.GetPromptResult, error) { + args := request.Params.Arguments + streamName := args["streamName"] + startTime := args["startTime"] + endTime := args["endTime"] + groupBy := args["groupBy"] + if groupBy == "" { + groupBy = "hour" + } + + // Determine the date_trunc interval + interval := "hour" + if groupBy == "day" { + interval = "day" + } + + promptText := `You are analyzing the Parseable data stream "` + streamName + `" for anomalies from ` + startTime + ` to ` + endTime + `. + +Follow these steps: + +1. Get event counts over time using query_data_stream: + - query: "SELECT date_trunc('` + interval + `', p_timestamp) as time_bucket, COUNT(*) as event_count FROM ` + streamName + ` GROUP BY time_bucket ORDER BY time_bucket" + - streamName: "` + streamName + `" + - startTime: "` + startTime + `" + - endTime: "` + endTime + `" + +2. Analyze the time series data: + - Calculate average event count per ` + interval + ` + - Identify spikes (periods with significantly higher counts) + - Identify drops (periods with significantly lower counts or zero events) + - Look for irregular patterns or gaps + +3. For any anomalies found, investigate further: + - Query sample events from anomalous periods + - Check if specific fields or patterns are different + +4. Report findings: + - Summary of normal activity baseline + - List of anomalies with timestamps and severity + - Potential causes or correlations + - Recommended actions (e.g., investigate specific time windows, check for system issues) + +Present a clear timeline of activity with highlighted anomalies.` + + return mcp.NewGetPromptResult( + "Find anomalies in "+streamName, + []mcp.PromptMessage{ + mcp.NewPromptMessage(mcp.RoleUser, mcp.TextContent{ + Type: "text", + Text: promptText, + }), + }, + ), nil + }) +}