Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 83 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ CLI and data enrichment utilities for the [Parallel API](https://docs.parallel.a
- **Content Extraction** - Extract clean markdown from any URL
- **Data Enrichment** - Enrich CSV, JSON, DuckDB, and BigQuery data with AI
- **Follow-up Context** - Chain research and enrichment tasks using `--previous-interaction-id`
- **Research Memory** - Recall, scope, evict, or clear saved Task, Monitor, and FindAll work
- **AI-Assisted Planning** - Use natural language to define what data you want
- **Multiple Integrations** - Polars, DuckDB, Snowflake, BigQuery, Spark

Expand Down Expand Up @@ -97,14 +98,18 @@ parallel-cli
│ ├── extend # Request additional candidates for a run
│ ├── schema # Get the schema for a FindAll run
│ └── cancel # Cancel a running FindAll
└── monitor # Continuous web change tracking
├── create # Create a new web monitor (event_stream or snapshot)
├── list # List monitors (cursor paginated)
├── get # Get monitor details
├── update # Update frequency, webhook, metadata
├── cancel # Cancel a monitor (irreversible)
├── events # List events for a monitor
└── trigger # Trigger an immediate one-off run
├── monitor # Continuous web change tracking
│ ├── create # Create a new web monitor (event_stream or snapshot)
│ ├── list # List monitors (cursor paginated)
│ ├── get # Get monitor details
│ ├── update # Update frequency, webhook, metadata
│ ├── cancel # Cancel a monitor (irreversible)
│ ├── events # List events for a monitor
│ └── trigger # Trigger an immediate one-off run
└── memory # Saved Task, Monitor, and FindAll entries
├── retrieve # Search Memory or list recent entries
├── evict # Remove one entry from Memory
└── clear # Permanently clear selected Memory
```

## Quick Start
Expand Down Expand Up @@ -189,6 +194,43 @@ parallel-cli enrich run \
parallel-cli enrich deploy --system bigquery --project my-gcp-project
```

### 6. Recall Memory Entries

Memory is an explicit recall layer for completed Task, Monitor, and FindAll work.
Saving is asynchronous, so a newly completed run may take a short time to appear.

```bash
# Retrieve prior work by relevance
parallel-cli memory retrieve --query "serverless inference vendors" --json

# List the most recent saved work
parallel-cli memory retrieve --query "" --json

# Keep application or workspace research in an isolated memory scope
parallel-cli research run "Compare inference vendors" \
--memory-scope-key workspace_acme
parallel-cli findall run "Serverless inference vendors" \
--memory-scope-key workspace_acme
parallel-cli monitor create "Track vendor pricing changes" \
--memory-scope-key workspace_acme

# Retrieve from the same memory scope
parallel-cli memory retrieve \
--query "inference vendors" \
--scope-key workspace_acme \
--json

# Remove one Memory entry without deleting the Task itself
parallel-cli memory evict --kind task --id trun_abc123 --json

# Permanently clear a specific memory scope (underlying resources remain)
parallel-cli memory clear --scope-key workspace_acme --confirm-clear --json
```

Omit `--scope-key` / `--memory-scope-key` to use personal Memory when the
credential and account are eligible. Application credentials require a stable
scope key containing only letters, digits, underscores, or hyphens.

## Non-Interactive Mode (for AI Agents & Scripts)

All commands support `--json` output and can be fully controlled via CLI arguments.
Expand Down Expand Up @@ -262,6 +304,9 @@ parallel-cli findall entity-search "AI startups in healthcare" -t companies -n 2
# Monitor: track web changes
parallel-cli monitor create "Track Tesla SEC filings" --frequency 1d --json

# Recall prior Memory entries
parallel-cli memory retrieve --query "Tesla filings" --json

# Plan without prompts (provide all args)
parallel-cli enrich plan -o config.yaml \
--source-type csv \
Expand Down Expand Up @@ -409,6 +454,36 @@ monitors = list_monitors()
details = get_monitor(monitor["monitor_id"])
```

### Memory

Retrieve and manage Memory entries:

```python
from parallel_web_tools import clear_memory, evict_memory, retrieve_memory

# Relevant prior work across Task, Monitor, and FindAll
memories = retrieve_memory("serverless inference vendors", limit=10)

# Retrieve from an isolated memory scope
scoped = retrieve_memory(
"inference vendors",
memory_scope_key="workspace_acme",
)

# Remove one run from Memory; the underlying Task run remains available
evict_memory(
"task",
"trun_abc123",
memory_scope_key="workspace_acme",
)

# Clear the selected memory; underlying runs remain available
clear_memory(memory_scope_key="workspace_acme")
```

These functions use the generated `client.beta.memory` resource provided by
`parallel-web` 1.2.0 or later.

## YAML Configuration Format

```yaml
Expand Down
1 change: 1 addition & 0 deletions parallel-web-tools.spec
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ a = Analysis(
'parallel_web_tools.core.runner',
'parallel_web_tools.core.schema',
'parallel_web_tools.core.research',
'parallel_web_tools.core.memory',
'parallel_web_tools.core.result',
# CLI (standalone mode - no planner)
'parallel_web_tools.cli',
Expand Down
15 changes: 15 additions & 0 deletions parallel_web_tools/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,23 @@
# Re-export everything from core for convenience
from parallel_web_tools.core import (
AVAILABLE_PROCESSORS,
MEMORY_KINDS,
Column,
DeviceCodeInfo,
InputSchema,
MemoryApiError,
MemoryInputError,
MemoryKind,
ParseError,
ProcessorType,
SourceType,
cancel_monitor,
clear_memory,
create_monitor,
enrich_batch,
enrich_single,
entity_search_findall,
evict_memory,
get_api_key,
get_async_client,
get_auth_status,
Expand All @@ -27,6 +33,7 @@
parse_schema,
poll_device_token,
request_device_code,
retrieve_memory,
run_enrichment,
run_enrichment_from_dict,
run_findall,
Expand Down Expand Up @@ -62,6 +69,14 @@
"enrich_batch",
"enrich_single",
"run_tasks",
# Memory
"MEMORY_KINDS",
"MemoryApiError",
"MemoryInputError",
"MemoryKind",
"clear_memory",
"evict_memory",
"retrieve_memory",
# Runner
"run_enrichment",
"run_enrichment_from_dict",
Expand Down
Loading
Loading