feat: v12.0 extensible platform — custom skills, user tools, multi-agent pipelines#1
Open
feat: v12.0 extensible platform — custom skills, user tools, multi-agent pipelines#1
Conversation
…c ArcPy tool access Previous approach (+ _arcpy_tools list concatenation) failed because the list was empty at module-load time and expanded before retry_arcpy_init() filled it. Using GeoProcessingToolset with include_arcpy=True ensures get_tools() dynamically reads _arcpy_funcs at call time, after ArcPy bridge is ready. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ent pipelines Add self-service extension capabilities for end users: - Custom Skills frontend CRUD in CapabilitiesView (create/edit/delete agents) - User-defined declarative tools (http_call, sql_query, file_transform, chain) with UserToolset exposing dynamic FunctionTool instances to ADK agents - Multi-agent pipeline composition via WorkflowEditor Skill Agent nodes (pipeline_type: "custom_skill" in DAG execution engine) - Capabilities browser tab (13th DataPanel tab) with filter/search - Knowledge Base frontend UI (14th tab: CRUD, doc management, semantic search) - Resizable three-panel layout with drag handles (240-700px range) - DataPanel tab horizontal scrolling for overflow Security fixes: - SEC-1: Remove hardcoded admin/admin123 DB fallback — require DB for auth - SEC-2: Add brute-force protection (5 consecutive failures → 15min lockout) Architecture improvements: - S-1: Extract intent_router.py (153 lines) and pipeline_helpers.py (284 lines) from app.py, reducing it from 3563 to 3267 lines - T-4: Track router token consumption separately (pipeline_type="router") - F-4: Add React ErrorBoundary wrapping all three panels - Upgrade ADK v1.26.0 → v1.27.2 (Session Rewind, CredentialManager, OTel) Bug fixes: - Fix arcpy_tools.py syntax error (duplicate import json + try block) - Fix test_knowledge_agent.py (wrong prompts path, broken imports) - Install APScheduler for workflow cron scheduling - Create chainlit_zh-CN.md for Chinese localization - Improve MCP Hub health check (distinguish disabled vs failed servers) Documentation: sync CLAUDE.md, technical-guide.md, roadmap.md, 7 DITA sources, 2 preview HTMLs with accurate metrics (92 endpoints, 23 toolsets, 18 skills, 2100+ tests, 17 DB tables). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update both README.md (Chinese) and README_en.md (English) to reflect: - 23 toolsets (was 22), 18 ADK Skills (was 16), 92 REST endpoints (was 85) - 2121 tests (was 2104), ADK v1.27.2 (was v1.26) - New v12.0 self-service extension section (Custom Skills CRUD, User Tools, multi-agent pipeline composition, capabilities/KB tabs) - Updated project structure with new modules (intent_router, pipeline_helpers, user_tools, user_tool_engines, capabilities, user_tools_toolset) - Frontend: 13 tabs (was 7), resizable panels, Error Boundaries - WorkflowEditor: 4 node types (added Skill Agent) - Security: brute-force protection, DB auth required - Updated GitHub repo description via gh repo edit Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document the three-layer architecture of the multi-agent system: - Tool: atomic FunctionTool, grouped in 23 BaseToolset subclasses - Skill: dual identity (instruction template + callable AgentTool), 3 sources (built-in ADK Skills, Custom Skills, Prompt YAML) - Agent: LlmAgent with model/instruction/tools/output_key, composed via SequentialAgent/ParallelAgent/LoopAgent Includes execution flow diagram, model tiering strategy, state passing mechanism, and user self-service extension loop (Tool → Skill → Pipeline). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document TUI vs Web UI differences, 6 core usage scenarios (batch processing, headless servers, CI/CD, cron automation, large data, Unix pipe integration), multi-channel unified architecture, file path handling (zero-copy local access), visualization degradation strategy, command design draft, and infrastructure readiness assessment. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document the complete memory system: - L1 Immediate: output_key state passing + ContextVar propagation - L2 Session: last_context multi-turn injection - L3 Cross-session: PostgresMemoryService + Memory ETL auto-extraction - L4 Long-term: spatial memory (6 types) + failure learning - L5 Knowledge: Knowledge Base (RAG + pgvector) + Knowledge Graph Includes injection flow diagram, comparison table, tool inventory, and design principles (layered isolation, auto+manual, dedup+quota). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document all 34 agents across 4 pipelines: - 22 LlmAgent (reasoning entities) - 6 SequentialAgent, 3 ParallelAgent, 3 LoopAgent (orchestrators) - Plus unlimited user-defined CustomSkill agents Includes per-pipeline hierarchy diagrams, model tier allocation (9 Fast + 14 Standard + 2 Premium), factory function mapping, and orchestration pattern summary. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document all 18 built-in ADK Skills across 8 domains: GIS (6), Governance (3), Database (2), Visualization (2), Analysis (2), Fusion (1), General (1), Collaboration (1). Includes trigger keywords, three-level incremental loading, SKILL.md structure, Custom Skills CRUD, and Skill Bundles. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document all tools grouped by 23 BaseToolset subclasses: GeoProcessing (26 incl. 8 ArcPy), Visualization (11), DataLake (10), SemanticLayer (10), KnowledgeBase (10), Location (9), Team (9), AdvancedAnalysis (8), RemoteSensing (8), Exploration (7), Admin (6), Database (6), SpatialT2 (6), Streaming (6), Fusion (5), Memory (5), SpatialStats (4), KnowledgeGraph (4), Watershed (4), Analysis (3), File (3), plus dynamic MCP and UserToolset. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…shboard Backend: - GET /api/system/status — aggregated health (DB, MCP, bots, A2A, features, models) - GET /api/bots/status — per-platform bot config (configured keys, missing env vars) Frontend (AdminDashboard.tsx): - "系统状态" tab: DB/MCP/ArcPy/Cloud status cards, model tier config table, feature flags - "Bot 管理" tab: WeChat/DingTalk/Feishu cards with config status, missing env hints - "A2A" tab: Agent Card display, service status, exposed skills list All three sections surface backend capabilities that previously had no frontend UI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Security: - SEC-3: Fix symlink bypass in sandbox path validation — use os.path.realpath() instead of os.path.abspath() in user_context.py and sharing.py - SEC-5: Change default ContextVar role from 'analyst' to 'anonymous', forcing explicit role assignment on every request Model configuration: - Make MODEL_FAST/STANDARD/PREMIUM configurable via env vars (agent.py) - Add get_model_config() API for frontend exposure - GET /api/config/models — returns tier config with provider detection - AdminDashboard "模型配置" tab: tier table, router model, env var guide, example configs for Gemini/Anthropic/OpenAI Route count: 92 → 95 (system/status, bots/status, config/models) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Skill Bundles frontend (Roadmap: Skill Bundles 前端 UI ✅): - CapabilitiesView: new "技能包" filter tab with full CRUD - Bundle form: name, description, toolset multi-select, skill multi-select, intent triggers, shared toggle - Fetches /api/bundles and /api/bundles/available-tools on mount API refactoring (S-4 — incremental): - Create data_agent/api/ package with helpers.py (shared auth) and bundle_routes.py (extracted from frontend_api.py) - frontend_api.py bundle handlers now delegate to api.bundle_routes - Pattern established for future domain module extraction Code quality: - T-3: Evaluation thresholds now configurable via env vars (EVAL_THRESHOLD_GENERAL, EVAL_THRESHOLD_OPTIMIZATION, etc.) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Knowledge Base GraphRAG UI (Roadmap ✅):
- GraphRAGSection component in KB detail view
- Build graph button → POST /api/kb/{id}/build-graph
- Entity/relation list with type badges and counts
- Graph search → POST /api/kb/{id}/graph-search
Thread safety (S-2 ✅):
- _mcp_started: double-checked locking with threading.Lock
- _a2a_started_at: threading.Lock guard in mark_started()
Frontend quality (F-2 ✅):
- Replace window.__resolveAnnotation / window.__deleteAnnotation
with document.dispatchEvent(CustomEvent) pattern
- Proper cleanup via removeEventListener
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
User Tools Phase 2 — Python Sandbox: - validate_python_code(): AST analysis with import whitelist (19 modules), forbidden builtins (exec/eval/__import__/open), forbidden attrs, must define tool_function() - python_sandbox.py: subprocess execution with sanitized env, restricted builtins, timeout (30s default, 60s max), 100KB output cap - API: create/update endpoints validate Python code via AST before persisting - Frontend: "Python 沙箱" template type in tool creation form React Context API (F-1 ✅): - contexts.ts: MapContext (layers/center/zoom/layerControl) + AppContext (userRole/dataFile/onDataUpdate) - App.tsx: MapContext.Provider + AppContext.Provider wrapping workspace - Components can use useMapContext()/useAppContext() instead of props Thread safety (S-2) + Global callbacks (F-2): - _mcp_started: double-checked locking with threading.Lock - _a2a_started_at: threading.Lock guard - MapPanel annotations: window.__* → CustomEvent dispatch Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WorkflowEditor — live execution status (Roadmap ✅):
- handleExecute captures run_id from execute response
- Polls /api/workflows/{id}/runs/{run_id}/status every 2s
- Live status panel shows per-node status (running/completed/failed)
with colored dots, duration, and overall progress
- Auto-stops polling on completion/failure
SEC-4 — Prompt injection hardening (Roadmap ✅):
- Expand FORBIDDEN_PATTERNS from 7 to 24 patterns covering:
role hijacking, prompt boundary markers, instruction override,
injection delimiters, data exfiltration attempts
- build_custom_agent(): wrap user instruction with safety boundary
+ explicit refusal directive for prompt leaking requests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
API refactoring (S-4 continued): - Extract 10 MCP handlers (275 lines) to api/mcp_routes.py - frontend_api.py: 2473 → 2180 lines (-293, delegate pattern) - Total extracted: bundle_routes.py + mcp_routes.py + helpers.py ADK v1.27 feature adoption: - capabilities.py: Replace manual SKILL.md YAML parsing with google.adk.skills.list_skills_in_dir() (Roadmap: list_skills_in_dir ✅) - Removes yaml dependency from capabilities module Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
S-4 API splitting (continued): - Extract 11 KB handlers (186 lines) to api/kb_routes.py - frontend_api.py: 2180 → 1996 lines (delegate pattern for KB) - Cumulative extraction: helpers + bundles + kb = 3 domain modules ADK v1.27 adoption: - capabilities.py: use google.adk.skills.list_skills_in_dir() instead of manual SKILL.md YAML parsing (Roadmap ✅) Note: 6 pre-existing TestMcpServerCrudAPI failures unrelated to this change (were broken before this session). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document 80+ security controls across 6 defense layers: - L1 Authentication: PBKDF2-SHA256, JWT, OAuth2, brute-force lockout - L2 Authorization: RBAC 3-tier, ContextVar isolation, file sandbox - L3 Input validation: SQL injection, prompt injection (24 patterns), SSRF, path traversal, AST code validation, MCP command whitelist - L4 Execution isolation: Python sandbox, env sanitization, timeout - L5 Output security: API key/password/token redaction, hallucination - L6 Audit: 30+ event types, 90-day retention, Prometheus metrics Includes encryption inventory, bot security, known limitations, and per-area strength ratings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The MapContext.Provider + AppContext.Provider wrapping caused React error #310 (too many re-renders) because useMemo dependencies (mapLayers, userRole) produced new references on each render cycle, triggering infinite context value updates. Fix: remove Provider wrapping from App.tsx, keep contexts.ts for future migration with stable refs (useRef-based values). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Restore original MCP handlers in frontend_api.py (mcp_routes.py was deleted but delegates still referenced it, causing ModuleNotFoundError) - Add ensure_workflow_template_tables() to startup init - Add ensure_skill_bundles_table() to startup init - Fixes "relation agent_workflow_templates/agent_skill_bundles does not exist" All 69 tests passing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…zation, UI redesign BP-3: Automatic analysis lineage recording - Add pipeline_run_id ContextVar to pipeline_helpers.py, set at pipeline start - Fix tool_params passthrough in sync_tool_output_to_obs → register_tool_output - Add pipeline_run_id column to agent_data_catalog with migration - Enhance lineage queries to return pipeline_run_id in ancestors/descendants - Add derives_from/feeds_into edge types + add_lineage_edge() to knowledge_graph BP-5: Industry analysis templates (first batch) - Add 3 industry templates: urban heat island, vegetation change, land use optimization - TemplatesView: add Chinese industry category filter buttons - CapabilitiesView: add 'template' filter type with /api/templates integration S-4: API route extraction (18% → 42%) - Extract mcp_routes.py (10 endpoints), workflow_routes.py (8), skills_routes.py (5) - Delegate from frontend_api.py, total route count unchanged at 95 UI: Cartographic Precision design system - Space Grotesk + JetBrains Mono fonts - Teal (#0d9488) / Amber (#d97706) color palette, warm Stone backgrounds - Topographic contour login page, underline-style tabs, upgraded shadows/radii - Lineage DAG visualization with SVG arrows in asset detail view Tests: 2123 passed (50 catalog + 69 API), 0 failures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…sc fixes - Add 10 new DITA technical guide topics (tg-*): database architecture, DRL optimization, evaluation CI/CD, fusion engine, knowledge graph, map rendering pipeline, multi-pipeline orchestration, multi-tenancy, observability, semantic intent router - Add A2A capabilities documentation (docs/a2a-capabilities.md) - Planner prompt v7.1.3: enforce auto-visualization after analysis, improve PDF report image embedding logic - Fix report_generator PNG regex to handle relative paths - Add watch_ignore for uploads/downloads/logs/db in chainlit config Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…t graph, metrics Semantic hybrid search (BP-2): - Add embedding JSONB column to agent_data_catalog with migration - Generate text-embedding-004 vectors on asset registration - Hybrid search: 60% fuzzy n-gram + 40% vector cosine similarity - Graceful degradation when embedding API unavailable Knowledge graph asset integration: - register_catalog_assets(): create data_asset nodes with belongs_to_domain edges - discover_related_assets(): traverse lineage + domain edges for related assets - Asset type → domain mapping (vector→GIS, raster→遥感, tabular→统计) Planner data discovery priority (v7.2.0): - New prompt section: search catalog before requesting data upload - Extract keywords from user request → search_data_assets → confirm → execute Semantic metrics (v12.2): - New table agent_semantic_metrics with migration 011 - register_metric/resolve_metric/list_metrics tool functions - seed_builtin_metrics: 5 presets (植被覆盖率, 建筑密度, 碎片化指数, 人口密度, 坡度均值) - Fuzzy + alias matching for natural language → SQL definition resolution Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… system, roadmap v14.x
v13.0 Virtual Data Layer:
- virtual_sources.py: CRUD + Fernet encryption + 4 connectors (WFS/STAC/OGC API/custom)
- VirtualSourceToolset: 5 ADK tools wired to General + Planner pipelines (24 toolsets)
- REST API: 6 endpoints /api/virtual-sources/* (virtual_routes.py)
- Frontend: DataPanel "数据源" tab with list/create/edit/delete/test UI
- Schema semantic mapping: text-embedding-004 cosine similarity + 35 canonical fields
- 64 unit tests (test_virtual_sources.py)
v13.1 MCP Server v2.0:
- 6 high-level metadata tools: search_catalog, get_data_lineage, list_skills,
list_toolsets, list_virtual_sources, run_analysis_pipeline
- MCP Server upgraded to v2.0 (36+ tools total)
v14.0 Rating & Clone System:
- Skills/Tools: rating_sum, rating_count, clone_count columns + migration
- REST endpoints: POST /api/skills/{id}/rate, /clone; POST /api/user-tools/{id}/rate, /clone
- 105 total REST API endpoints
Roadmap: v14.x four-version plan across 5 directions (NLP, marketplace,
DRL, SPA, multi-agent) written to docs/roadmap.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…kpoints
Marketplace:
- GET /api/marketplace — aggregated shared skills/tools/templates/bundles
- DataPanel "市场" tab with search, type filter, sort (rating/usage/recent)
- Star rating + clone buttons per item
Rating & Clone:
- Skills/Tools: rating_sum/rating_count/clone_count columns (migration 013)
- POST /api/skills/{id}/rate, /clone; POST /api/user-tools/{id}/rate, /clone
- rate_skill(), clone_skill(), rate_tool(), clone_tool() functions
Workflow Checkpoints (v14.0):
- node_checkpoints JSONB column on workflow_runs (migration 014)
- Per-layer checkpoint save during DAG execution
- get_run_checkpoint() + retry_workflow_node() for single-node retry
- POST /api/workflows/{id}/runs/{run_id}/retry
- GET /api/workflows/{id}/runs/{run_id}/checkpoint
- 108 total REST API endpoints
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Heatmap (2D + 3D): - MapPanel: leaflet.heat integration for point/polygon data with intensity - Map3DView: density-colored ScatterplotLayer for 3D heatmap rendering - Both support value_column for intensity weighting Measurement Tools: - MapPanel: distance + area measurement via click-to-add points - Polyline distance (meters/km) + shoelace area (m²/km²) - Measurement overlay with clear button 3D Layer Control: - Map3DView: layer panel with show/hide toggles per layer - Layer visibility state integrated into deckLayers useMemo Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
….0 milestone DRL Scenario Templates: - DRLScenario config class with 3 built-in scenarios: farmland_optimization, urban_green_space, facility_siting - Each scenario defines source/target types, reward weights, max_conversions - list_scenarios() API + GET /api/drl/scenarios endpoint Memory Search: - GET /api/memory/search?q=keyword&type=region — search user spatial memories - Reuses existing recall_memories() from memory.py v14.0 Complete: 110 REST API endpoints total. - Intent disambiguation (already existed) - Rating + clone system (skills/tools) - Marketplace gallery - Workflow checkpoints + node retry - Heatmap + measurement + 3D layer control - DRL scenario templates - Memory search API Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
QC workflow templates (7 total, +4 new): - qc_dlg: DLG digital line map QC (7 steps, 90min SLA) — feature classification, topology, attribute encoding, edge matching - qc_dom: DOM orthophoto QC (6 steps, 60min SLA) — image quality (arcpy.ia), geometric accuracy, edge quality, color consistency - qc_dem: DEM elevation model QC (6 steps, 60min SLA) — elevation accuracy, slope/aspect validation, contour consistency, NoData check - qc_3dmodel: 3D model QC (6 steps, 120min SLA) — geometry quality (trimesh), texture quality (cv-service), positional accuracy, LOD consistency Dashboard API (GET /api/qc/dashboard): - Aggregated stats: templates count, review stats (total/pending/approved/rejected/fixed), workflow stats (total/running/completed/failed/sla_violated), alert stats (total_rules/enabled_rules/recent_alerts) - Recent reviews list (last 10) - Used by QcMonitorTab dashboard section for real-time monitoring Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…kflow progress Dashboard section (default view) with 3 panels: - Stat cards: templates count, pending reviews, running workflows, recent alerts - Recent Reviews table: file path, defect code, severity badge (A/B/C color-coded), status badge, formatted timestamp - Workflow Stats bars: visual progress bars for completed/running/failed/sla_violated with percentages Fetches from GET /api/qc/dashboard. Dark theme styling (#111827 bg, #1f2937 borders). Null-safe with dashboard?.field || 0 pattern. Section switcher: 概览 (dashboard) | 模板 | 缺陷分类 | 复核 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removed from git tracking (files kept locally): - docs/surveying_qc_demo_script.md - docs/surveying_qc_demo_script.docx - docs/surveying_qc_agent_design.md - docs/surveying_qc_agent_design.docx These are client-facing documents not intended for public repository. Also sanitized local data paths in scripts/qc_e2e_test.py. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
zhouning
force-pushed
the
feat/v12-extensible-platform
branch
from
4246db6 to
45dd67a
Compare
…bsystems Updated core metrics: - Tests: 2700+ (was 2650+) - Toolsets: 40+ (was 38), GovernanceToolset 18 tools, DataCleaningToolset 11, PrecisionToolset 5 - Skills: 22 (was 21, +surveying-qc) - REST APIs: 203+ (was 191+) - DataPanel: 24 tabs (was 22, +QcMonitor +Alerts) - Connectors: 9 (was 8, +ReferenceData) New v15.7 sections: - QC workflow templates (7: 3 generic + DLG/DOM/DEM/3D model) - Defect taxonomy (30 codes, 5 categories, GB/T 24356) - ArcGIS Pro dual-engine MCP (4 basic + 5 DL tools) - 4 independent subsystems (cv-service/cad-parser/arcgis-mcp/reference-data) - Alert rule engine + real-time monitoring dashboard - E2E validation: 107K features, 8s full QC pipeline Updated GitHub About description. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removed from tracking:
- docs/causal_inference_paper.tex
- docs/world_model_paper_response_r2.md
- docs/generate_fusion_paper_docx.py
- docs/technical_paper_fusion_engine.md
Added .gitignore rules for:
- docs/*paper*.{tex,docx,pdf,md}
- docs/*paper*_{cn,en}.*
- docs/surveying_qc_demo_script.*
- docs/surveying_qc_agent_design.*
Generated locally (not committed):
- docs/causal_inference_paper_en.docx (English, 50KB)
- docs/causal_inference_paper_cn.docx (Chinese, 50KB)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- 045: agent_prompt_versions table for prompt version control - 046: scenario/project columns in token_usage for FinOps - 047: agent_eval_datasets table for scenario-based evaluation
- Add 8 BCG platform API endpoints to frontend_api.py: * GET /api/prompts/versions - list prompt versions * POST /api/prompts/deploy - deploy prompt to env * GET /api/gateway/models - list available models * GET /api/gateway/cost-summary - cost breakdown by scenario * GET /api/context/preview - preview context blocks * POST /api/eval/datasets - create eval dataset * POST /api/eval/run - run evaluation * GET /api/eval/scenarios - list scenarios - Enhance prompts/__init__.py: DB fallback in get_prompt() - Enhance token_tracker.py: scenario/project_id params - Enhance agent.py: task-aware routing in get_model_for_tier() - Enhance eval_history.py: scenario/dataset_id/metrics support All changes backward compatible with optional parameters.
- Update CLAUDE.md with BCG Platform Features section - Create bcg-platform-features.md user guide (6 features) - Update bcg-progress-report.md - all phases complete - Verified: 12/12 tests passing, migrations idempotent Total: 7 commits, 20 files, 8 new API endpoints, 100% test coverage
- Update version to v15.8 in both Chinese and English README - Add BCG platform capabilities section (6 modules) - Update metrics: 2712+ tests, 211+ API endpoints - Add 8 new BCG platform API endpoints description
- Change asyncio.create_task to direct await for _execute_workflow_with_steps - Fixes Chainlit context loss causing Steps/Messages not reaching frontend - Workflow now shows real-time progress in chat
…Alchemy - Fix parameter collision: :param::jsonb confuses SQLAlchemy text() parser - Use CAST(:param AS jsonb) instead - Add json.dumps() for dict→JSON string conversion - Create missing agent_data_assets + agent_metadata_schemas tables - Add missing version column to agent_data_catalog
- Show QC report content in chat completion message (not just step count) - Match file by name mentioned in user text before falling to most recent - Exclude .json from spatial file list to avoid mapconfig mismatches - Use rglob to find files in subdirectories
Upload flow now registers files in both agent_data_catalog (for LLM tool queries via list_data_assets) and agent_data_assets (for metadata system). Previously only wrote to agent_data_assets, making uploaded files invisible to the agent's data browsing tools.
- Add accumulated_context to pass results between workflow steps - Extract summary (800 chars) + files after each step completion - Inject as [上一步结果] prefix into next step's prompt - Test: test_workflow_context.py verifies context flows correctly - Resolves #1 pilot blocker: governance agents now see previous step data
- Migration 048: migrate data, rename old table, create compatibility VIEW - data_catalog.py: all functions now read/write agent_data_assets directly with 4-layer metadata format (technical/business/operational/lineage) - app.py: remove dual-write (register_uploaded_file_metadata no longer needed) - Add unique index (asset_name, owner_username) for upsert support - All 50 data catalog tests pass Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TD-003: Auto-migration system - migration_runner.py: discovers and applies pending .sql migrations - Tracks applied versions in schema_migrations table - Called at startup after all ensure_*_table() functions TD-006: Non-blocking workflow execution - Propagate Chainlit context into asyncio.create_task() - Workflow runs in background, user can send other messages - Completion messages delivered correctly via context propagation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add v15.5 (论文修订 + DRL Dreamer), v15.7 (测绘质检智能体), v15.8 (BCG 企业平台 + 技术债务清零) sections - Update benchmark tracking table with v15.8 column (19 capabilities) - Update governance assessment: 75% → 78% (QC + unified metadata) - Update summary table: add 测绘质检/企业平台/技术债务 rows Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
A Direction (3/3): - DRL weight UI: presets + tooltips + balance indicator - Field mapping: drag-drop view + table view toggle - MCP external: stdio entry point + Claude Desktop guide B Direction (4/4): - Intent disambiguation v2: TaskDecompositionCard component - Memory extraction: MemoryConfirmationCard + batch save API - Message bus monitoring: MessageBusTab + 4 management endpoints - Skill SDK: validator/CLI/tests complete, 13/13 passing New files: - data_agent/api/messaging_routes.py - data_agent/mcp_server_stdio.py - frontend/src/components/datapanel/MessageBusTab.tsx - frontend/src/components/MemoryConfirmationCard.tsx - frontend/src/components/TaskDecompositionCard.tsx - docs/mcp_external_client_guide.md Modified: - frontend/src/components/datapanel/OptimizationTab.tsx - frontend/src/components/datapanel/FieldMappingEditor.tsx - frontend/src/components/DataPanel.tsx - data_agent/frontend_api.py - docs/roadmap.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
zhouning
pushed a commit
that referenced
this pull request
Mar 29, 2026
- Add accumulated_context to pass results between workflow steps - Extract summary (800 chars) + files after each step completion - Inject as [上一步结果] prefix into next step's prompt - Test: test_workflow_context.py verifies context flows correctly - Resolves #1 pilot blocker: governance agents now see previous step data
zhouning
force-pushed
the
feat/v12-extensible-platform
branch
2 times, most recently
from
9445cdf to
34e9d9c
Compare
Implements all 7 v16.0 roadmap directions for conditional autonomy: **S-4: Semantic Operator Layer** - semantic_operators.py: 4 operators (Clean/Integrate/Analyze/Visualize) - Auto-strategy selection based on DataProfile - operator_tools.py: 5 ADK tools - 42 tests **S-5: Multi-Agent Collaboration** - 4 specialized agents: DataEngineer, Analyst, Visualizer, RemoteSensing - 2 composite workflows: FullAnalysis, RSAnalysis - prompts/multi_agent.yaml: agent instructions - Planner extended from 7→13 sub-agents - 25 tests **S-6: Plan Refinement & Error Recovery** - error_recovery.py: 5-strategy chain (Retry→Alternative→Simplify→Skip→Escalate) - plan_refiner.py: auto-insert repair steps, remove redundant - Integrated into workflow_engine.py DAG execution - 38 tests **D-4: Guardrails Policy Engine** - guardrails.py: YAML-driven tool-level access control - 3 effects: deny/require_confirmation/allow - standards/guardrail_policies.yaml: viewer/analyst/admin policies - GuardrailsPlugin integration - ~30 tests **Remote Sensing Agent Phase 1** - spectral_indices.py: 15+ indices (NDVI/EVI/SAVI/NDWI/NDBI/NBR/etc) - standards/rs_experience_pool.yaml: 6 preset analysis cases - standards/satellite_presets.yaml: 5 STAC data sources - Cloud cover assessment + auto-recommendation - 30 tests **S-7: Tool Evolution** - tool_evolution.py: unified metadata registry (50+ tools) - Failure-driven discovery + reliability scoring - Dynamic tool registration/deactivation - evolution_tools.py: 8 ADK tools - 48 tests **D-5: AI-Assisted Skill Creation** - skill_generator.py: NL description → Skill config - Keyword-based toolset recommendation - skills/skill-creator/SKILL.md: workflow definition - POST /api/skills/generate endpoint - 41 tests Total: 254 new tests, ~2600 lines of new code, 7/7 roadmap tasks complete. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Highlights: - Version bump: v15.9 → v16.0 - 7 major directions: Semantic Operators + Multi-Agent + Error Recovery + Guardrails + Remote Sensing Phase 1 + Tool Evolution + AI Skill Creation - Test count: 2712 → 2966 (254 new tests) - Toolsets: 40 → 41 (OperatorToolset + ToolEvolutionToolset) - Skills: 22 → 23 (skill-creator) - REST API: 215 → 228 endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
将 Data Agent 从功能封闭的分析平台升级为高度可扩展的 Agent 平台，用户可自助扩展 Skills、Tools 和多 Agent 工作流。
新增能力
安全修复
架构改进
Bug 修复
文档
Test plan
🤖 Generated with Claude Code