Skip to content

PACKAGE_EXAMPLES.md

Latest commit

 

History

History
250 lines (192 loc) · 6.01 KB

PACKAGE_EXAMPLES.md

File metadata and controls

250 lines (192 loc) · 6.01 KB

Package Examples

This document provides detailed examples of the package structure for components, modules, and workflows.

Component Package

A component package is a self-contained, reusable component with its own configuration and optional test manifest.

Directory Structure

components/
  title_uppercase/
    component.yaml

Example: components/title_uppercase/component.yaml

kind: component
name: title_uppercase
description: Transforms a title field to uppercase
factory: text_transform
config:
  field: title
  operation: uppercase

# Optional: standalone execution manifest
run:
  manifest:
    id: sample-run
    data:
      title: "hello world"

# Optional: test expectations
test:
  expect:
    status: success
    data:
      title: "HELLO WORLD"

Fields

  • kind: Must be "component"
  • name: Unique component name (used for registration)
  • description: Human-readable description
  • factory: Built-in factory name or custom factory
  • config: Configuration passed to the factory
  • run.manifest: Optional manifest for standalone execution
  • test.expect: Optional test expectations

Module Package

A module package groups multiple components into a reusable workflow module. It can include local components that are only available within this module.

Directory Structure

modules/
  analyze-document-mod/
    module.yaml
    components/
      local_content_word_count/
        component.yaml

Example: modules/analyze-document-mod/module.yaml

kind: module
name: analyze-document-mod
description: Analyzes a document by transforming and counting words
manifest:
  id: document-analysis
  data:
    title: "my first document"
    content: "this is example content"

components:
  - name: title_uppercase      # Global component
    scope: global
  - name: local_content_word_count  # Local component
    scope: local
    config:
      field: content
      target_field: word_count

# Optional: test expectations
test:
  expect:
    status: success
    data:
      title: "MY FIRST DOCUMENT"
      word_count: 4

Local Component: modules/analyze-document-mod/components/local_content_word_count/component.yaml

kind: component
name: local_content_word_count
description: Local component for counting words in content field
factory: word_count
config:
  field: content
  target_field: word_count

Fields

  • kind: Must be "module"
  • name: Unique module name
  • description: Human-readable description
  • manifest: Input manifest for module execution
  • components: List of component references
    • name: Component name (global or local)
    • scope: "global" (default) or "local"
    • config: Optional component-specific config override
  • test.expect: Optional test expectations

Component Scope

  • Global components: Loaded from workspace components/ directory
  • Local components: Defined in modules/<module-name>/components/ directory
  • Local components are prefixed with @<module-name>/ at runtime

Workflow Package

A workflow package orchestrates multiple modules into a complete workflow.

Directory Structure

workflows/
  doc-processor-workflow/
    workflow.yaml

Example: workflows/doc-processor-workflow/workflow.yaml

kind: workflow
name: doc-processor-workflow
description: Processes documents through multiple analysis modules

modules:
  - name: analyze-document-mod
    path: analyze-document-mod  # Optional: relative or absolute path
  - name: another-module

# Optional: test expectations
test:
  expect:
    status: success
    modules:
      - id: analyze-document-mod
        status: success
        data:
          title: "MY FIRST DOCUMENT"

Fields

  • kind: Must be "workflow"
  • name: Unique workflow name
  • description: Human-readable description
  • modules: List of module package references
    • name: Module name (used for lookup)
    • path: Optional path override (defaults to name)
  • test.expect: Optional test expectations
    • status: Expected workflow status
    • modules: Per-module expectations

Testing Packages

All package types support testing via the test command:

# Test a component
./workflow-cli test component title_uppercase

# Test a module
./workflow-cli test module analyze-document-mod

# Test a workflow
./workflow-cli test workflow doc-processor-workflow

Test expectations support:

  • Status validation: status: success or status: failure
  • Data validation: Partial match on manifest data fields
  • Module-level validation: Per-module status and data checks

Running Packages

# Run a standalone component (requires run.manifest)
./workflow-cli run component title_uppercase

# Run a module
./workflow-cli run module analyze-document-mod

# Run a workflow
./workflow-cli run workflow doc-processor-workflow

# Run all workflows in a directory
./workflow-cli run workflow --dir ./workflows/

# Run specific workflows by name
./workflow-cli run workflow --dir ./workflows/ --name doc-processor-workflow

Scaffolding Packages

# Scaffold a global component
./workflow-cli scaffold component my-component

# Scaffold a module with local components
./workflow-cli scaffold module my-module

# Scaffold a workflow referencing a module
./workflow-cli scaffold workflow my-workflow --module my-module

Best Practices

  1. Component Naming: Use kebab-case for component names
  2. Local vs Global: Use local components for module-specific logic
  3. Test Coverage: Add test expectations to all packages
  4. Documentation: Include descriptions in all package manifests
  5. Modularity: Keep modules focused on a single responsibility
  6. Reusability: Extract common components to global scope

Advanced: Module Dependencies

Modules can reference other modules by including them in a workflow:

kind: workflow
name: complex-workflow
modules:
  - name: preprocessing-module
  - name: analysis-module
  - name: postprocessing-module

Each module receives the manifest from the previous module, creating a pipeline.