A Python library for building queue-driven GIS layer update workflows on ArcGIS portals. Define tools that read from input feature layers, process spatial data, and write results to output layers — all coordinated through a persistent queue stored in an ArcGIS Feature Service.
giscado provides a structured framework for automating GIS data processing pipelines. You define a Toolbox — a collection of tools, input layers, and output layers — and giscado handles provisioning the ArcGIS Feature Service (including queue and manifest tables), routing work items through the queue, and tracking status.
The intended architecture has three separate concerns:
- Provisioning — run
set_up()once (e.g. as a one-off deployment step) to create the Feature Service, output layers, queue table, and manifest table in ArcGIS. - Enqueueing — a GIS web application (or similar front end) adds rows directly to the queue table in the Feature Service whenever a user triggers a processing job. giscado's
add_queue_item()method exists for local testing, but in production the web app writes to the queue table directly without needing the library. - Processing — a worker process (e.g. an Azure Function or a Kubernetes job) instantiates the toolbox and calls
process_queue()on a schedule, picking up queued items and writing results to the output layers.purge_queue()can be called by the same worker to recover items that stalled in thePROCESSINGstate.
╔═══════════════════════════════════════════════════════════════════╗
║ DEPLOYMENT (run once) ║
║ ║
║ Your script ║
║ toolbox.set_up() ──> ArcGIS Feature Service ║
║ ├── ToolboxManifest (table) ║
║ ├── ToolboxQueue (table) ║
║ └── <output layers> (feature layers) ║
╚═══════════════════════════════════════════════════════════════════╝
╔═══════════════════════════════════════════════════════════════════╗
║ RUNTIME ║
║ ║
║ GIS Web App ║
║ (user triggers job) ──> ToolboxQueue (status: QUEUED) ║
║ │ ║
║ │ (on schedule) ║
║ ▼ ║
║ Worker Process ║
║ (Azure Function / K8s) ║
║ toolbox.process_queue() ║
║ │ ║
║ ┌──────────┴──────────┐ ║
║ ▼ ▼ ║
║ Read inputs Write outputs ║
║ (input layers) (output layers) ║
║ ║
║ status: COMPLETED / FAILED ║
╚═══════════════════════════════════════════════════════════════════╝
- Python
- ArcGIS Portal (Enterprise or Online) with a user account that has permission to create hosted Feature Services
To install from source:
git clone https://github.com/your-org/giscado.git
cd giscado
pip install -e .Note: package server installation coming soon.
The example below defines a toolbox with a single tool that reads polygon features, computes their centroids, and writes a point to an output layer. add_queue_item() is used here for convenience during local testing — in production, your GIS web app would write to the queue table directly.
from giscado import (
ToolboxBase,
Tool,
ToolContext,
InputLayerDefinition,
OutputLayerDefinition,
FieldDefinition,
GeometryDefinition,
SettingDefinition,
)
def hello_action(context: ToolContext) -> bool:
context.info("Hello from the Hello Tool!")
input_features = context.get_input_features(return_centroid=True)
if not input_features:
context.error("No input features provided.")
return False
output_layer = context.get_output_layer("hello_output")
if output_layer is None:
context.error("Output layer 'hello_output' not found.")
return False
for feature in input_features:
if feature.centroid is None:
context.error("Feature centroid is None.")
continue
output_feature = output_layer.new_point(
x=feature.centroid.x,
y=feature.centroid.y,
)
output_feature["greeting"] = "Hello from GIScado!"
output_layer.add_feature(output_feature)
return True
class HelloToolbox(ToolboxBase):
feature_service_description = "Hello Toolbox"
tools = [
Tool(
name="Hello Tool",
code="hello",
action=hello_action,
feature_input_type="single",
input_layer_key="hello_input",
),
]
output_layers = [
OutputLayerDefinition(
name="hello_output",
description="Hello Output Layer",
geometry=GeometryDefinition(geometry_type="point"),
fields=[
FieldDefinition(name="greeting", field_type="string"),
],
),
]
input_layers = [
InputLayerDefinition(
key="hello_input",
geometry=GeometryDefinition(geometry_type="polygon"),
),
]
settings = [
SettingDefinition(
name="my_setting",
description="An example configuration value",
field_type="string",
),
]
toolbox = HelloToolbox(
arcgis_portal_url="https://your-portal.example.com/arcgis",
arcgis_username="your_username",
arcgis_password="your_password",
feature_service_name="hello_toolbox",
epsg_code=27700,
workspace_layer_url="https://your-portal.example.com/.../FeatureServer/0",
input_layer_urls={
"hello_input": "https://your-portal.example.com/.../FeatureServer/1",
},
setting_values={
"my_setting": "some_value",
},
)
# Run once at deployment time to provision the Feature Service,
# output layers, queue table, and manifest table in ArcGIS.
toolbox.set_up()
# For local testing only — add a queue item directly via the library.
# In production, your GIS web app writes to the queue table directly.
toolbox.add_queue_item(
tool_code="hello",
workspace_id="{430CDFE1-5841-4ED4-A89A-688E4547C118}",
input_feature_ids=["{D45CD7C3-3509-410D-9C64-EB520C0C5774}"],
)
# Called by a worker process (e.g. Azure Function or Kubernetes job) on a schedule.
toolbox.process_queue()A ToolboxBase subclass is the entry point for your processing pipeline. You define your tools, input layers, output layers, and settings as class attributes.
When instantiated, the toolbox connects to ArcGIS and checks whether the associated Feature Service already exists. set_up() is a one-time deployment step that creates the Feature Service, output layers, queue table, and manifest table if they are absent, or updates metadata if they already exist (safe to re-run). It does not need to be called by the worker process on every run.
A Tool wraps a Python function (the action) with metadata that giscado uses to route queue items:
| Parameter | Description |
|---|---|
name |
Human-readable tool name (stored in the manifest table). |
code |
Short identifier used to reference the tool in queue items. |
action |
A callable (context: ToolContext) -> bool. Return True on success. |
feature_input_type |
"single", "multi", or "none" — how many input features are expected. |
input_layer_key |
Key that maps to one of the declared input_layers (required when feature input type is "single" or "multi"). |
Every tool action receives a ToolContext that exposes everything needed to perform the work:
def my_action(context: ToolContext) -> bool:
# Retrieve input features (optionally including centroids)
features = context.get_input_features(return_centroid=True)
# Get the workspace feature (the polygon that defines the area of work)
workspace = context.get_workspace_feature()
# Access a named output layer
output = context.get_output_layer("my_output")
# Access a named input layer directly
layer = context.get_input_layer("my_input")
# Read a configured setting value
value = context.get_setting("my_setting")
# Log messages (written to the queue row in the Feature Service)
context.info("Processing...")
context.warning("Something looks off.")
context.error("Something went wrong.")
return TrueOutput layers are provisioned by giscado inside the Feature Service. Define them with OutputLayerDefinition:
OutputLayerDefinition(
name="my_output", # snake_case layer name
description="My output",
geometry=GeometryDefinition(geometry_type="point"), # point | polyline | polygon
fields=[
FieldDefinition(name="label", field_type="string", length=512),
FieldDefinition(name="score", field_type="double"),
FieldDefinition(name="recorded_at", field_type="date"),
],
)Supported field_type values: "string", "integer", "double", "date", "guid".
Input layers are existing Feature Layers that giscado reads from. Declare them with InputLayerDefinition and supply their URLs at instantiation time:
InputLayerDefinition(
key="my_input",
geometry=GeometryDefinition(geometry_type="polygon"),
globalid_field_name="globalid", # field used to look up features by ID
)Settings are typed configuration values passed at instantiation and made available inside tool actions via context.get_setting(). Declare them with SettingDefinition:
SettingDefinition(
name="api_endpoint",
description="URL of the upstream API",
field_type="string", # string | integer | boolean
)giscado validates that all declared settings are present and correctly typed at startup.
Each queue item passes through the following states:
| Status | Value | Meaning |
|---|---|---|
QUEUED |
0 |
Item is waiting to be processed. |
PROCESSING |
10 |
Item is currently being executed. |
COMPLETED |
20 |
Tool action returned True. |
FAILED |
30 |
Tool action returned False or raised an exception. |
CANCELLED |
40 |
Item was cancelled before processing. |
PURGED |
41 |
Item was stuck in PROCESSING and removed by purge_queue(). |
You can import the status codes directly:
from giscado import (
TOOLBOX_QUEUE_STATUS_QUEUED,
TOOLBOX_QUEUE_STATUS_PROCESSING,
TOOLBOX_QUEUE_STATUS_COMPLETED,
TOOLBOX_QUEUE_STATUS_FAILED,
)If your data spans multiple coordinate reference systems that require datum transformations, you can supply a lookup table hosted as an ArcGIS Feature Layer (or Table). giscado queries this table at runtime to find the correct transformation EPSG code when reprojecting features between coordinate systems.
toolbox = MyToolbox(
...,
use_datum_transformation_lookup_table=True,
datum_transformation_lookup_table_url="https://your-portal.example.com/.../FeatureServer/0",
)The Feature Layer must contain the following fields:
| Field | Type | Description |
|---|---|---|
from_epsg |
Integer | EPSG code of the source coordinate reference system. |
to_epsg |
Integer | EPSG code of the target coordinate reference system. |
datum_transformation_epsg |
Integer | EPSG code of the datum transformation to apply. |
Each row defines a transformation for a specific source/target pair. giscado will also attempt the reverse lookup (swapping from_epsg and to_epsg) if no direct match is found. Results are cached in memory for the lifetime of the toolbox instance.
The example script uses python-dotenv to load configuration from a .env file:
ARCGIS_PORTAL_URL=https://your-portal.example.com/arcgis
ARCGIS_USERNAME=your_username
ARCGIS_PASSWORD=your_password
WORKSPACE_LAYER_URL=https://...
INPUT_LAYER_URL=https://...
cd examples
cp .env.example .env # fill in your values
python hello_toolbox.pysrc/giscado/
├── __init__.py # Public API
├── toolbox.py # ToolboxBase and setup logic
├── tool.py # Tool, ToolContext, ToolAction
├── definitions.py # Layer, field, geometry, and setting definitions
├── gis.py # ArcGIS connectivity helpers
├── input.py # InputLayer
├── output.py # OutputLayer
├── messaging.py # Queue-row message handler
└── status_codes.py # Queue status constants
MIT — see LICENSE for details.