Skip to content

PROTOCOL_GUIDE.md

Latest commit

 

History

History
788 lines (610 loc) · 16.6 KB

PROTOCOL_GUIDE.md

File metadata and controls

788 lines (610 loc) · 16.6 KB
Raw

Industrial Data Streaming Protocols - Complete Guide

The "Why Can't We Just Pick ONE Protocol?" Reference

Patrick Ryan, Fireball Industries CTO
Protocol collector and industrial therapy provider since 2025

Overview

Welcome to Protocol Paradise (or Hell, Depending on Your Day)

This document explains all industrial data streaming protocols, their use cases, and implementation status in this OPC UA Server.

Look, I get it. You're probably thinking "Why do we need 9 different protocols?" The short answer is: because industrial automation is a beautiful disaster of legacy systems, vendor lock-in, and "we've always done it this way." The long answer is... also that, but with more acronyms.

Implemented Protocols ✅

The Whole Circus

1. OPC UA Server (Original)

Status: ✅ Fully Implemented

What it is: Industry-standard protocol for industrial automation (aka "the one everyone pretends to understand")

Use Cases:

  • SCADA systems (Ignition, Wonderware, etc.) - The big boys
  • MES/ERP integrations - Enterprise software that costs more than your car
  • PLC communications - Talking to robots, basically
  • Legacy industrial systems - That PLC from 2003 that no one dares touch

Configuration:

{
  "OPC_ENDPOINT": "opc.tcp://0.0.0.0:4840/freeopcua/server/"
}

Clients: Ignition, UaExpert, Prosys OPC UA Browser, KEPServerEX

2. MQTT

Status: ✅ Fully Implemented

What it is: Lightweight publish/subscribe messaging protocol (IoT's favorite child)

Use Cases:

  • IoT devices - Your smart toaster, probably
  • Cloud integration - Because "cloud" = magic
  • Mobile apps - For when you need to check factory temps from the beach
  • Microservices - Buzzword compliance ✅

Configuration:

{
  "mqtt": {
    "enabled": true,
    "broker": "localhost",
    "port": 1883,
    "topic_prefix": "industrial/opcua"
  }
}

Clients: Node-RED, Mosquitto clients, AWS IoT, Azure IoT Hub

3. Sparkplug B ⭐

Status: ✅ Fully Implemented

What it is: MQTT-based specification designed for SCADA/IIoT

Why Special:

  • Native Ignition Edge support
  • Store-and-forward capabilities
  • Birth/death certificates
  • Efficient binary encoding
  • Sequential message ordering

Use Cases:

  • Ignition Edge (primary use case)
  • Industrial IoT gateways
  • SCADA data acquisition
  • Factory floor data collection

Configuration:

{
  "sparkplug_b": {
    "enabled": true,
    "broker": "localhost",
    "port": 1883,
    "group_id": "Sparkplug B Devices",
    "edge_node_id": "OPC_UA_Gateway",
    "device_id": "EdgeDevice"
  }
}

Topics:

  • NBIRTH: spBv1.0/{group}/NBIRTH/{node}
  • DBIRTH: spBv1.0/{group}/DBIRTH/{node}/{device}
  • DDATA: spBv1.0/{group}/DDATA/{node}/{device}
  • NDEATH: spBv1.0/{group}/NDEATH/{node}

Best For: Ignition Edge, Cirrus Link modules

4. Apache Kafka

Status: ✅ Fully Implemented

What it is: Distributed event streaming platform

Use Cases:

  • High-throughput data pipelines
  • Stream processing (Kafka Streams, Flink)
  • Event sourcing
  • Log aggregation
  • Real-time analytics

Configuration:

{
  "kafka": {
    "enabled": true,
    "bootstrap_servers": ["localhost:9092"],
    "topic": "industrial-data",
    "compression": "gzip"
  }
}

Best For: Enterprise data lakes, big data analytics, microservices

5. AMQP (RabbitMQ)

Status: ✅ Fully Implemented

What it is: Advanced Message Queuing Protocol

Use Cases:

  • Enterprise service bus
  • Reliable messaging
  • Work queues
  • RPC patterns
  • Multi-protocol routing

Configuration:

{
  "amqp": {
    "enabled": true,
    "host": "localhost",
    "port": 5672,
    "exchange": "industrial.data",
    "exchange_type": "topic"
  }
}

Best For: Enterprise applications, guaranteed delivery, complex routing

6. WebSocket

Status: ✅ Fully Implemented

What it is: Full-duplex communication over TCP

Use Cases:

  • Real-time web dashboards
  • Browser-based HMIs
  • Live data visualization
  • React/Angular/Vue apps

Configuration:

{
  "websocket": {
    "enabled": true,
    "host": "0.0.0.0",
    "port": 9001
  }
}

Best For: Web browsers, JavaScript clients, real-time UIs

7. REST API

Status: ✅ Fully Implemented

What it is: HTTP-based request/response API

Use Cases:

  • Web applications
  • Mobile apps
  • Polling-based systems
  • Simple integrations

Configuration:

{
  "rest_api": {
    "enabled": true,
    "host": "0.0.0.0",
    "port": 5000
  }
}

Endpoints:

  • GET /api/tags - All tags
  • GET /api/tags/<name> - Specific tag
  • POST /api/tags/<name> - Write tag

Best For: Simple HTTP clients, periodic polling, web apps

8. MODBUS TCP Server

Status: ✅ Fully Implemented

What it is: Act as a MODBUS TCP server that other devices can poll

Use Cases:

  • Legacy PLCs that read MODBUS
  • Industrial HMIs expecting MODBUS
  • SCADA systems with MODBUS drivers
  • Universal protocol for many vendors

Configuration:

{
  "modbus_tcp": {
    "enabled": true,
    "host": "0.0.0.0",
    "port": 502,
    "register_mapping": {
      "Temperature": {"register": 0, "type": "float"},
      "Pressure": {"register": 2, "type": "float"},
      "Counter": {"register": 4, "type": "int"}
    }
  }
}

Register Types:

  • Float: 2 registers (32-bit IEEE 754)
  • Integer: 1 register (16-bit signed)
  • Boolean: 1 register (0 or 1)
  • String: 32 registers (64 bytes)

Best For: Legacy PLCs, SCADA systems with MODBUS support

9. GraphQL API

Status: ✅ Fully Implemented

What it is: Modern query language for APIs (like REST but you're in control)

Use Cases:

  • Modern web applications (React, Vue, Angular)
  • Mobile apps
  • Data analytics dashboards
  • Flexible tag querying

Why GraphQL:

  • Request exactly the data you need
  • Strong typing and schema introspection
  • Single endpoint for all queries
  • Built-in GraphiQL IDE

Configuration:

{
  "graphql": {
    "enabled": true,
    "host": "0.0.0.0",
    "port": 5002,
    "graphiql": true,
    "cors_enabled": true
  }
}

Example Query:

{
  tags(filter: "Temperature") {
    name
    value
    timestamp
  }
}

Best For: Modern web apps, mobile apps, analytics platforms

Feasible to Add 🟡

1. OPC UA (Original)

Status: ✅ Fully Implemented

What it is: Industry-standard protocol for industrial automation

Use Cases:

  • SCADA systems (Ignition, Wonderware, etc.)
  • MES/ERP integrations
  • PLC communications
  • Legacy industrial systems

Configuration:

{
  "OPC_ENDPOINT": "opc.tcp://0.0.0.0:4840/freeopcua/server/"
}

Clients: Ignition, UaExpert, Prosys OPC UA Browser, KEPServerEX

2. MQTT

Status: ✅ Fully Implemented

What it is: Lightweight publish/subscribe messaging protocol

Use Cases:

  • IoT devices
  • Cloud integration
  • Mobile apps
  • Microservices

Configuration:

{
  "mqtt": {
    "enabled": true,
    "broker": "localhost",
    "port": 1883,
    "topic_prefix": "industrial/opcua"
  }
}

Clients: Node-RED, Mosquitto clients, AWS IoT, Azure IoT Hub

3. Sparkplug B ⭐

Status: ✅ Fully Implemented

What it is: MQTT-based specification designed for SCADA/IIoT

Why Special:

  • Native Ignition Edge support
  • Store-and-forward capabilities
  • Birth/death certificates
  • Efficient binary encoding
  • Sequential message ordering

Use Cases:

  • Ignition Edge (primary use case)
  • Industrial IoT gateways
  • SCADA data acquisition
  • Factory floor data collection

Configuration:

{
  "sparkplug_b": {
    "enabled": true,
    "broker": "localhost",
    "port": 1883,
    "group_id": "Sparkplug B Devices",
    "edge_node_id": "OPC_UA_Gateway",
    "device_id": "EdgeDevice"
  }
}

Topics:

  • NBIRTH: spBv1.0/{group}/NBIRTH/{node}
  • DBIRTH: spBv1.0/{group}/DBIRTH/{node}/{device}
  • DDATA: spBv1.0/{group}/DDATA/{node}/{device}
  • NDEATH: spBv1.0/{group}/NDEATH/{node}

Best For: Ignition Edge, Cirrus Link modules

4. Apache Kafka

Status: ✅ Fully Implemented

What it is: Distributed event streaming platform

Use Cases:

  • High-throughput data pipelines
  • Stream processing (Kafka Streams, Flink)
  • Event sourcing
  • Log aggregation
  • Real-time analytics

Configuration:

{
  "kafka": {
    "enabled": true,
    "bootstrap_servers": ["localhost:9092"],
    "topic": "industrial-data",
    "compression": "gzip"
  }
}

Best For: Enterprise data lakes, big data analytics, microservices

5. AMQP (RabbitMQ)

Status: ✅ Fully Implemented

What it is: Advanced Message Queuing Protocol

Use Cases:

  • Enterprise service bus
  • Reliable messaging
  • Work queues
  • RPC patterns
  • Multi-protocol routing

Configuration:

{
  "amqp": {
    "enabled": true,
    "host": "localhost",
    "port": 5672,
    "exchange": "industrial.data",
    "exchange_type": "topic"
  }
}

Best For: Enterprise applications, guaranteed delivery, complex routing

6. WebSocket

Status: ✅ Fully Implemented

What it is: Full-duplex communication over TCP

Use Cases:

  • Real-time web dashboards
  • Browser-based HMIs
  • Live data visualization
  • React/Angular/Vue apps

Configuration:

{
  "websocket": {
    "enabled": true,
    "host": "0.0.0.0",
    "port": 9001
  }
}

Best For: Web browsers, JavaScript clients, real-time UIs

7. REST API

Status: ✅ Fully Implemented

What it is: HTTP-based request/response API

Use Cases:

  • Web applications
  • Mobile apps
  • Polling-based systems
  • Simple integrations

Configuration:

{
  "rest_api": {
    "enabled": true,
    "host": "0.0.0.0",
    "port": 5000
  }
}

Endpoints:

  • GET /api/tags - All tags
  • GET /api/tags/<name> - Specific tag
  • POST /api/tags/<name> - Write tag

Best For: Simple HTTP clients, periodic polling, web apps

8. MODBUS TCP Server

Status: ✅ Fully Implemented

What it is: Act as a MODBUS TCP server that other devices can poll

Use Cases:

  • Legacy PLCs that read MODBUS
  • Industrial HMIs expecting MODBUS
  • SCADA systems with MODBUS drivers
  • Universal protocol for many vendors

Configuration:

{
  "modbus_tcp": {
    "enabled": true,
    "host": "0.0.0.0",
    "port": 502,
    "register_mapping": {
      "Temperature": {"register": 0, "type": "float"},
      "Pressure": {"register": 2, "type": "float"},
      "Counter": {"register": 4, "type": "int"}
    }
  }
}

Register Types:

  • Float: 2 registers (32-bit IEEE 754)
  • Integer: 1 register (16-bit signed)
  • Boolean: 1 register (0 or 1)
  • String: 32 registers (64 bytes)

Best For: Legacy PLCs, SCADA systems with MODBUS support

9. OPC UA Client Mode

Status: 🟡 Can be added

What it is: Connect to another OPC UA server and write values

Use Cases:

  • Push data to Ignition's OPC UA server
  • Bridge between OPC UA servers
  • Upload to cloud OPC UA servers

How it would work:

  • Connect as OPC UA client
  • Write tag values to remote server
  • Subscribe to remote tags (bidirectional)

Example configuration:

{
  "opcua_client": {
    "enabled": true,
    "server_url": "opc.tcp://ignition-server:4840",
    "namespace": "EdgeDevices",
    "tag_mapping": {
      "Temperature": "ns=2;s=Plant.Line1.Temperature"
    }
  }
}

Complex/Specialized Protocols ⚠️

These require specialized hardware, drivers, or are not suitable for publishing tag data:

10. EtherNet/IP

Status: ⚠️ Not Suitable

What it is: Industrial protocol by Rockwell Automation (Allen-Bradley)

Why Complex:

  • Requires specialized hardware/drivers
  • CIP protocol stack complexity
  • Typically for PLC-to-PLC communication
  • Not designed for general data publishing

Alternative: Use OPC UA or MQTT instead

  • Most Allen-Bradley PLCs support OPC UA
  • Or use a gateway (KEPServerEX, Ignition) to bridge EtherNet/IP to OPC UA

11. PROFINET

Status: ⚠️ Not Suitable

What it is: Siemens industrial Ethernet protocol

Why Complex:

  • Requires real-time Ethernet stack
  • Specialized hardware needed
  • PROFINET controller/device roles
  • Not designed for cloud/IT systems

Alternative:

  • Siemens PLCs support OPC UA
  • Use SIMATIC S7 connector modules
  • Bridge via KEPServerEX or Ignition

12. EtherCAT

Status: ⚠️ Not Suitable

What it is: Real-time industrial Ethernet protocol

Why Complex:

  • Requires dedicated EtherCAT master hardware
  • Sub-millisecond cycle times (not needed for SCADA)
  • Complex synchronization mechanisms
  • Designed for motion control, not data acquisition

Alternative: Use OPC UA from the EtherCAT master

13. IO-Link

Status: ⚠️ Not Applicable

What it is: Point-to-point sensor/actuator communication

Why Not Applicable:

  • Sensor-level protocol
  • Requires IO-Link master hardware
  • Not for publishing tag data
  • Used for device configuration and diagnostics

Alternative: IO-Link masters typically provide OPC UA or MQTT interfaces

Recommended for Implementation

Based on your needs (Ignition Edge, Node-RED, industrial data streaming):

Priority 1: ✅ Already Done

  • ✅ Sparkplug B (for Ignition)
  • ✅ MQTT (for Node-RED, IoT)
  • ✅ REST API (for web apps)
  • ✅ WebSocket (for real-time UIs)

Priority 2: 🟢 Should Add

  1. MODBUS TCP Server - Many legacy systems expect this
  2. OPC UA Client Mode - Push to Ignition/other servers
  3. GraphQL API - Modern web applications

Priority 3: 🟡 Nice to Have

  1. gRPC - High-performance RPC
  2. CoAP - Constrained IoT devices
  3. DDS - Real-time systems
  4. MQTT 5.0 - Enhanced MQTT features

Protocol Comparison Matrix

Protocol Latency Throughput Complexity Best For
OPC UA Low Medium Medium SCADA, Ignition
Sparkplug B Low High Low Ignition Edge ⭐
MQTT Very Low High Low IoT, Node-RED ⭐
Kafka Medium Very High Medium Big Data, Analytics
AMQP Medium High Medium Enterprise Messaging
WebSocket Very Low Medium Low Web Browsers ⭐
REST API High Low Very Low Web Apps, Polling
MODBUS TCP Low Medium Low Legacy PLCs ⭐
EtherNet/IP Low Medium High ❌ Not suitable
PROFINET Very Low High High ❌ Not suitable
EtherCAT Ultra Low Very High Very High ❌ Not suitable

Use Case Recommendations

For Ignition Edge

Use: Sparkplug B (primary) + OPC UA (backup)

python opcua_server.py -c config/config_ignition.json

For Node-RED

Use: MQTT + WebSocket + REST API

python opcua_server.py -c config/config_nodered.json

For Cloud Platforms

Use: Kafka + MQTT + REST API

{
  "kafka": {"enabled": true, "bootstrap_servers": ["kafka.cloud:9092"]},
  "mqtt": {"enabled": true, "broker": "mqtt.cloud", "use_tls": true},
  "rest_api": {"enabled": true}
}

For Enterprise Integration

Use: AMQP + Kafka + OPC UA

{
  "amqp": {"enabled": true},
  "kafka": {"enabled": true},
  "opcua": "Native support"
}

For Legacy Systems

Use: MODBUS TCP + OPC UA

{
  "modbus_tcp": {"enabled": true},
  "opcua": "Native support"
}

Which Protocols Should We Add Next?

Based on your mention of wanting more protocols, here are my recommendations:

Immediate Value (Let me know if you want these!)

  1. MODBUS TCP Server

    • Effort: 2-3 hours
    • Value: High (many legacy systems)
    • Complexity: Medium

  2. OPC UA Client Mode

    • Effort: 1-2 hours
    • Value: High (push to Ignition)
    • Complexity: Low

  3. GraphQL API

    • Effort: 2 hours
    • Value: Medium (modern web apps)
    • Complexity: Low

Future Consideration

  1. gRPC - High performance RPC
  2. CoAP - IoT constrained devices
  3. MQTT 5.0 - Enhanced MQTT
  4. InfluxDB Publisher - Time-series storage

Summary

You currently have:

  • ✅ 7 fully functional protocols
  • ✅ Complete Ignition Edge integration (Sparkplug B)
  • ✅ Complete Node-RED integration (MQTT, WebSocket, REST)
  • ✅ Enterprise streaming (Kafka, AMQP)

You DON'T need:

  • ❌ EtherNet/IP, PROFINET, EtherCAT, IO-Link (wrong abstraction layer)

We CAN add if needed:

  • 🟢 MODBUS TCP Server (high value for legacy)
  • 🟢 OPC UA Client Mode (push to other servers)
  • 🟡 GraphQL, gRPC, CoAP (modern protocols)

Let me know which you'd like me to implement next!

Most practical for your use cases:

  1. MODBUS TCP Server - Bridge to legacy systems
  2. OPC UA Client Mode - Push data to Ignition's OPC UA server

Would you like me to implement either of these? 🚀

Let me know!