Enhanced PDF + XML to DEXPI Converter

This project provides a comprehensive solution to extract vector graphics and text from PDF files and combine them with electrical data from XML files to create enhanced DEXPI (Data Exchange in the Process Industry) format output.

Features

PDF Processing

• Vector Graphics Extraction: Extracts all vector paths, lines, and shapes from PDF files using PyMuPDF
• Text Extraction: Extracts text elements with precise coordinates, fonts, and styling information
• SVG Conversion: Converts PDF pages to SVG format with visibility patches
• Comprehensive Object Handling: Properly handles all PyMuPDF object types (Point, Rect, Matrix, Quad, etc.)

XML Processing

• Component Extraction: Extracts electrical components with properties, coordinates, and ports
• Connection Analysis: Identifies electrical connections and their relationships
• Metadata Processing: Extracts drawing information and parameters

Enhanced DEXPI Output

• Enhanced JSON Format: Complete DEXPI-like structure with visual and electrical data
• Enhanced XML Format: Structured XML output using lxml
• Visual Integration: Cross-references electrical components with their visual representations
• Coordinate Mapping: Links PDF visual elements to XML electrical data

DEXPI RDF/XML Compliance

• RDF/XML Syntax: Proper RDF/XML document structure
• ISO 15926 Namespaces: Uses standard DEXPI, RDF, OWL, and Dublin Core namespaces
• Semantic Web Standards: URN-based resource identifiers
• DEXPI Classes: Equipment, ElectricalComponent, ProcessInstrument, Signal, etc.
• Dublin Core Metadata: Standard document metadata (title, creator, created date)
• Coordinate System: Position and coordinate information for spatial data

Project Structure

├── Main.py                    # Main entry point
├── lib/
│   ├── __init__.py           # Library package
│   ├── pdf_extractor.py      # PDF processing and vector extraction
│   └── enhanced_converter.py # Enhanced DEXPI conversion logic
├── dexpi_output/             # Generated output files
│   ├── enhanced_electrical_schematic_dexpi.json
│   ├── enhanced_electrical_schematic_dexpi.xml
│   ├── enhanced_component_list.csv
│   ├── enhanced_connection_list.csv
│   ├── visual_elements_report.csv
│   ├── enhanced_conversion_report.txt
│   └── svg_pages/
│       ├── page_1.svg
│       └── extracted_pdf_data.json
├── requirements.txt          # Python dependencies
└── README.md                # This file

Setup

Prerequisites

• Python 3.7 or higher
• Virtual environment (recommended)

Installation

1. Clone or download the project files

2. Create and activate virtual environment:

   Windows PowerShell:

   .\activate_env.ps1

   Windows Command Prompt:

   activate_env.bat

3. Install dependencies:

   pip install -r requirements.txt

Usage

Basic Usage

1. Place your files in the project directory:

   • Creoschematic 1.pdf (PDF file with vector graphics)
   • CreoSchematic 1.xml (XML file with electrical data)

2. Run the converter:

   python Main.py

Output Files

The converter generates several output files in the dexpi_output/ directory:

Main DEXPI Files

• enhanced_electrical_schematic_dexpi.json - Complete DEXPI format with visual data
• enhanced_electrical_schematic_dexpi.xml - XML version of the DEXPI format

Analysis Reports

• enhanced_component_list.csv - Detailed component information with coordinates
• enhanced_connection_list.csv - Connection details with visual paths
• visual_elements_report.csv - All visual elements extracted from PDF
• enhanced_conversion_report.txt - Summary of conversion process

Visual Files

• svg_pages/page_1.svg - SVG conversion of PDF page
• svg_pages/extracted_pdf_data.json - Raw PDF data in JSON format

Technical Details

PDF Processing Capabilities

• DEXPI RDF Generation: Converts electrical schematics to ISO 15926 compliant format
• Component Processing: Extracts and classifies electrical components with attributes
• Connection Analysis: Maps electrical connections between components
• Metadata Handling: Processes drawing and project metadata
• Geometric Data: Preserves spatial information for visualization:
  • Component positions and dimensions
  • Connection paths and routing
  • Text placement and orientation
  • Quad objects (quadrilaterals)
  • Complex drawing paths and items

XML Processing Features

• Component Types: COMPONENT, NORMAL_SPOOL, SINGLE, POINT
• Connection Types: SINGLE wiring connections
• Metadata: Drawing parameters and properties
• Port Analysis: Component ports and connection points

Integration Features

• Coordinate Matching: Links XML components to PDF visual elements
• Visual Cross-Reference: Each component includes related visual element IDs
• Path Mapping: Connections include visual path representations with DEXPI RDF/XML compliance
• Statistics: Comprehensive analysis of visual integration and DEXPI validation success

DEXPI RDF/XML Format Details

The converter generates DEXPI-compliant RDF/XML files with the following structure:

Core Components

• Document Metadata: Includes title, creator, and version information
• Electrical Components: Mapped to dexpi:ElectricalComponent with position data
• Process Instruments: Represented using dexpi:ProcessInstrument class
• Connections: Modeled using dexpi:Signal with source/target relationships

Namespace Compliance

• Implements standard DEXPI, RDF, ISO15926, and Dublin Core namespaces
• Full compatibility with DEXPI ontology specifications
• Validated against DEXPI schema requirements

Example Results

From a typical conversion:

• PDF Processing: 1 page, 888 text elements, 5,000 vector elements
• XML Processing: 1,018 components, 181 connections
• DEXPI Validation: 100% ontology compliance, complete namespace validation
• Integration: 214/1,018 components with coordinates, 100% visual element mapping

Dependencies

• PyMuPDF (fitz): Advanced PDF processing and vector extraction
• lxml: XML processing and RDF/XML generation
• rdflib: DEXPI ontology handling and validation
• PyPDF2: Fallback PDF operations
• Standard libraries: json, xml, csv, pathlib, datetime, uuid

Error Handling

The converter includes comprehensive error handling:

• Graceful fallback when PDF is not available
• DEXPI schema validation and error reporting
• Progress indicators for large files
• Memory management with configurable limits
• Detailed error reporting and logging

Performance Optimizations

• Vector Limit: Configurable limit (5,000) to prevent memory issues
• Progress Indicators: Real-time feedback during processing
• Efficient Serialization: Optimized PyMuPDF object conversion
• Memory Management: Proper cleanup and resource management

Troubleshooting

Common Issues

1. "Object of type Point is not JSON serializable"

   • Fixed in current version with comprehensive object serialization

2. Large file processing

   • Vector elements are limited to 5,000 per page by default
   • Adjust max_vectors_per_page in lib/pdf_extractor.py if needed

3. Missing dependencies

   • Run: pip install -r requirements.txt
   • Ensure PyMuPDF is properly installed: pip install PyMuPDF

Performance Tips

• For very large PDFs, consider increasing the vector limit gradually
• Monitor memory usage during processing
• Use the progress indicators to track processing status

Future Enhancements

• Schema Validation: Validate against official DEXPI schemas
• Advanced Coordinate Matching: Improved geometric analysis for component-visual linking
• Multi-page Support: Enhanced handling of multi-page electrical schematics
• Export Formats: Additional output formats (DWG, DXF, etc.)
• Interactive Visualization: Web-based viewer for DEXPI data

License

This project is provided as-is for educational and development purposes.

Support

For issues or questions:

1. Check the conversion report for detailed processing information
2. Review the error messages in the console output
3. Verify input file formats and dependencies