🏗️ Australian Building Regulations RAG API

A sophisticated Retrieval-Augmented Generation (RAG) system designed for Australian building regulatory compliance queries. This system combines multimodal document processing with AI-powered question answering to provide expert guidance on building massing, setbacks, and compliance requirements.

🚀 Technology Stack

Core Technologies

• ColPali - Multimodal RAG using ColBERT + PaliGemma for visual document understanding
• Qdrant - High-performance vector database for similarity search
• OpenAI GPT-4o - Large language model for generating expert responses
• Flask - Lightweight web framework for API endpoints

Document Processing

• PDF2Image - Convert PDF pages to images for visual processing
• PyPDF2 - PDF metadata extraction
• PIL (Pillow) - Image processing and manipulation
• Base64 - Image encoding for API responses

Machine Learning

• ColPali - Advanced document embedding encoder and retriever
• Transformers - Hugging Face transformers for model processing
• PyTorch - Deep learning framework
• CUDA - GPU acceleration support

🎯 What This System Does

Document Indexing

1. PDF Processing: Converts building regulation PDFs into processable format
2. Visual Embedding: Creates multimodal embeddings that understand both text and visual layout
3. Vector Storage: Stores embeddings in Qdrant for fast similarity search
4. Metadata Preservation: Maintains document filename, page numbers, and other metadata

Intelligent Querying

1. Natural Language Input: Accept complex regulatory questions in plain English
2. Semantic Search: Find relevant document sections using advanced similarity matching
3. Expert Analysis: Generate comprehensive compliance assessments using GPT-4o
4. Regulatory Citations: Provide specific clause references and calculations

🛠️ Setup Instructions

Prerequisites

• Python 3.8+
• CUDA-compatible GPU (recommended)
• OpenAI API key
• Qdrant cloud instance or local installation

1. Environment Setup

# Clone the repository
git clone <repository-url>
cd mmrag_hackathon

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Environment Variables

Create a .env file in the project root:

OPENAI_API_KEY=your_openai_api_key_here
QDRANT_API_KEY=your_qdrant_api_key_here
HF_TOKEN=your_huggingface_token_here

3. Document Preparation

# Place your PDF documents in the regulations folder
mkdir -p regulations/
# Copy your building regulation PDFs to this folder

4. Vector Database Setup

# Run the database update script to process and index documents
python update_db.py

5. Start the API Server

# Launch the Flask API
python app.py

The API will be available at http://localhost:5000

📡 API Endpoints

POST /query

Main RAG endpoint for building regulatory queries.

Request Format:

{
    "query": "What are the setback requirements for a 2-storey residential building in NSW?"
}

Response Format:

{
    "response": "**COMPLIANCE ASSESSMENT:** Requires Clarification...",
    "docs": [
        {
            "filename": "nsw_building_code.pdf",
            "page_number": 15,
            "score": 0.89,
            "image": "base64_encoded_image_data"
        }
    ]
}

POST /get-summary

Get summary information about regulations.

Request Format:

{
    "language": "en"
}

🔧 Usage Examples

Python API Client

import requests
import json

def query_regulations(question):
    url = "http://localhost:5000/query"
    headers = {"Content-Type": "application/json"}
    data = {"query": question}
    
    response = requests.post(url, headers=headers, data=json.dumps(data))
    
    if response.status_code == 200:
        result = response.json()
        print("Expert Analysis:", result["response"])
        print(f"Found {len(result['docs'])} relevant documents")
        return result
    else:
        print(f"Error: {response.status_code}")
        return None

# Example usage
question = """
I have a 800m² site in Melbourne, VIC. I want to build a 3-storey apartment building 
with 60% site coverage. What are the required setbacks and height limits?
"""

result = query_regulations(question)

curl Example

curl -X POST http://localhost:5000/query \
  -H "Content-Type: application/json" \
  -d '{
    "query": "What is the maximum building height allowed in R3 Medium Density Residential zones in NSW?"
  }'

Test Script

Use the provided test script for quick validation:

python api_test.py

🏛️ System Architecture

┌─────────────────────────────────────────────────────────────────┐
│                      RAG SYSTEM ARCHITECTURE                    │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  📄 PDF Documents    →    🔄 ColPali Processing    →    🗄️ Qdrant    │
│  (Building Codes)          (Visual + Text)              (Vector DB) │
│                                                                 │
│  👤 User Query       →    🔍 Similarity Search    →    📊 Results   │
│                                                                 │
│  📋 Retrieved Docs   →    🤖 GPT-4o Analysis     →    ✅ Response  │
│  (Relevant Pages)          (Expert System)           (Compliance)  │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Key Components

1. Document Processing Pipeline:

• update_db.py - Indexes PDFs and creates vector embeddings
• multimodal_utils.py - Handles PDF to image conversion
• models.py - Defines ColPali model configurations

2. RAG Query Engine:

• rag_retrieval.py - Implements retrieval and response generation
• app.py - Flask API endpoints and routing

3. Utilities:

• api_test.py - Testing and validation scripts

📊 Features

Advanced Document Understanding

• Visual Layout Recognition: Understands tables, diagrams, and complex layouts
• Multimodal Processing: Combines text and visual information
• Precise Retrieval: Finds relevant sections with high accuracy

Expert-Level Analysis

• Australian Building Code Expertise: Specialized knowledge of NCC, Australian Standards
• Regulatory Compliance Checking: Identifies non-compliant elements
• Quantitative Analysis: Performs calculations and provides specific measurements
• Citation References: Provides exact clause numbers and standards

Production-Ready API

• Error Handling: Robust timeout and retry mechanisms
• Response Formatting: Structured, professional outputs
• Scalable Architecture: Designed for high-volume queries
• Performance Optimization: GPU acceleration and caching

🎯 Use Cases

For Architects & Designers

• Verify building designs against regulatory requirements
• Calculate setback requirements for specific sites
• Understand height limits and bulk controls
• Check site coverage and plot ratio compliance

For Property Developers

• Assess development potential of sites
• Understand zoning restrictions and opportunities
• Navigate complex regulatory frameworks
• Optimise building designs for compliance

🚨 Important Notes

Professional Advice Disclaimer

This system provides general guidance based on available regulatory documents. Always consult with:

• Licensed architects and building designers
• Local council planning departments
• Building certifiers and surveyors
• Legal professionals for complex matters

Regulatory Updates

• Building codes and planning schemes change regularly
• Ensure your document database is current
• Check for updates to Australian Standards
• Verify local council amendments

System Limitations

• Responses are based on documents in the vector database
• Complex site-specific issues may require human expertise
• Performance depends on quality of source documents
• GPU recommended for optimal processing speed

🔧 Troubleshooting

Common Issues

OpenAI API Timeout:

• Increase timeout values in rag_retrieval.py
• Check OpenAI API status
• Verify API key permissions

Qdrant Connection Issues:

• Verify API key and endpoint in .env
• Check network connectivity
• Ensure collection exists

GPU Memory Issues:

• Reduce batch sizes in update_db.py
• Use CPU fallback if necessary
• Monitor GPU memory usage

Document Processing Errors:

• Verify PDF files are not corrupted
• Check file permissions
• Ensure sufficient disk space

🤝 Contributing

We welcome contributions to improve the system:

• Bug fixes and performance improvements
• Additional regulatory document processing
• Enhanced query capabilities
• Documentation improvements