Skip to content

productionizing.md

Latest commit

 

History

History
265 lines (199 loc) · 6.94 KB

productionizing.md

File metadata and controls

265 lines (199 loc) · 6.94 KB

Productionizing Prism

This guide covers recommendations for running Prism in a production environment.

Security

Authentication & Authorization

The default password-based authentication is suitable for demos but not production. Consider:

  1. Microsoft Entra ID (Azure AD)

    • Integrate with your organization's identity provider
    • Use MSAL for frontend authentication
    • Validate tokens in the backend

  2. Managed Identity for Azure Services

    • Prism uses DefaultAzureCredential for Azure OpenAI and Storage authentication
    • Container Apps use system-assigned managed identity (no API keys required)
    • Required RBAC roles are automatically assigned during deployment:
      • Cognitive Services OpenAI User on AI Services account
      • Storage Blob Data Contributor on Storage account
    • Benefits: No secrets to manage, automatic credential rotation, audit trail

  3. Secret Management

    • Use Azure Key Vault for any remaining secrets
    • Container Apps secrets for environment-specific values
    • Never commit secrets to source control

  4. Role-Based Access Control

    • Implement project-level permissions
    • Separate admin and user roles
    • Audit access logs

Network Security

  1. Enable HTTPS

    • Container Apps provides automatic HTTPS
    • For custom domains, configure TLS certificates

  2. Private Endpoints

    • Use Azure Private Link for Azure OpenAI
    • Use Private Endpoints for Azure AI Search
    • Deploy in a Virtual Network

  3. Firewall Rules

    • Restrict inbound traffic to known IPs
    • Use Azure Front Door or Application Gateway

Data Protection

  1. Encryption

    • Data at rest: Azure Storage encryption (enabled by default)
    • Data in transit: HTTPS/TLS
    • Consider customer-managed keys (CMK)

  2. Data Residency

    • Deploy in regions that meet compliance requirements
    • Understand data flows to Azure AI services

  3. Content Filtering

    • Azure OpenAI includes content filtering
    • Review and configure filters for your use case
    • Implement additional input validation

Reliability

High Availability

  1. Container Apps

    • Configure minimum replicas > 1
    • Use multiple availability zones
    • Set appropriate CPU/memory limits
    // In container-apps.bicep
minReplicas: 2
maxReplicas: 10

  2. Azure AI Search

    • Use Standard tier or higher for SLA
    • Configure replicas for high availability
    • Consider geo-replication for DR

  3. Azure OpenAI

    • Deploy to multiple regions
    • Implement failover logic
    • Monitor quota and throttling

Backup & Recovery

  1. Project Data

    • Back up the projects/ directory regularly
    • Consider Azure Blob Storage for document storage
    • Implement versioning

  2. Search Index

    • Index can be rebuilt from source documents
    • Document the rebuild process
    • Test recovery procedures

  3. Configuration

    • Version control all configuration
    • Use Infrastructure as Code (Bicep)
    • Automate deployments with CI/CD

Performance

Scaling

  1. Horizontal Scaling

    • Container Apps auto-scales based on HTTP traffic
    • Configure scale rules for CPU/memory
    • Set appropriate max replicas

  2. Vertical Scaling

    • Increase container CPU/memory for large documents
    • Upgrade Azure AI Search tier for more capacity
    • Request higher Azure OpenAI quotas

Optimization

  1. Caching

    • Cache frequently accessed project configs
    • Consider Redis for session/query caching
    • Use CDN for static frontend assets

  2. Async Processing

    • Large documents process asynchronously
    • Consider Azure Queue Storage for job queuing
    • Implement progress tracking

  3. Database

    • For production, consider moving from file-based storage to:
      • Azure Cosmos DB for project metadata
      • Azure Blob Storage for documents
      • This enables better scaling and querying

Monitoring

Application Insights

  1. Enable Tracing

    • Distributed tracing across frontend/backend
    • Track Azure OpenAI API calls
    • Monitor search query performance

  2. Alerts

    • Set up alerts for error rates
    • Monitor latency thresholds
    • Alert on Azure service issues

  3. Dashboards

    • Create operational dashboards
    • Track usage metrics
    • Monitor costs

Logging

  1. Structured Logging

    • Use JSON format for logs
    • Include correlation IDs
    • Log at appropriate levels

  2. Log Retention

    • Configure Log Analytics retention
    • Export to long-term storage if needed
    • Comply with data retention policies

Cost Management

Optimization Strategies

  1. Azure OpenAI

    • Use GPT-4o-mini for simpler tasks
    • Optimize prompt length
    • Cache responses where appropriate

  2. Azure AI Search

    • Right-size the tier for your index size
    • Use Basic tier for development
    • Monitor and optimize query patterns

  3. Container Apps

    • Use consumption plan for variable workloads
    • Set appropriate min/max replicas
    • Schedule scale-down during off-hours

Monitoring Costs

  1. Azure Cost Management

    • Set up budgets and alerts
    • Review cost breakdown by service
    • Identify optimization opportunities

  2. Resource Tags

    • Tag resources by environment/project
    • Enable cost allocation reporting

Compliance

Considerations

  1. Data Classification

    • Understand what data will be processed
    • Implement appropriate controls
    • Document data flows

  2. Regulatory Requirements

    • GDPR, HIPAA, SOC 2, etc.
    • Azure compliance certifications
    • Customer data handling

  3. Audit Trail

    • Log all user actions
    • Track document access
    • Retain logs per policy

Deployment

CI/CD Pipeline

  1. Automated Testing

    • Unit tests for backend
    • Integration tests for API
    • E2E tests for critical flows

  2. Staged Rollout

    • Deploy to dev/staging first
    • Use deployment slots or blue-green
    • Implement rollback procedures

  3. Infrastructure as Code

    • All infrastructure in Bicep
    • Version control changes
    • Review infrastructure changes

Environment Management

  1. Separate Environments

    • Development, Staging, Production
    • Isolated Azure subscriptions/resource groups
    • Different credentials per environment

  2. Configuration Management

    • Use Azure App Configuration
    • Environment-specific settings
    • Feature flags for gradual rollout

Checklist

Before going to production, verify:

  • Authentication integrated with corporate identity
  • Managed identity enabled with proper RBAC roles assigned
  • HTTPS enabled with valid certificates
  • Private endpoints configured for Azure services
  • Minimum 2 replicas for high availability
  • Application Insights configured with alerts
  • Backup procedures documented and tested
  • Cost alerts and budgets configured
  • Security review completed
  • Load testing performed
  • Runbook created for common operations
  • Incident response plan documented