v0.7.1

What's Changed

• feat: Implement General Dataset API with CRUD Operations by @rbardaji in #91
• Fix: PATCH endpoint replacing resources instead of adding them by @rbardaji in #93

Full Changelog: 0.7.0...0.7.1

0.7.0

Release v0.7.0

🎯 Overview

This release marks a significant milestone in project maturity with comprehensive documentation overhaul, enhanced testing coverage reaching 70%, and improved CI/CD pipeline. Major focus on developer experience and system reliability.

✨ New Features

🔧 Service Registration

• NEW: Service registration endpoint (POST /services) for dynamic service management
• Enhanced API routing with integrated services endpoint
• Added ServiceRequest model for structured service registration

📊 Enhanced System Metrics

• Organization field now included in system metrics payload
• Swagger version tracking in metrics for better API versioning
• Comprehensive metrics logging with configurable organization settings

🛡️ Improved Validation & Error Handling

• CKAN field validation before resource creation (fixes #77)
• Enhanced pre-CKAN server validation with better error messages
• Custom file types support in URLRequest model for greater flexibility

🔧 Technical Improvements

🧪 Testing Excellence

• Achieved 70% test coverage milestone
• Comprehensive test suites for:
  • add_url service with edge cases
  • update_url service operations
  • search_datasource module functionality
• Resolved all test failures with proper mocking strategies
• Updated CI pipeline for faster checks and better reliability

🏗️ Infrastructure & DevOps

• Simplified Docker setup: Replaced custom scripts with standard docker-compose commands
• Improved CI pipeline with code formatting (Black, isort)
• Enhanced logging configuration with rotating file handlers
• Better environment variable management with root .env file

📚 Documentation Overhaul

• Complete README redesign with federated data management focus
• New CONTRIBUTING.md guide for better contributor onboarding
• Updated project banners and visual identity
• Comprehensive API documentation improvements
• Enhanced setup and configuration guides

🔄 Configuration & Environment

• Streamlined environment configuration with single root .env file
• Improved service discovery and conditional logging
• Enhanced Docker compose setup for development consistency
• Better integration with external services (Keycloak, Kafka, DXSpaces)

🐛 Bug Fixes

• Fixed boolean parsing in environment variable tests
• Resolved asyncio fixture scope warnings
• Corrected test attribute mappings (public → is_public)
• Enhanced error handling for SSL connections in external services
• Improved validation messages for better user experience

🔧 Breaking Changes

• Docker setup: Custom Docker scripts removed in favor of standard docker-compose commands
• Environment variables: Consolidated into single root .env file
• Test attributes: Updated from public to is_public (ensure your tests are updated)

📈 Performance & Reliability

• Enhanced system metrics collection and reporting
• Improved error handling across all endpoints
• Better resource management with optimized Docker setup
• Faster CI/CD pipeline execution

🚀 Migration Guide

1. Update Docker setup: Replace custom Docker scripts with docker-compose up
2. Environment configuration: Migrate all environment variables to root .env file
3. Test updates: Update any tests using public attribute to is_public
4. Dependencies: Run pip install -r requirements.txt for new testing dependencies

📊 Stats

• 70% test coverage achieved
• 6 major pull requests merged

Full Changelog: 0.6.0...0.7.0

0.6.0

What's Changed

• fix: Kafka registration and search integration test (#48) by @rbardaji in #49
• fix: Fix swagger_version as constant in settings by @rbardaji in #51
• fix: Add optional 'server' param to /organization route (#52) by @rbardaji in #53
• feat: Implement hourly metrics logging (#54) by @rbardaji in #56
• Implement Redirect Endpoint by @rbardaji in #58
• Feat/metrics logging services by @rbardaji in #60
• [fix] Disallow 'pre_ckan' in search endpoint and improve test stability by @rbardaji in #62
• [Feature] Persist system metrics to log file by @rbardaji in #64
• [Task] Include service URLs and flags in system metrics logs by @rbardaji in #66
• [Feature] Display system metrics in dashboard by @rbardaji in #68
• [Feature] Expose DXSpaces integration in dashboard (#69) by @rbardaji in #70
• [Refactor] Load configuration from root .env file by @rbardaji in #72
• Update docker-compose.yml by @saleemalharir1 in #74
• Update default metrics_endpoint and env_file path by @rbardaji in #76

New Contributors

• @saleemalharir1 made their first contribution in #74

Full Changelog: 0.5.0...0.6.0

0.5.0

What's Changed

• Feat:add jupyer endpoint by @rbardaji in #21
• Add optional 'pre_ckan' integration for dataset registration and search by @rbardaji in #23
• Fix: Default search server to 'global' and exclude 'local' if disabled by @rbardaji in #25
• Fix: Ensure global CKAN errors return user-friendly message by @rbardaji in #27
• feat: set default server to 'global' and improve pre_ckan error handling by @rbardaji in #29
• fix: update test_search_datasource_post to use 'global' as the default server by @rbardaji in #31
• feat: add query param server selection for POST /kafka and fix duplicate error message by @rbardaji in #33
• feat: add optional server query param to POST /s3 for local or pre_ckan by @rbardaji in #35
• feat: add optional query param server to POST /url for local or pre_ckan by @rbardaji in #37
• feat: add optional ?server=local/pre_ckan query param to PUT /kafka/{dataset_id} by @rbardaji in #39
• feat: add optional ?server=local/pre_ckan query param to PUT /url/{resource_id} by @rbardaji in #41
• feat: add optional ?server=local/pre_ckan query param to PUT /s3/{resource_id} by @rbardaji in #43
• feat: add optional ?server=local/pre_ckan to DELETE /organization/{organization_name} by @rbardaji in #45
• feat: add optional ?server=local/pre_ckan to DELETE /resource and /resource/{resource_name} by @rbardaji in #47

Full Changelog: 0.4.0...0.5.0

v0.4.0

What's Changed

• feat: Add JupyterLab integration with optional configuration by @rbardaji in #14
• fix: Improve error handling for duplicate dataset registration by @rbardaji in #17
• feat: add kafka_prefix and max_streams to Kafka details endpoint by @rbardaji in #18
• feat: make local CKAN optional, refine dashboard styling, and conditionally skip tests by @rbardaji in #20

Full Changelog: 0.3.0...0.4.0

v0.3.0

What's Changed

• [Feat] Add token-based authentication support in Swagger by @rbardaji in #6
• fix: Escape special characters in keys parameter to prevent Solr syntax errors by @rbardaji in #9
• feat: Add Kafka connection details endpoint and tests by @rbardaji in #12

Full Changelog: 0.2.0...0.3.0

v0.2.0: Enhanced Search Endpoint and Configurability

This release introduces enhanced search functionality for the API:

New Features

• Enhanced /search endpoint:

  • Now supports key-based searches for specific fields.
  • Enables mixed global and key-specific term searches.

• Public/Private Status Configuration:

  • Added an environment variable public to define the status of the system as public or private (#1).

Other Updates

• The default configuration was updated to reflect the latest changes.

v0.1.0: Initial Project

Key Features

• Initial API functionalities.
• Home template layout improvements for better user experience.
• Comprehensive README with installation and usage instructions.
• Search dataset feature improvements and bug fixes.
• Token handling updates to support test users.

CI/CD and Testing

• Initial setup of Continuous Integration (CI) configuration.
• Basic test suite added, covering main API endpoints.

This release sets the foundation for further development, providing core functionality and a stable environment for continuous integration and testing.