Conversation
* Introduced a new API endpoint to retrieve agent information. * Implemented GetAgentInfo method in the OperatorHandler to handle requests and return agent details. * Updated Operator interface and OperatorService to support fetching agent information. * Added GetAgentInfo implementation in OtelOperator to provide version and agent metadata.
* Added support for "fluent-bit" and its aliases in the adapter creation logic. * Implemented environment variable handling for AGENT_TYPE with default fallback. * Introduced validation for AGENT_TYPE against a list of supported types, defaulting to "otel" if invalid. * Updated tests to cover new adapter type and validation scenarios. * Implemented GetAgentInfo method in mockAdapter for interface satisfaction. * Enhanced MockOperator with GetAgentInfo method to facilitate testing. * Added unit test for OperatorService to validate GetAgentInfo functionality. * Ensured proper assertions for returned agent information in tests.
…ests * Implemented GetAgentInfo method in MockOperator for improved test coverage. * Updated router_test.go to mock GetAgentInfo and validate agent information retrieval. * Added TestGetAgentInfo_Success in operator_test.go to ensure correct handling of agent info requests and response validation.
* Added Fluent Bit as an alternative telemetry backend, including a new FluentBitAdapter for process management and API interactions. * Created comprehensive documentation covering integration, quick start, and migration from OpenTelemetry to Fluent Bit. * Introduced a sample configuration file and Docker support for deployment. * Enhanced the agent's README to reflect support for Fluent Bit and updated environment variable handling. * Implemented extensive unit and integration tests for the FluentBitAdapter, ensuring robust functionality and error handling. * Updated constants and models to accommodate the new agent type and configuration requirements.
…updates * Added Dockerfile for building the Fluent Bit agent image. * Updated README with instructions for building and running the Fluent Bit agent container. * Introduced a sample Fluent Bit configuration file for testing. * Refactored test commands in documentation to reflect new directory structure.
* Added HTTP server configuration options including http_server, http_listen, and http_port. * Enabled hot reload feature for Fluent Bit integration.
* Introduced GetMetrics method in the Adapter interface for metrics retrieval. * Implemented GetMetrics in OTELAdapter with a placeholder for future functionality. * Updated Operator interface and OperatorService to include GetMetrics support. * Added corresponding tests for GetMetrics in MockOperator and other related files. * Enhanced API router to expose a new endpoint for fetching agent metrics.
* Updated the agent registration process to include agent type as a parameter. * Modified the MockAgentQueue and related tests to accommodate the new agent type. * Enhanced the FrontendAgentRepository to fetch agent type information from the database. * Implemented metrics extraction strategies for both OTEL and Fluent Bit agents. * Refactored the configuration compilation logic to support different agent types. * Added comprehensive tests for the new metrics extraction strategies and updated existing tests to ensure compatibility.
…imeout * Added context-based shutdown handling for the HTTP server and agent queue. * Introduced a timeout for the HTTP client to prevent indefinite hangs during metric fetching. * Enhanced agent status model to include an 'InFlight' field for tracking ongoing checks. * Refactored worker logic to ensure safe handling of agent checks and panic recovery. * Updated queue repository to include agent type in monitoring queries. * Improved logging for shutdown processes and agent status updates.
* Changed the configuration file reference from fb-conf.yaml to config.yaml in both Dockerfile and README. * Ensured consistency in the configuration path for the Fluent Bit agent setup.
* Updated the DefaultConfigOTEL to include specific gRPC and HTTP endpoint configurations for the OTLP receiver. * Added support for both protocols to improve telemetry data handling.
…checks * Introduced a new MetricsSource struct to define endpoints for metrics fetching. * Updated Fluent Bit and OTEL metrics strategies to return multiple sources for improved data collection. * Enhanced the checkAgentStatus method to merge metrics from multiple sources and log warnings for failed fetch attempts. * Increased HTTP client timeout for fetching metrics to 10 seconds to prevent hangs. * Added logging for fetched metrics to improve observability.
* Updated the pipeline graph compilation logic to enhance support for both Fluent Bit and OTEL agents. * Introduced helper functions for analyzing graph structures and building Fluent Bit node configurations. * Improved error handling for missing inputs and outputs in pipeline configurations. * Refactored existing tests to accommodate new graph compilation methods and ensure comprehensive coverage. * Added new sample graphs for testing Fluent Bit and OTEL configurations.
* Updated the pipeline graph compilation logic to use a unified CompileGraph method for both Fluent Bit and OTEL agents. * Adjusted the URL for sending configuration to a specific endpoint for testing purposes. * Improved code clarity by consolidating agent type handling in the configuration compilation process.
* Introduced new alias generation functions for OTEL and Fluent Bit components. * Implemented a unified compilation process for both agent types, enhancing the pipeline graph handling. * Added comprehensive tests for alias generation and pipeline compilation, ensuring robust functionality. * Refactored existing code to improve clarity and maintainability, including the removal of deprecated helper functions. * Enhanced error handling for graph validation and cycle detection in pipeline configurations.
* Added a HeartbeatManager to periodically send metrics to the backend. * Introduced configuration for heartbeat interval via environment variable. * Implemented SendHeartbeat function to handle the HTTP request for sending metrics. * Enhanced main function to start and stop the heartbeat manager during agent lifecycle. * Created HeartbeatRequest model to structure the metrics payload sent to the backend.
* Added random jitter to the heartbeat interval to prevent synchronization issues across multiple agents. * Updated the heartbeat sending mechanism to use a calculated interval with ±10% variation. * Ensured the first heartbeat is sent immediately after a delay for registration completion.
* Cleaned up the go.mod file by removing the "// indirect" comments for the Prometheus client_model and common dependencies. * This change improves clarity and maintains consistency in the dependency management.
* Added a Heartbeat endpoint to process periodic metrics from agents, including CPU and memory utilization. * Introduced a StalenessChecker to periodically mark agents as disconnected if no heartbeat is received within a specified timeout. * Updated the AgentService and related models to support heartbeat processing and metrics storage. * Refactored the queue repository to include methods for fetching stale agents and updating their status. * Enhanced the main application logic to initialize and manage the staleness checker alongside existing services.
* Replaced MockAgentQueue with MockMetricsRepository in agent service tests to streamline metric handling. * Updated test cases to reflect changes in the service initialization and ensure proper functionality without queue dependencies. * Removed unused MockQueue struct from frontend agent service tests for improved clarity and maintainability.
* Removed the old dummy input configuration and replaced it with node_exporter and fluentbit metrics inputs for better monitoring. * Updated the Fluent Bit output configuration to match the new metrics tags. * Enhanced the metrics extraction logic to utilize node_exporter for CPU and memory utilization calculations. * Added new utility functions for extracting CPU and memory metrics from Prometheus data. * Cleaned up test cases to reflect changes in the configuration and metrics handling.
…ved code clarity and maintainability.
* Introduced JSON schema files for various Fluent Bit components including ctrlb_output, grep_filter, http_output, modify_filter, opentelemetry_output, stdout_output, syslog_input, and tail_input. * Added corresponding UI schema files to facilitate configuration through the user interface. * Updated schema manifest to include new input, filter, and output types for Fluent Bit, enhancing the overall configuration management. * Ensured all new schemas are structured to support comprehensive configuration options for each component.
…ation * Introduced new test files for Fluent Bit and OTEL configuration compilation, ensuring robust validation of pipeline graphs. * Implemented tests for alias generation functions, verifying correct formats and expected outputs. * Added utility functions to create sample pipeline graphs for testing purposes, enhancing test coverage and maintainability. * Included edge case tests to handle various scenarios, such as empty graphs and cycles in the pipeline. * Refactored existing test cases for clarity and consistency across the testing suite.
* Introduced `run_tests.sh` scripts for both agent and backend, enabling automated testing of Go packages. * Each script includes color-coded output for test results, coverage reporting, and a summary of passed and failed tests. * The agent script tests packages related to internal components, while the backend script covers a broader range of packages, enhancing test coverage across the project.
* Updated the agent registration process to select the appropriate default pipeline graph based on the agent type (Fluent Bit or OTEL). * Introduced a new Fluent Bit pipeline graph definition to support specific configurations. * Modified agent models to include a type field for better identification. * Enhanced repository queries and tests to accommodate the new agent type field, ensuring accurate data retrieval. * Implemented validation logic to ensure pipeline graphs are compatible with the attached agent types, improving overall system robustness.
…nce output metrics support * Added "prometheus_scrape_input" and "opentelemetry_input" to the component type map for Fluent Bit inputs. * Updated signal support mappings to reflect new input types and expanded output metrics for "ctrlb_output", "http_output", and "stdout_output". * Improved overall configuration management for Fluent Bit components.
…g the application's robustness during runtime. * Cleaned up variable naming for better clarity and consistency in the codebase.
* Updated the agent registration process to ensure the agent type is correctly set when registering a new agent. * Modified the pipeline creation and synchronization logic to include the agent type, improving compatibility handling between agents and pipelines. * Adjusted the database schema to support additional component roles in the pipeline components table. * Refactored related tests and interfaces to accommodate the new agent type parameter, ensuring consistency across the codebase. Enhance Fluent Bit configuration with new input and output definitions * Added a new "tail" input configuration for syslog, allowing for better log management. * Updated the output configuration to include a "stdout" output with JSON lines format, improving logging capabilities.
- Modified the download URL in `agent-fluentbit-install.sh` to align with the new release structure on GitHub. - Added an echo statement to display the binary download URL for better visibility during installation.
…nload URL - Introduced a new variable `COLLECTOR_VERSION` to standardize the versioning in `agent-fluentbit-install.sh`. - Updated the download URL echo statement to reflect the new version variable, ensuring consistency in the installation process.
…version - Changed the download URL in `agent-fluentbit-install.sh` to use the fixed version V2.0.0 for the CtrlB Collector. - Updated the echo statement to reflect the new version for improved clarity during installation.
…on V2.0.0 - Modified the download URL in `agent-install.sh` to point to the fixed version V2.0.0 of the CtrlB Collector. - Adjusted the echo statement to reflect the new version for clearer installation feedback.
- Updated the echo statements in `agent-install.sh` to provide more informative feedback during the installation process. - Added a new message to display the binary download URL before the actual download, improving user awareness of the source.
…stency - Updated both `agent-fluentbit-install.sh` and `agent-install.sh` to utilize the `COLLECTOR_VERSION` variable for the download URL, ensuring uniformity in versioning. - Enhanced echo statements to reflect the version variable, improving clarity during the installation process.
…mp settings - Changed the allowed log payload format to only "json" for consistency. - Updated the default JSON date key from "time" to "_timestamp" for clarity. - Modified the JSON date format options to include "epoch_ms" as the new default, enhancing timestamp precision. Update Fluent Bit installation commands to use new script names and templates - Changed the download URLs in the installation commands for Linux, macOS, Kubernetes, and OpenShift to point to the updated script names and templates. - Ensured consistency across all platforms by aligning the script references with the latest naming conventions.
- Updated the FrontendPipelineService to include a new method for detaching all agents from a pipeline, improving agent management capabilities. - Modified the service constructor to accept an agent service interface, allowing for better integration with agent operations. - Implemented logic in the DeletePipeline method to stop and detach agents before deleting a pipeline, ensuring a clean removal process. - Added unit tests to verify the new functionality and error handling for agent detachment and stopping processes.
- Implemented a new endpoint to update the IP address of agents, enhancing agent management capabilities. - Added corresponding handler, service, and repository methods to support the IP update functionality. - Introduced unit tests to validate the new feature and ensure proper error handling for invalid requests and agent existence checks. - Updated the agent model to include a request structure for IP updates.
- Added onPipelineDeleted prop to DeletePipelineDialog for better integration with parent components. - Updated PipelineTable and ViewPipelineDetails to handle pipeline deletion and refresh the pipeline list accordingly. - Removed unnecessary page reload after deletion, improving user experience.
- Introduced ValidateConfigOnDisk method in the Adapter interface for validating Fluent Bit configurations against the binary. - Implemented FluentBitOperator to manage Fluent Bit-specific operations, including configuration updates and validations. - Updated the OperatorService constructor to support dynamic operator creation based on agent type. - Enhanced existing adapters to include new validation methods and updated unit tests to cover these changes.
…for correctness and improved performance - Updated SQL queries in GetHealthMetricsForGraph and GetRateMetricsForGraph methods to retrieve the latest 20 metrics in descending order and then sort them in ascending order for better data presentation. - Enhanced query structure for clarity and maintainability.
zestyoreo
added
documentation
- Modified the JSON date format options to include an empty string as a valid enum value and set it as the default. - Enhanced the UI schema for JSON date format to include a rule for hiding the control based on the condition of another property, improving user experience and form logic.
- Introduced validation functions for Fluent Bit input and filter configurations, ensuring correct parameters and formats. - Added unit tests for the new validation logic, covering various scenarios including valid and invalid configurations. - Updated existing tests to reflect changes in the configuration structure and validation requirements.
…ng and conditional field cleanup - Implemented flattening of nested configuration objects to ensure compatibility with Fluent Bit's expected format. - Added cleanup logic for conditional fields based on the presence of related configuration values, improving the integrity of generated configurations. - Updated validation functions to utilize the new flattened configuration structure for input, filter, and output components.
…cture - Changed the 'type' field to 'component_role' in PluginDropdownOptions for better clarity on data representation. - Updated HealthMetricPoint interface to allow dynamic keys, enhancing flexibility in health metric data handling.
…nt Bit configuration logging
- Introduced SKIP_CONFIG_VALIDATION constant to allow optional skipping of Fluent Bit binary config validation based on environment variable. - Updated FluentBitOperator to conditionally validate configuration against the binary. - Enhanced logging to provide more detailed information during temp config file creation and command execution.
- Changed the input tag from 'syslog_tail_input' to 'auth_tail_input' and updated the path to '/var/log/auth.log' to reflect the new logging source. - Adjusted the match parameter accordingly to ensure proper data handling. Update Fluent Bit configuration names and paths for clarity - Renamed input and output components to "Fluent Bit Tail Input Configuration" and "Fluent Bit Stdout Output Configuration" for better clarity. - Updated the input path from '/var/log/syslog' to '/var/log/auth.log' to align with the new logging source.
…ation - Updated the installation script to first check if Fluent Bit is already installed, providing a message and skipping installation if it is. - Improved the structure of the installation process for various distributions, ensuring clarity and consistency in the installation steps. - Added verification for the existence of the Fluent Bit binary post-installation to confirm successful installation.
- Introduced SKIP_CONFIG_VALIDATION variable to allow users to optionally skip configuration validation during the installation process. - Updated the environment file and script to reflect this new variable, enhancing flexibility in the installation workflow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.