Refactor telemetry_chargeback to use scenario_result dictionary#376
Refactor telemetry_chargeback to use scenario_result dictionary#376myadla wants to merge 5 commits into
Conversation
Bring in the full Loki data lifecycle testing workflow: - Synthetic data generation with scenario configs - Loki ingestion, flush, and retrieval tasks - CloudKitty rating comparison - Setup and cleanup tasks
Replace file-suffix variables with a scenario_result dictionary that carries metadata (file_name, num_values, total_rate, summaries) through the pipeline via set_fact. This addresses review feedback from PR #335 (OSPRH-27861) to avoid saving intermediate data to files and re-reading them across tasks. Changes: - Remove 5 suffix variables from vars/main.yml - Build scenario_result dict in gen_synth_loki_data.yml - Update retrieve_loki_data.yml to use and enrich the dict - Update loki_rate.yml to use dict and add CloudKitty ratings - Replace file-diff comparison in run_test_scenarios.yml with direct dict-value assertions - Update ingest_loki_data.yml to use dict for file path - Update README.md to document the new dictionary structure
|
Unable to freeze job graph: Unable to modify final job <Job noop explicit: None implied: None source: None#0> attribute parent=noop with variant <Job feature-verification-tests-noop explicit: None implied: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/feature-verification-tests/.zuul.yaml@master#139> |
|
recheck |
|
Unable to freeze job graph: Unable to modify final job <Job noop explicit: None implied: None source: None#0> attribute parent=noop with variant <Job feature-verification-tests-noop explicit: None implied: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/feature-verification-tests/.zuul.yaml@master#139> |
|
PR addressing rdo/github-check failure: #369 |
|
recheck |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 37m 45s |
|
recheck |
|
Build failed (check pipeline). Post ❌ telemetry-openstack-meta-content-provider-master FAILURE in 12m 08s |
|
recheck |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 12m 52s |
|
OpenStackDataPlaneDeployment(EDPM) timed out. The telemetry_chargeback role didnot got a chance to run. |
|
recheck |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 25m 04s |
The Loki query {service="cloudkitty"} returns data from all scenarios,
causing count and rate mismatches when comparing per-scenario results.
Fix by adding a scenario-specific label to the Loki stream during data
generation (-s/--scenario flag), then filtering by it during retrieval.
Changes:
- gen_synth_loki_data.py: Add --scenario CLI argument that injects a
scenario label into the loki_stream dict
- loki_data_templ.j2: Render all loki_stream keys dynamically instead
of hardcoding only service
- gen_synth_loki_data.yml: Pass -s "{{ item }}" to the script
- retrieve_loki_data.yml: Query with scenario filter
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 49m 10s |
1 participant
Depends-On: #369
These are the enhancements as per Jira https://redhat.atlassian.net/browse/OSPRH-27861 for PR#335
scenario_resultdictionary that carries metadata (file_name,num_values,total_rate, summaries) through the pipeline via set_fact