Add RVS testcase documentation#192
Open
ElzaJose wants to merge 4 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds RDKV RVS (stability) testcase documentation pages under docs/rdkv_stability/ so that each testcase has a consistent, step-by-step description of objectives, prerequisites, JSON-RPC calls, and expected results.
Changes:
- Added new documentation pages for 6 RVS stability testcases (graphics app load, power-state toggling, reboot stress, synchronous RDKService calls, long-duration HLS playback, long-duration 4K DASH playback).
- Documented preconditions, operational steps (including Thunder JSON-RPC payloads), and per-test attributes (models, duration, priority, release version).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/rdkv_stability/RDKV_CERT_RVS_WebKitBrowser_Load_GraphicsApp.md | New doc for a 6-hour graphics/lightning app run with periodic CPU/memory validation and app install/launch flow. |
| docs/rdkv_stability/RDKV_CERT_RVS_Toggle_PowerStates.md | New doc for stress toggling ON/STANDBY with event verification and per-iteration resource checks. |
| docs/rdkv_stability/RDKV_CERT_RVS_Reboot.md | New doc for reboot stress testing with uptime/interface/plugin validations after each iteration. |
| docs/rdkv_stability/RDKV_CERT_RVS_RDKService_APIs_SynchronousCall.md | New doc for repeatedly invoking multiple RDK service APIs “simultaneously” and aggregating failures. |
| docs/rdkv_stability/RDKV_CERT_RVS_LongDuration_VideoPlayback_4K_DASH.md | New doc for 10-hour 4K DASH playback in Lightning unified player with periodic resource validation. |
| docs/rdkv_stability/RDKV_CERT_RVS_LongDuration_HLS_VideoPlayback.md | New doc for 10-hour HLS playback in Lightning unified player with periodic resource validation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| |-|----------| | ||
| |1|WPEFramework process should be up and running in the device.| | ||
| |2|`graphics_app_url` must be configured in StabilityTestVariables with a valid graphics/animation app URL (e.g., `http://<TM-IP>:8080/rdk-test-tool/fileStore/lightning-apps/tdkobjectanimations/build/index.html?count=100&showfps=false&object=Rect&autotest=true&duration=21600`).| | ||
| |3|`animation_graphics_app_download_url` must be configured in MediaValidationVariables with the full `.bolt` package URL for the animation/graphics app to be downloaded and installed on the device.| |
| | 2 | Check Device Pre-condition State | Validate initial CPU and memory resource usage before the test begins using `check_device_state()`. Activates DeviceInfo plugin if not active, invokes `rdkservice_validateResourceUsage`, then reverts DeviceInfo plugin state. <br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.status@DeviceInfo"}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.activate", "params": {"callsign": "DeviceInfo"}}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "rdkservice_validateResourceUsage"} | CPU and memory usage should be within the expected range before the test starts. | | ||
| | 3 | Check and Activate Required Plugins | Read the current status of DeviceInfo and org.rdk.PersistentStore plugins. If either is not in the required state (`activated`), activate them and save the original statuses for revert on exit. <br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.status@DeviceInfo"}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.status@org.rdk.PersistentStore"}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.activate", "params": {"callsign": "DeviceInfo"}}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.activate", "params": {"callsign": "org.rdk.PersistentStore"}} | Both DeviceInfo and org.rdk.PersistentStore plugins should be in activated state. | | ||
| | 4 | Set Graphics App URL in PersistentStore | Store the configured `graphics_app_url` (from StabilityTestVariables) into the `lightningURL` key of the `MVS` namespace in PersistentStore. This makes the URL available to the Lightning app on launch. <br>{"jsonrpc": "2.0", "id": 1, "method": "org.rdk.PersistentStore.setValue", "params": {"namespace": "MVS", "key": "lightningURL", "value": "<graphics_app_url>"}} | The graphics app URL should be set successfully in PersistentStore under namespace `MVS`, key `lightningURL`. | | ||
| | 5 | Check if Graphics App is Installed | Query PackageManagerRDKEMS for the list of installed packages to determine whether the animation/graphics app (derived from `animation_graphics_app_download_url`) is already installed on the device. <br>{"jsonrpc": "2.0", "id": 1234567890, "method": "org.rdk.PackageManagerRDKEMS.1.listPackages"} | Package list should be retrieved successfully. If the app is already installed, the installation steps are skipped. | |
| | 4 | Set Graphics App URL in PersistentStore | Store the configured `graphics_app_url` (from StabilityTestVariables) into the `lightningURL` key of the `MVS` namespace in PersistentStore. This makes the URL available to the Lightning app on launch. <br>{"jsonrpc": "2.0", "id": 1, "method": "org.rdk.PersistentStore.setValue", "params": {"namespace": "MVS", "key": "lightningURL", "value": "<graphics_app_url>"}} | The graphics app URL should be set successfully in PersistentStore under namespace `MVS`, key `lightningURL`. | | ||
| | 5 | Check if Graphics App is Installed | Query PackageManagerRDKEMS for the list of installed packages to determine whether the animation/graphics app (derived from `animation_graphics_app_download_url`) is already installed on the device. <br>{"jsonrpc": "2.0", "id": 1234567890, "method": "org.rdk.PackageManagerRDKEMS.1.listPackages"} | Package list should be retrieved successfully. If the app is already installed, the installation steps are skipped. | | ||
| | 6 | Activate App Manager Plugins (if app not installed) | If the graphics app is not installed, check the status of org.rdk.DownloadManager, org.rdk.PackageManagerRDKEMS, and org.rdk.AppManager plugins and activate them if not already activated. <br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.status@org.rdk.DownloadManager"}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.status@org.rdk.PackageManagerRDKEMS"}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.status@org.rdk.AppManager"}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.activate", "params": {"callsign": "org.rdk.DownloadManager"}}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.activate", "params": {"callsign": "org.rdk.PackageManagerRDKEMS"}}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.activate", "params": {"callsign": "org.rdk.AppManager"}} | All three app management plugins should be in activated state before download and install. | | ||
| | 7 | Download Graphics App Bundle | Download the animation/graphics app `.bolt` package from `animation_graphics_app_download_url` using DownloadManager. <br>{"jsonrpc": "2.0", "id": 1234567890, "method": "org.rdk.DownloadManager.1.download", "params": {"url": "<animation_graphics_app_download_url>"}} | Download should complete successfully and return a download ID. | |
Comment on lines
+1
to
+4
| ## TestCase ID | ||
| RDKV_STABILITY_66 | ||
| ## TestCase Name | ||
| RDKV_CERT_RVS_RDKService_APIs_SynchronousCall |
Comment on lines
+1
to
+4
| ## TestCase ID | ||
| RDKV_STABILITY_70 | ||
| ## TestCase Name | ||
| RDKV_CERT_RVS_LongDuration_VideoPlayback_4K_DASH |
| | 1 | Pre-requisite Reboot | Conditionally reboot the device before starting the test. `pre_requisite_reboot()` reads the `PRE_REQ_REBOOT` key from the device-specific config file. If set to "Yes", it issues a reboot via `Controller.1.harakiri` and waits 150 seconds for the device to recover. <br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.harakiri"} | Device should come back online successfully if reboot was triggered. | | ||
| | 2 | Check Device Pre-condition State | Validate initial CPU and memory resource usage before the test begins using `check_device_state()`. Ensures DeviceInfo plugin is activated, then invokes `rdkservice_validateResourceUsage`. DeviceInfo plugin status is reverted after the check. <br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.status@DeviceInfo"}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "Controller.1.activate", "params": {"callsign": "DeviceInfo"}}<br>{"jsonrpc": "2.0", "id": 1234567890, "method": "rdkservice_validateResourceUsage"} | CPU and memory usage should be within the expected range before the test starts. | | ||
| | 3 | Read Device Config and Video URL | Read the device-specific config file to determine `LOGGING_METHOD`. Retrieve the 4K DASH video URL from `MediaValidationVariables.video_src_url_4k_dash` and confirm both `video_src_url_4k_dash` and video type ("dash") are non-empty. | Device config must be accessible and both video URL and type must be configured. | | ||
| | 4 | Build Video Player App URL | Construct the complete Lightning video player test app URL by assembling all URL arguments: execID, execDevId, resultId, logging method, TM server URL, operation (`close(36000)` for 10-hour playback), video source URL (4K DASH), options (looptest), autotest=true, type=dash. The player "hlsjs" is selected from `MediaValidationVariables.codec_hls_h264`, resolving to `lightning_video_test_app_url` as the base app URL. | Video player test app URL should be constructed successfully with all required parameters. | |
| @@ -0,0 +1,48 @@ | |||
| ## TestCase ID | |||
| RDKV_STABILITY_66 | |||
| @@ -0,0 +1,49 @@ | |||
| ## TestCase ID | |||
| RDKV_STABILITY_66 | |||
| @@ -0,0 +1,44 @@ | |||
| ## TestCase ID | |||
| RDKV_STABILITY_66 | |||
| @@ -0,0 +1,55 @@ | |||
| ## TestCase ID | |||
| RDKV_STABILITY_70 | |||
| @@ -0,0 +1,50 @@ | |||
| ## TestCase ID | |||
| RDKV_STABILITY_70 | |||
Comment on lines
+35
to
+36
| | 6 | Hibernate Cobalt | Hibernate Cobalt via `org.rdk.RDKShell.1.suspend`. Verify Cobalt is suspended. Record memory usage during hibernate. | Cobalt should enter hibernated state. Memory usage should decrease after hibernation. | | ||
| | 7 | Restore Cobalt from hibernate | Restore Cobalt from hibernate via `org.rdk.RDKShell.1.restoreApp`. Resume Cobalt via `org.rdk.RDKShell.1.launch`. Verify Cobalt is resumed and video playback resumes. | Cobalt should restore successfully. Video should resume playing. | |
| | 2 | Check and set plugin preconditions | Get the status of Cobalt and DeviceInfo plugins. Set required states: Cobalt=resumed, DeviceInfo=activated. | All required plugins should be in their expected states. | | ||
| | 3 | Measure Cobalt memory before hibernate | Launch Cobalt via `launch_plugin`. Set video URL via deeplink. Get current memory usage for the Cobalt process via DeviceInfo.1.systeminfo. | Cobalt should launch successfully. Initial memory usage should be recorded. | | ||
| | 4 | Hibernate Cobalt | Hibernate Cobalt via `org.rdk.RDKShell.1.suspend`. Get memory usage after hibernation. | Cobalt should enter hibernated state. Memory usage should decrease after hibernation. | | ||
| | 5 | Restore Cobalt | Restore Cobalt from hibernate via `org.rdk.RDKShell.1.restoreApp`. Verify Cobalt is restored and memory is within acceptable limits. | Cobalt should restore successfully. Memory should return to expected levels. | |
| | 5 | Restore Cobalt | Restore Cobalt from hibernate via `org.rdk.RDKShell.1.restoreApp`. Verify Cobalt is restored and memory is within acceptable limits. | Cobalt should restore successfully. Memory should return to expected levels. | | ||
| | 6 | Launch second app and measure memory | Launch a secondary Cobalt instance (YouTube TV app URL) via deeplink. Get memory usage before hibernation. | Second app should launch successfully. Memory usage should be recorded. | | ||
| | 7 | Hibernate second app | Hibernate the second app via `org.rdk.RDKShell.1.suspend`. Get memory usage after hibernation. | Second app should enter hibernated state. Memory should decrease after hibernation. | | ||
| | 8 | Restore second app | Restore second app from hibernate via `org.rdk.RDKShell.1.restoreApp`. Validate memory within expected limits. | Second app should restore successfully. Memory should return to acceptable levels. | |
Comment on lines
+33
to
+35
| | 4 | Restore first Cobalt app | Restore Cobalt from hibernate via `org.rdk.RDKShell.1.restoreApp`. Resume Cobalt. Verify memory is within acceptable limits after restore. | Cobalt should restore successfully after long-duration hibernation. Memory should return to acceptable levels. | | ||
| | 5 | Measure and hibernate second app | Launch a secondary Cobalt instance via deeplink. Get memory usage. Hibernate via `org.rdk.RDKShell.1.suspend`. Wait for hibernate_wait_duration seconds. Get memory during long hibernate. | Second app should hibernate. Memory should decrease during hibernation. | | ||
| | 6 | Restore second app | Restore second app from hibernate via `org.rdk.RDKShell.1.restoreApp`. Resume. Validate memory within expected limits. | Second app should restore successfully. Memory should return to acceptable levels. | |
| |-|---------|-----------------|----------------| | ||
| | 1 | Check device state | Verify the device is in a stable state before starting the test using `check_device_state`. | Device should be in healthy state. | | ||
| | 2 | Check and set plugin preconditions | Get the status of Cobalt, WebKitBrowser, and DeviceInfo plugins. Set required states: Cobalt=resumed, WebKitBrowser=deactivated, DeviceInfo=activated. | All required plugins should be in their expected states. | | ||
| | 3 | Launch Cobalt and hibernate | Launch Cobalt via `launch_plugin`. Hibernate Cobalt via `org.rdk.RDKShell.1.suspend`. Verify Cobalt status is "suspended" (hibernated). | Cobalt should launch and enter hibernated state. | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.