Copilot/add yamcs opcode recognition#376
Open
Mikefly123 wants to merge 17 commits into
Open
Conversation
- yamcs/yamcs-data/etc/: YAMCS server config (global + fprime-project instance) with spacecraftId=68, frameLength=248, UDP ports 50000/50001 - yamcs/yamcs-data/etc/processor.yaml: YAMCS realtime processor config - yamcs/yamcs-data/mdb/.gitkeep: placeholder for generated fprime.xtce.xml - yamcs/fprime-gds.yml: yamcs-specific GDS config (omits options unknown to fprime-yamcs such as output-unframed-data and file-uplink-cooldown) - yamcs/pyyaml-override.txt: uv override to satisfy fprime-yamcs>=0.1.0 which requires PyYAML>=6.0.3 while fprime submodule pins ==6.0.2 - yamcs/docker-compose.yml: Docker Compose for remote server deployment - tools/yamcs/proves_adapter.py: auth bridge; serial/TCP <-> YAMCS UDP with HMAC-SHA256 wrapping via AuthenticateFramer (Option B) - Makefile: yamcs-dict, yamcs, yamcs-server, yamcs-adapter-tcp targets; fprime-yamcs-events launched alongside YAMCS; pyyaml uv override wired in - .codespell-ignore-words.txt: add 'ser' (pyserial handle variable) Verified: YAMCS 5.12.0 boots to RUNNING, all UDP data links OK, Beacon container present in MDB from generated XTCE. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fprime-yamcs 0.1.1 has two bugs when used with --no-app and --communication-selection none: 1. launch_app() is called unconditionally even when --no-app is set, crashing on parsed_args.app.name (NoneType). Fix: skip launch_app when noapp=True. 2. fprime-yamcs-events (run by YAMCS ProcessRunner, now by the Makefile) and other venv binaries are not on PATH for Java subprocesses spawned by Maven. Fix: prepend sys.executable's parent (venv bin) to PATH in launch_yamcs(). patches/fprime-yamcs-noapp-path.patch encodes both fixes. The fprime-venv Makefile target applies it after pip install, using a grep sentinel to detect whether it has already been applied (idempotent). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Pass --dictionary to fprime-yamcs-events in Makefile yamcs target so the events bridge can resolve F Prime event IDs (previously failing with "Supply --dictionary or set the FPRIME_DICTIONARY env var") - Fix _forward_tm_serial() to accumulate bytes in a loop until exactly frame_length bytes are available before forwarding; pyserial read() with a short timeout returns partial data, causing YAMCS to receive truncated frames Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- proves_adapter.py: add per-frame CRC validation in the steady-state read loop; re-sync byte-by-byte on failure instead of forwarding misaligned frames to YAMCS permanently. Prints first 6 bytes of any failed frame for diagnosis. Add sys.stdout line-buffering so diagnostics flush immediately to logs. - Makefile yamcs-stop: kill fprime_yamcs Python wrapper and Maven in addition to the JVM; poll until the JVM exits (up to 5 s, then SIGKILL) before returning, preventing port-8090 conflicts on restart. - fprime-yamcs-events busy-wait fix: replace the 100% CPU while True: pass loop with subscription.result(); persist via tools/apply-events-cpu-fix.py wired into the fprime-venv Make target. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- yamcs.fprime-project.yaml: replace placeholder MyPacketPreprocessor with org.yamcs.tctm.cfs.CfsPacketPreprocessor + useLocalGenerationTime. CcsdsPacketPreprocessor is abstract; GenericPacketPreprocessor requires timestampOffset/seqCountOffset args; CfsPacketPreprocessor handles the standard CCSDS primary header and, with useLocalGenerationTime: true, uses reception time instead of a CFS secondary header — correct for F Prime packets. Also remove MyCommandPostprocessor placeholder (not required for TC frames). - tools/apply-yamcs-instance-config-fix.py: new script that patches the fprime-yamcs venv template (which is what the yamcs-maven-plugin actually reads) with the same fixes: correct preprocessor, frame length 248, remove ProcessRunner for fprime-yamcs-events. - Makefile fprime-venv: wire apply-yamcs-instance-config-fix.py into the venv setup so the template is re-patched on every reinstall. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fprime-to-xtce does not designate a root container in the generated XTCE. Mdb.getRootSequenceContainer() therefore returns null, causing StreamTmPacketProvider to log "MDB does not have a root sequence container and none was defined under streamConfig -> tm" and XtceTmExtractor to run with rootContainer=null — packets arrived in the Links/Packets tabs but zero parameters were ever extracted from any packet. Fix: add rootContainer to the tm_realtime streamConfig entry pointing to CCSDSSpacePacket, the CCSDS primary-header root container from which the full F Prime container hierarchy (FPrimeTelemetryPacket/Channel/Event → concrete per-packet/per-channel containers) descends. Also wired into tools/apply-yamcs-instance-config-fix.py so the fix survives venv reinstalls (the venv template is what yamcs-maven-plugin actually reads). Result: Beacon packet parameters (BootCount, CurrentMode, etc.) now appear and update in the YAMCS Parameters tab. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The adapter's TC path had two problems preventing commands from reaching the FSW correctly: 1. Wrong framing order — the adapter wrapped auth around the entire TC Transfer Frame from YAMCS, producing Auth(SDLink(SpacePacket)). The FSW uplink pipeline (frameAccumulator → tcDeframer → authenticate → spacePacketDeframer) expects SDLink(Auth(SpacePacket)). Fix: extract the SpacePacket from YAMCS's TC Transfer Frame, auth-wrap just the SpacePacket, then re-frame in a new TC Transfer Frame using SpaceDataLinkFramerDeframer. 2. CCSDS Packet Data Length was zero — fprime-to-xtce encodes this as a FixedValueEntry of 0 in the XTCE, and no built-in YAMCS command postprocessor fills it in without also adding a CFS checksum that corrupts F Prime's FW_PACKET_COMMAND field. Fix: patch Packet Data Length and Source Sequence Count in the adapter before auth-wrapping. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The FSW interleaves debug/console output (e.g. [Os::...]) on the same UART as CCSDS TM frames. The previous fixed-length read approach would read a mix of console text and partial frame data, causing CRC errors and frame loss during event bursts. Replace the fixed-length read + resync approach with a rolling buffer that scans for TM sync headers and validates CRC before accepting each frame. Non-frame bytes are efficiently skipped, so frames on both sides of console text are recovered. Also enlarge the OS serial receive buffer to 64 KB to absorb TM bursts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Open-Source-Space-Foundation/proves-core-reference/sessions/26a2a047-98ce-4d48-8143-4d17dcf73dd8 Co-authored-by: Mikefly123 <61564344+Mikefly123@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Open-Source-Space-Foundation/proves-core-reference/sessions/26a2a047-98ce-4d48-8143-4d17dcf73dd8 Co-authored-by: Mikefly123 <61564344+Mikefly123@users.noreply.github.com>
…ode-recognition # Conflicts: # Makefile # patches/fprime-yamcs-noapp-path.patch # requirements.txt # tools/apply-yamcs-instance-config-fix.py # tools/yamcs/proves_adapter.py # yamcs/yamcs-data/etc/yamcs.fprime-project.yaml # yamcs/yamcs-data/etc/yamcs.yaml Co-authored-by: Mikefly123 <61564344+Mikefly123@users.noreply.github.com>
… dual-stage acks Three bugs fixed: - get_id_dict() returns a single dict, not a 3-tuple (ValueError on startup) - Event extra key was "opCode" but F Prime names it "Opcode" (acks never fired) - YAMCS XTCE uses '/'-separated SpaceSystem paths, not '|' (pending queue always empty) Added OpCodeDispatched → Acknowledged ack alongside existing OpCodeCompleted → CommandComplete (Completion). Timestamps sent via both _Time STRING attribute and protobuf attr.time field. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Comment on lines
+50
to
+53
| @echo "Applying fprime-yamcs-events opcode-ack fix..." | ||
| @EVENTS_PROC=$$(ls $(shell pwd)/fprime-venv/lib/python*/site-packages/fprime_yamcs/events/processor.py 2>/dev/null | head -1); \ | ||
| if [ -z "$$EVENTS_PROC" ]; then echo "⚠ events processor not found, skipping"; exit 0; fi; \ | ||
| $(VIRTUAL_ENV)/bin/python tools/apply-opcode-ack-fix.py "$$EVENTS_PROC" |
Collaborator
There was a problem hiding this comment.
Let's fork instead. Thoughts?
Contributor
Author
|
Won't merge this one, we will fork |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Without this change, YAMCS command history shows all issued commands as perpetually pending — there is no path from the spacecraft back to YAMCS to confirm that a command was executed.
This PR adds a lightweight bridge (tools/yamcs/opcode_ack_bridge.py) that subscribes to the YAMCS event stream and command history, then maps F Prime OpCodeDispatched / OpCodeCompleted events back to two-stage YAMCS command acknowledgements:
Acknowledged_Status = OK — posted when OpCodeDispatched is received (command reached its target component)
CommandComplete_Status = OK — posted when OpCodeCompleted is received (command finished execution)
A companion patch script (tools/apply-opcode-ack-fix.py) patches the installed fprime_yamcs FPrimeEventProcessor._publish_event so that the raw F Prime opcode integer is exposed in the YAMCS event extra dict under the key Opcode, making it machine-readable by the bridge.
Makefile integration:
Before Screenshot
After Screenshot
Generative AI Notice
This PR was developed in tandem by GitHub Copilot and Claude Code.