NeXa RoVe

A personal local-first AI and robotics project exploring voice interaction, visual feedback, Raspberry Pi hardware, sensors and robotics experiments.

Start here

This README is the guided tour for NeXa RoVe. It is designed to help a recruiter, engineer or project reviewer move through the project in a clear order: what it is, what I am building, what I have learned, what hardware is involved, how the system works and which examples can be explored.

Open the visual project tour A static dark-mode project experience with the NeXa face controller, animated panels, gallery, video, hardware tour, code lab, calendar and mini game. For the live hosted version, enable GitHub Pages from the repository root. The same index.html becomes the visual tour entry point.

System design Understand the runtime shape, command flow and assistant pipeline.	Command flow See how spoken or typed input becomes an assistant response.	Physical build Review the Raspberry Pi, display, sensors, cameras and mobile base.	Example code Run small Python files that explain the main design ideas.
Gallery See the current physical setup and selected build images.	Diagrams Follow the runtime, vision, model and hardware safety diagrams.	Engineering story Read the build journey, decisions, challenges and progress notes.	Demo Watch a short video of the current NeXa RoVe setup.

What is NeXa RoVe?

NeXa RoVe is my personal AI and robotics project. I am using it to learn how software, local AI, voice interaction, UI feedback and real hardware can work together.

The project combines a local assistant direction, a Visual Shell interface, Raspberry Pi hardware, sensors, cameras and robotics experiments. It started as an assistant idea and grew into a broader engineering project about runtime design, hardware integration, feedback, testing and documentation.

It is active development, not a finished commercial product. The value of the project is the process: building pieces, testing them on real hardware, finding weak points, fixing them and documenting what I learned.

What I am trying to build

I am working toward an assistant that can listen, respond and show what it is doing. The screen matters because voice systems can feel unclear when they are silent, slow or unsure. The Visual Shell gives the assistant a place to show status, thinking, responses, panels and system feedback.

I am also building a hardware platform for experiments with sensors, cameras and movement. The goal is not just to make a chatbot on wheels. The interesting part is how a local assistant can understand requests, explain its state, use nearby hardware and stay careful around physical actions.

The direction is local-first where it makes sense. Some tasks should be quick deterministic commands. Some can use small local model responses. Some need helper flows, safety checks or a clearer question from the user.

NeXa RoVe also includes learning support ideas: study help, explanations, quizzes, plans and routines. I am treating it as a serious engineering project built through iteration, tests and documentation rather than a one-off demo.

What I am learning

The project touches several areas at once, which is what makes it useful as a learning project. Each part has taught me something different about building software that has to interact with people and real hardware.

• System design for assistant-style applications
• Python runtime structure and small testable modules
• Local AI trade-offs and model routing ideas
• Speech interaction, wake/listen behavior and command recovery
• Command understanding and fallback design
• UI feedback for listening, thinking, responding and blocked states
• Godot interface work for a physical assistant screen
• Raspberry Pi hardware integration
• Camera and sensor reliability
• Robotics safety thinking for movement requests
• Testing, debugging and live verification
• Technical documentation that explains design decisions clearly

See the physical build

The gallery is the first visual stop in the tour. It shows the current NeXa RoVe setup, the front display and the internal hardware layout without putting a huge image at the top of the README.

_{Current NeXa RoVe setup.}

Front view of the current build.	Visual Shell running on the front display.
Top view showing mounting and build progress.	Inside view of the hardware layout.

Hardware used

The hardware gives the project a real environment to work against. Devices can be missing, slow, noisy or unreliable, so the software has to report state clearly and behave conservatively.

Raspberry Pi 5 Main local computer used for development, integration and running the project on real hardware.	AI HAT+ Used while exploring local AI and vision acceleration ideas.	8 inch DSI display Shows assistant feedback, status panels and interface screens.	ReSpeaker microphone Used for voice input experiments and local interaction work.
Camera Module 3 Wide Used for camera feedback and vision experiments.	OAK-D Lite Used while exploring depth and vision hardware options.	6x4 mobile base Physical base for movement experiments and safety thinking.	Pan-tilt hardware Used for camera positioning and movement experiments.
ToF sensor Used for distance and nearby-object sensing experiments.	BME688 sensor Used for environment and status sensing experiments.	Orientation sensor Used while exploring motion and orientation awareness.	Power / UPS hardware Supports power work for the physical build.
SSD Local storage used during development and testing.	Speaker Used for local audio output experiments.	USB hub Helps connect and test multiple devices during development.	I2C expansion board Used while exploring connected sensor layouts.

How NeXa RoVe works

The rough flow is:

Voice or text input -> command understanding -> assistant decision -> visual or spoken feedback -> optional hardware action -> testing and improvement loop

Voice or text input starts the interaction. A typed message and a spoken phrase can follow the same broad path once the text is prepared.

Command understanding decides whether the request looks like a status command, learning request, camera request, movement request, general question or unclear input.

Assistant decision chooses the next step. Some requests are deterministic. Some need a local response path. Some should ask a follow-up question. Hardware requests need extra checks.

Visual or spoken feedback makes the assistant easier to understand. The interface can show listening, thinking, responding, blocked, learning and hardware-check states.

Optional hardware action only makes sense after checks. Movement and sensing work should default to waiting or stopping when the state is uncertain.

Testing and improvement is a constant loop. The project has grown through small fixes, reports, focused tests and live hardware checks.

Diagram tour

These diagrams are the fastest way to understand the system shape. Start with the runtime pipeline, then move into command understanding, vision, local model routing and hardware safety.

Runtime pipeline How input moves through the assistant flow.	Command understanding How commands and fallback paths are separated.	Vision flow How camera and detection ideas fit into the system.	Hardware safety How movement ideas are checked before action.
Local AI model flow How local response routes can be selected by task type.	Visual Shell flow How UI state helps explain what the assistant is doing.	Build stage map How the project has moved through stages.	System overview A readable overview of the design direction.

Engineering journey

NeXa RoVe has been built through many small iterations. I worked through voice issues, runtime design, UI feedback, hardware testing, sensors, cameras, robotics safety and documentation. Real hardware made the work harder because devices can be missing, slow, noisy or unreliable.

I used reports, tests and small focused improvements to keep the project moving. The project is as much about engineering judgment as it is about features: deciding what to test, what to simplify, what to show on screen and when the system should wait instead of acting.

Engineering story A readable account of the project direction and build process.

Build map Project stages and how the pieces connect over time.

Challenges Problems I worked through and how I approached them.

Engineering log Selected progress notes and project updates.

Technical brief A recruiter-friendly summary of the engineering work.

Explore the example code

The example files are small runnable Python files that show the main design ideas in a simple form. They use fake data and the Python standard library, so they can be read, run and tested without the full hardware setup.

This is the best place to try the ideas directly. Start with the system flow, then look at command understanding, runtime routing, hardware safety, vision confidence, UI state, model route selection and learning flow.

System flow Shared command classification and movement decision idea.	Command understanding Classifies status, hardware, learning, camera and movement commands.	Runtime pipeline Runs voice and text through the same simplified response flow.	Hardware safety gate Shows ALLOW, WAIT, STOP and BLOCKED decisions from fake state.
Vision confidence Uses fake detections to show confidence and freshness checks.	UI state Maps assistant events to visible interface states.	Local model route Shows a simple route choice between commands, local responses and helpers.	Learning flow Maps learning phrases to lesson, quiz, plan and explanation modes.

Run the examples

python3 examples/public_demo/system_flow_example.py
python3 examples/public_demo/command_understanding_example.py
python3 examples/public_demo/runtime_pipeline_example.py
python3 examples/public_demo/hardware_safety_gate_example.py
python3 examples/public_demo/vision_confidence_example.py
python3 examples/public_demo/ui_state_example.py
python3 examples/public_demo/local_model_route_example.py
python3 examples/public_demo/learning_flow_example.py

Run the example tests:

python3 -m unittest discover examples/public_demo -p "test_*.py"

Short demo video

The demo shows the current physical setup and the direction of the interface on the front display.

Public boundaries

This public repository is designed to show my work on NeXa RoVe in a controlled way. The main working repository remains private for now, but this repo explains the project, hardware, design thinking, diagrams, simplified examples and selected progress.

This public repository does not include private source code, prompts, memory files, logs, diagnostics, raw recordings, private configuration or the full internal runtime.

See docs/public-boundaries.md and docs/what-can-be-shown-publicly.md for the public sharing rules I am using.