Nemo Assistant is a lightweight desktop assistant that brings selection actions, screenshot OCR, Markdown notes, sticky notes, tool calling, and AI chat into one local-first workflow.
It is not another chat window waiting for you to switch context. It is a set of small desktop actions that stay close to your work: translate while reading, polish while writing, capture and understand screen content, pin a quick sticky note, and let AI call tools, read notes, or save memories when needed.
Built with PyQt6 + Fluent Design. Notes, memories, configuration, and conversations are stored locally by default. API keys are saved through the system keyring instead of plaintext config files.
- People who switch between browsers, PDFs, IDEs, and chat apps and need quick translation, explanation, or rewriting.
- Users who want a local-first AI desktop assistant instead of moving every workflow into a web app.
- People who like Markdown notes, desktop stickies, global shortcuts, and lightweight toolboxes.
- Users who want AI to safely call local tools: read and write files, search the web, manage notes, save memories, and create reminders.
Select text in any app, and Nemo Assistant shows an action bar near the cursor. You can:
- Explain, translate, polish, or fix grammar.
- Continue asking about the selected text in a temporary session.
- Save the selection to your note library.
- Rewrite the selected text in place and replace the original selection.
Selection capture uses UIA with a clipboard fallback: it reads directly when possible, injects Ctrl+C only when needed, and restores the clipboard afterward to avoid disrupting your workflow.
After clicking Explain, a lightweight result card appears near the original text, so you do not need to copy the content into a separate chat window.
Buttons in the selection action bar can be enabled or disabled in Settings. If you want to explore a selected passage in more depth, click Continue Session or New Session, and the content will be saved under Quick Sessions in the chat view.
Press Ctrl+Alt+A to select an area of the screen, then recognize text, pin the image, or send it to AI:
- Recognize text locally with RapidOCR / ONNX Runtime.
- Pin screenshots on the desktop as temporary visual references.
- Send screenshots directly to vision-capable models.
- Use the screenshot button in the top-right corner of the chat window; the entry point matches the main chat interface.
Screenshot support is built into the floating-window workflow rather than delegated to an external tool: one shortcut handles region selection, OCR, AI analysis, saving, or pinning.
- Desktop stickies: press
Ctrl+Alt+Nto create a floating sticky note with automatic saving. - Notebook: supports Markdown, folders, tags, full-text search,
[[wiki links]], and syntax highlighting. - AI collaboration: chat can read, create, and update notes, and important information can be saved as long-term memory.
Built-in tools can be viewed, enabled, disabled, and managed in one place. You can also add your own Python tool scripts and let AI call them during a conversation.
Built-in tools cover:
- Web search and web page fetching.
- File reading, saving, and directory listing.
- Python code execution.
- Notes, memory, reminders, and scheduled tasks.
- Multi-model consultation.
- Clipboard read/write.
In a conversation, the assistant can summarize the tools it has available and put them to work on your request.
Nemo Assistant records the execution chain for each conversation turn, making debugging, evaluation, and review easier:
- LLM latency, token usage, and input/output statistics.
- Tool call records.
- Security audit details.
- Eval samples that can be saved as test cases.
Models are connected through LiteLLM, with support for OpenAI, Anthropic, DeepSeek, Gemini, and OpenAI-compatible APIs. Each model can have its own api_base, vision capability setting, max tokens, temperature, and other parameters.
After the first launch, open Settings → API, add a model, and set it as the default to enable AI features across chat and desktop actions.
Web search tools can use DuckDuckGo without an API key. Bing, Tavily, Brave, and Bocha search can be enabled from Settings → Tools by selecting a provider and saving its API key in the system keyring.
App configuration is stored in
config/app_config.jsonand is not committed. It is generated automatically on first launch; seeconfig/app_config.example.jsonfor an example.
| Shortcut | Action |
|---|---|
Ctrl+Alt+Q |
Open quick ask |
Ctrl+Alt+A |
Screenshot + OCR |
Ctrl+Alt+N |
Create desktop sticky note |
Ctrl+Alt+Space |
Show / hide main window |
Shortcuts can be changed in Settings → Hotkeys. The selection action bar appears automatically after text is selected.
Windows only for now. Nemo Assistant relies on Windows-specific libraries (
pywin32,uiautomation, global hotkeys), and CI runs on Windows only. macOS / Linux are not supported yet.
Grab the latest packaged build from the Releases page — no Python setup required. Unzip and run the executable.
Requires Python 3.12+.
git clone https://github.com/SevenBT/nemo-assistant.git
cd nemo-assistant
pip install .
python main.pyFor development:
pip install -e ".[dev]"
python main.pyFor reproducible development with uv:
uv sync --extra dev
uv run pytest -q
uv run python main.pyOn Windows, you can also run:
run.batNemo Assistant uses a layered architecture: desktop tools have independent implementations, while the AI layer connects through a unified tool protocol.
main.py Entry: dependency checks → crash logs → Qt startup
│
├── app/ui/ UI layer, with dedicated controllers for key features
│ ├── main_window Frameless floating main window (chat / notes / workshop)
│ ├── selection_controller Selection capture + action bar
│ ├── screenshot_* Screenshot overlay / OCR / pinned images
│ ├── sticky_note_* Desktop sticky notes
│ ├── text_actions Explain / translate / polish / write back to selection
│ └── components/ Reusable components (message bubbles, Markdown editor, etc.)
│
├── app/core/ Core business layer
│ ├── llm_gateway Unified LLM gateway (LiteLLM / rate limit / retry / streaming)
│ ├── agent_loop Agent state machine (prepare → stream → execute → feedback → finalize)
│ ├── note_manager Notes / stickies / todo storage (SQLite)
│ ├── memory_manager Long-term memory
│ ├── consolidator / dream Conversation compression and background memory consolidation
│ ├── scheduler Scheduled tasks (APScheduler)
│ └── session_manager Multi-session management
│
├── app/tools/ Tool system
│ ├── registry Registration / discovery / execution / error classification and retry
│ ├── script_adapter Dynamic loading for user-defined Python tools
│ └── *.py Built-in tools (search, fetch, files, shell, notes, memory, etc.)
│
└── app/models/ Data models (Message / Session / Note / Memory)
Each session runs in its own QThread, so conversations do not block each other. One turn follows this lifecycle:
prepare → stream → execute → feedback → finalize
prompt stream run tools feed back persist session
Runs can be cancelled at any time, and failures can recover from checkpoints.
All tools inherit from BuiltinTool, declare name / description / parameters / execute, and are managed by ToolRegistry. They are exported in OpenAI Functions format. Tool sources include:
- Built-in tools: shipped with the app.
- User scripts: dynamically loaded after adding
tool.pyto the tools directory. - AI-generated tools: describe a need and let the model help generate tool code.
- GUI: PyQt6 + PyQt-Fluent-Widgets
- Model access: LiteLLM
- Storage: SQLite (notes, memory) + JSON (sessions, config)
- OCR: RapidOCR (ONNX Runtime, local recognition)
- Scheduling: APScheduler
- Other: keyring (secrets), keyboard (global hotkeys), BeautifulSoup (web parsing)
build.batThe script invokes PyInstaller and uses Nemo_Assistant.spec to build the desktop app.
Before publishing a binary release, review docs/en/release-checklist.md.
Some implementation trade-offs from building a frameless floating window and cross-app desktop workflows are documented in docs/en/development-notes.md:
- Dragging uses
startSystemMove(); resizing uses a QApplication event filter +setGeometryto avoid ghost borders fromstartSystemResize(). - Selection capture may inject
Ctrl+C, which can accidentally trigger SIGINT in terminals, so the outer layer handlesKeyboardInterruptand restores the clipboard. - FluentWindow reapplies internal styles, so QTextEdit foreground color is enforced in
focusInEvent. - Embedded-panel confirmation dialogs use native
QMessageBoxinstead of qfluentwidgetsMessageBox, becauseMaskDialogBaseexpects a top-level window.
The wiki-link parser, find/replace, and Markdown syntax highlighting in the note editor are ported / adapted from noteration (MIT). See THIRD_PARTY_NOTICES.md for full third-party license notices.
This project uses third-party open-source code. See THIRD_PARTY_NOTICES.md for copyright and license notices.