From 4fa69447d54f3fa6a97ad20284b89ceffade3e5a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alex=20Hern=C3=A1ndez?= Date: Tue, 16 Jun 2026 10:09:18 +0200 Subject: [PATCH] docs: add front hot-reload, troubleshooting and developer setup guides MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - hot-reload.md: new page covering source mode, daily use, how it works, verification steps and per-editor rebuild fallback; reflects automatic npm ci bootstrap (no manual install step) - troubleshooting.md: new page covering Git/WSL caveats, npm ci EUSAGE, Nextcloud occ permissions, submodule branches and FAQ - setup.md: rewrite with actual make targets (make local), service layout table, first-build commands and front build model explanation - nextcloud.md: add local development section with setup, permissions fix and verification steps - architecture.md: add editing session lifecycle sequence diagram - mkdocs.yml: register hot-reload and troubleshooting in the nav Co-Authored-By: Claude Haiku 4.5 Signed-off-by: Alex Hernández --- docs/development/hot-reload.md | 123 +++++++++++++++++++++++++ docs/development/setup.md | 87 ++++++++++++++---- docs/development/troubleshooting.md | 136 ++++++++++++++++++++++++++++ docs/integration/nextcloud.md | 49 ++++++++++ docs/introduction/architecture.md | 38 ++++++++ mkdocs.yml | 2 + 6 files changed, 418 insertions(+), 17 deletions(-) create mode 100644 docs/development/hot-reload.md create mode 100644 docs/development/troubleshooting.md diff --git a/docs/development/hot-reload.md b/docs/development/hot-reload.md new file mode 100644 index 0000000..110deff --- /dev/null +++ b/docs/development/hot-reload.md @@ -0,0 +1,123 @@ +# Front hot-reload + +The {{ brand.name }} editor front-end (`web-apps`) can be served **from source**, so +editing a file (a template, a view, a stylesheet) is reflected in the editor by a +browser reload — **no build step**. With the optional live-reload server the tab +reloads on its own. + +This mirrors ONLYOFFICE's official "developer mode" (`build_tools/develop`): `api.js`, +`web-apps` and `sdkjs` are all served from the source tree, mutually coherent, instead +of mixing compiled pieces. + +!!! note "Scope" + Hot-reload applies to **`web-apps`** (the UI: toolbar, views, panels, dialogs, + styles) — the bulk of front-end work. The **JS** of `sdkjs` is also served from + source and reloads on a refresh, but the live-reload watcher only watches + `web-apps` by default, and WASM/C++ parts of the SDK are pre-compiled. `server` + and `core` are compiled processes and need a rebuild + restart. + +## One-time setup + +Generate the SDK "develop loader" in the source tree, coherent with the front: + +```bash +cd ~/repos/DocumentServer/develop +make sdkjs-dev +``` + +`make sdkjs-dev` runs `grunt develop` in `sdkjs` and places the generated +`AllFonts.js`, so `/develop/sdkjs` serves a complete SDK that matches your front. +Re-run it only when you change the `sdkjs` submodule. `make front-dev` refuses to +start if this step has not been run, so you can't accidentally serve a half-built SDK. + +!!! note "Live-reload dependencies install themselves" + The first `make front-dev-live` runs `npm ci` for the watcher + (`develop/setup/livereload-tools`) automatically when `node_modules` is missing — + no manual install step. The lockfile is committed, so the install is reproducible + across machines. + +## Daily use + +```bash +make front-dev-live # source mode + live-reload server +# edit e.g. web-apps/apps/documenteditor/main/app/template/Toolbar.template +# (change a class/id), SAVE → the editor tab reloads itself +make front-prod # back to production (compiled) mode when done +``` + +Variants: + +- `make front-dev` — source mode **without** auto-reload (you press ++ctrl+shift+r++). +- `make front-prod` — restore the production (compiled) editor and versioned cache. + +## How it works + +All of this lives in `develop/setup/ds-docservice-dev.conf` (swapped in by the make +targets) and `develop/setup/livereload-tools/`: + +1. **Serve from source.** The dev nginx config serves `/web-apps/` and `/sdkjs/` from + `/develop` with `no-store`, and drops the production version-redirect. +2. **No version prefix.** `api.js` is served from source. DocsAPI only adds the + `/X.Y.Z-/` version prefix when `api.js` is *compiled* (its + `{{ '{{PRODUCT_VERSION}}' }}` placeholder substituted). Served from source, the + placeholder is intact, so **no prefix is added** — the editor loads unversioned and + the SDK load order (synchronous `document.write` → RequireJS) works as designed. +3. **SDK from source.** `make sdkjs-dev` generates the develop loader under + `/develop/sdkjs`. The dev config falls back to `$EO_ROOT` for generated binaries + (`.wasm`, `theme.bin`) that are not present in the source tree. +4. **Auto live-reload.** A small `livereload` Node server watches + `/develop/web-apps/apps` in **polling** mode (inotify events do not cross the + Windows↔WSL boundary reliably; polling catches saves from any editor). The client + is injected via nginx `sub_filter` and the websocket is proxied through `:8080`, so + no extra ports are exposed and the container is not recreated. + +!!! info "WSL2" + Because the browser fetches from nginx on each reload (and nginx reads the + bind-mounted source live), hot-reload does **not** depend on `inotify`. Saving from + Windows VS Code over `\\wsl.localhost\` works. + +## Verify it + +Open a document from Nextcloud and check **DevTools → Network**: + +- Editor URLs are `…:8080/web-apps/…` and `…:8080/sdkjs/…` **without** a `/9.3.1-…/` + prefix. +- You see `app_dev.js` plus individual `.js` / `.template` files and SDK sources + (first load is slower). +- Edit a template, save → the tab reloads and the change is visible, with no console + errors. + +!!! tip "First run: clear stale cache" + If you previously opened the editor in compiled mode, clear the browser state once: + DevTools → Application → Service Workers → **Unregister**, then **Clear site data**, + and keep **Disable cache** ticked in Network. The dev config also neutralises the + editor service worker. + +!!! note "First load is slower" + In source mode the SDK loads as many individual files, so the first time you open an + editor is slower than the compiled bundle; subsequent reloads are fast. If the editor + ever fails to open after an upstream change (e.g. an SDK/front mismatch), re-run + `make sdkjs-dev` or fall back to the per-editor rebuild below. + +## Fallback: fast per-editor rebuild + +Not instant, but the officially supported cycle and **always works**: rebuild only the +editor you touched and flush the nginx cache (tens of seconds): + +```bash +docker compose exec eo bash -lc 'cd /develop/web-apps/build && \ + BUILD_ROOT=$EO_ROOT grunt --skip-babel --skip-imagemin --skip-sprites deploy-documenteditor && \ + HASH=$(date +%s|openssl md5|cut -d" " -f2); echo "set \$cache_tag \"$HASH\";">/etc/nginx/includes/ds-cache.conf; \ + service nginx reload' +``` + +Targets exist per editor (`deploy-documenteditor`, `deploy-spreadsheeteditor`, …) and +`grunt less-all` rebuilds styles only. The `ds-cache.conf` re-tag is required, otherwise +the browser keeps the cached bundle. Then hard-reload the browser. + +| | Source mode | Per-editor rebuild | +|---|---|---| +| "save → visible" latency | auto reload (~1–2 s, no build) | rebuild 1 editor (tens of s) + reload | +| commands per change | none (`make front-dev-live` once) | one per change | +| SDK | from source (develop loader) | compiled | +| best for | intensive UI iteration | real builds / SDK changes | diff --git a/docs/development/setup.md b/docs/development/setup.md index e4b16c6..905fb8e 100644 --- a/docs/development/setup.md +++ b/docs/development/setup.md @@ -3,40 +3,93 @@ The full {{ brand.name }} stack runs locally via the docker-compose-based environment in [`DocumentServer/develop/`]({{ brand.repo }}/DocumentServer/tree/main/develop). -It builds each component from source, exposes hot-reload paths for the editor -UI, and includes a paired Nextcloud instance for end-to-end testing. +It builds each component from source, can serve the editor UI from source for +[hot-reload](hot-reload.md), and includes a paired Nextcloud instance for +end-to-end testing. ## Prerequisites - Docker 24+ with `buildx`. - 16 GB RAM minimum (the first build is heavy). -- Git with SSH access to `{{ brand.repo }}`. +- Git with access to `{{ brand.repo }}`. - A Linux x86_64 or arm64 host. macOS works for everything except native-core - C++ debugging. + C++ debugging. On Windows, use WSL2 (see [Troubleshooting](troubleshooting.md) + for the host/WSL Git caveats). -## Clone and bootstrap +## Clone and start ```bash git clone --recurse-submodules {{ brand.repo }}/DocumentServer.git cd DocumentServer/develop -./scripts/up.sh # placeholder — exact entrypoint TBD +make local ``` +`make local`: + +1. Brings up the containers (`docker compose up -d`). +2. Waits for the document server to answer `/healthcheck` and for Nextcloud to + finish installing. +3. Runs `refresh-urls`, which sets `DocumentServerUrl`, `trusted_domains` and the + shared `jwt_secret` in Nextcloud — **always as `www-data`**. +4. Drops you into a shell inside the `eo` container. + ## Layout of the dev environment -The compose file spins up: +The compose file (`develop/docker-compose.yml`) starts: + +| Service | Role | URL | +|---|---|---| +| `eo` | {{ brand.name }} document server, built from local source (image `{{ brand.image }}:latest-dev`) | `http://localhost:8080` | +| `nextcloud` | Storage provider with the {{ brand.name }} integration app | `http://localhost:8081` | +| `onlyoffice` | Stock ONLYOFFICE, for comparison/regression | `http://localhost:8082` | + +PostgreSQL, Redis and RabbitMQ run **inside** the `eo` container. The repository +root is bind-mounted into `eo` at `/develop` (`EO_DEV`), and the compiled/deployed +tree that nginx serves lives at `$EO_ROOT` +(`/var/www/{{ brand.product_slug }}/documentserver` by default). -- {{ brand.name }} document server (built from local source). -- Nextcloud (with the {{ brand.name }} integration app pre-installed). -- PostgreSQL, Redis, RabbitMQ. -- An nginx front-end with self-signed TLS. +## First build + +The pre-built dev image already contains a working editor. To rebuild a component +from your local source, run the internal Makefile inside the container: + +```bash +# you are already inside eo after `make local`; otherwise: docker compose exec eo bash +make -f /Makefile web-apps # front-end: npm ci + full grunt build +make -f /Makefile sdkjs # document engine (JS/WASM) +make -f /Makefile server # Node backend (docservice, converter, …) +``` + +The compiled front lands in `$EO_ROOT/web-apps/…`, which nginx serves by default. + +## Front build model (compiled vs source) + +The editor nginx serves in production is **not your sources**: it is a single +`app.js` bundle optimised by **r.js (RequireJS)** and minified, with `.template` +files inlined at build time and LESS compiled to CSS. + +Each editor ships two HTML entry points: + +- `index.html` — **source / dev mode**: `data-main="app_dev"`, RequireJS loads each + module individually from source, templates via the `text!` plugin at runtime, + LESS compiled in the browser. The SDK is loaded as global scripts listed by a + *develop loader*. +- `index.html.deploy` — **compiled mode**: loads the optimised `app.js` bundle. + The build copies it over `index.html` on deploy. + +Serving the **source** entry point is what enables [hot-reload](hot-reload.md): +edit a file, reload, see the change — no build. ## Mobile / non-localhost testing -When testing from a phone or a separate machine, override the public URL via -the docker-compose `.env` so the editor connects through your LAN IP. +`make local` runs against `localhost`. To reach the editor from a phone, emulator +or another machine on the LAN, use `make mobile` (it injects the detected host LAN +IP into Nextcloud's `DocumentServerUrl` and `trusted_domains`). After an IP change, +re-run `make refresh-urls`. + +## Next steps -!!! note "Coming soon" - The full reference for the develop environment, migrated from - [`DocumentServer/develop/README.md`]({{ brand.repo }}/DocumentServer/blob/main/develop/README.md). - That document is currently the source of truth. +- [Front hot-reload](hot-reload.md) — edit the UI and see it without rebuilding. +- [Troubleshooting](troubleshooting.md) — Git/WSL, `npm ci`, Nextcloud recovery. +- [Building from source](building.md) — produce the Docker image and packages. +- [Nextcloud integration](../integration/nextcloud.md) — the connector app. diff --git a/docs/development/troubleshooting.md b/docs/development/troubleshooting.md new file mode 100644 index 0000000..37290cc --- /dev/null +++ b/docs/development/troubleshooting.md @@ -0,0 +1,136 @@ +# Troubleshooting + +Common issues when running the local dev environment, and how to resolve them. + +## Git: hundreds of "modified" files in VS Code (Windows + WSL2) + +**Symptom:** VS Code (on Windows, opening the repo over `\\wsl.localhost\`) shows +hundreds of modified files and doesn't reflect real changes. + +**Cause:** Windows VS Code uses the **Windows `git.exe`** with its own global config +(`C:\Users\\.gitconfig`), typically `core.autocrlf=true` and no `safe.directory`. +Reading LF files (from the WSL ext4 filesystem) with `autocrlf=true` and no +`.gitattributes` flags hundreds of files as modified, and "dubious ownership" prevents +correct status. The WSL git sees only the real changes. + +**Fix.** In **PowerShell/CMD on Windows** (not WSL), then reload VS Code: + +```powershell +git config --global core.autocrlf false +git config --global core.filemode false +git config --global --add safe.directory '*' +``` + +Apply the same in WSL for terminal consistency: + +```bash +git config --global --add safe.directory '*' +git config --global core.filemode false +git config --global core.autocrlf false +``` + +!!! tip + Opening the project with the **WSL** extension (VS Code runs its server inside WSL + and uses the WSL git) avoids the whole class of host↔guest discrepancies. + +## Git: "dubious ownership" + +The container writes into the `/develop` bind-mount with its own UID; a build running as +another UID can leave submodules owned by a foreign user. Fix ownership **without sudo** +from the container (the bind-mount writes to the host): + +```bash +HUID=$(id -u); HGID=$(id -g) +docker exec -u root eo bash -lc "chown -R $HUID:$HGID /develop/sdkjs" +``` + +## Git: build-generated files keep dirtying the tree + +The `web-apps` build rewrites `apps/*/main/locale/*.json` (English backfill via +`merge_and_check.py`), regenerates `icons.svg`, and bumps the `build` number in +`build/*.json`. None of this should be committed. Hide them locally (reversible): + +```bash +cd web-apps +git ls-files 'apps/*/main/locale/*.json' | xargs git update-index --skip-worktree +git ls-files 'apps/*/main/resources/img/toolbar/icons.svg' | xargs git update-index --skip-worktree +# revert with: … | xargs git update-index --no-skip-worktree +``` + +!!! note + In [`front-dev-live`](hot-reload.md) mode nothing is compiled, so the tree does not + get dirtied while you develop the front-end. + +## Submodule branches + +Submodules in *detached HEAD* is normal, but to commit you'll want named branches. Keep +them on the commit the superproject pins (the coherent `v…-rc` set) rather than moving +individual submodules to `develop`/`main`, which can diverge and break cross-component +compatibility: + +```bash +# in a submodule already at the pinned commit: +git checkout -B HEAD # named branch at the current commit, code unchanged +``` + +## `npm ci` aborts with `EUSAGE` + +`npm ci` is strict: it aborts if `package.json` and `package-lock.json` aren't perfectly +consistent. This happens when the lock was generated on a different npm version or +**platform** (a lock made on Windows/macOS lacks the Linux optional binaries). Always +regenerate the lock **inside the Linux container**: + +```bash +docker compose exec eo bash -lc 'cd /develop/web-apps/build && npm install --package-lock-only' +docker compose exec eo bash -lc 'cd /develop/web-apps/build && npm ci --dry-run >/dev/null && echo "LOCK OK"' +git -C web-apps add build/package-lock.json +git -C web-apps commit -m "build: regenerate package-lock.json inside Linux container" +``` + +Never hand-edit the lock; regenerate it in the same commit whenever `package.json` changes. + +## Nextcloud returns 500 after running `occ` as root + +!!! danger "Always run `occ` as `www-data`" + Apache runs as `www-data`. Running `occ` as **root** creates root-owned files under + `config/`, `data/`, `apps/` → Apache can no longer read/write its config → 500. + +Use an alias so you never slip: + +```bash +alias occ='docker compose -f ~/repos/DocumentServer/develop/docker-compose.yml exec -u www-data nextcloud php occ' +``` + +Recovery sequence: + +```bash +cd ~/repos/DocumentServer/develop +docker compose exec -u www-data nextcloud php occ maintenance:mode --on || true +docker compose exec -u root nextcloud bash -c ' + chown -R www-data:www-data /var/www/html/config /var/www/html/data /var/www/html/apps + find /var/www/html/custom_apps -maxdepth 1 -mindepth 1 ! -name {{ brand.product_slug }} \ + -exec chown -R www-data:www-data {} +' +docker compose restart nextcloud +docker compose exec -u www-data nextcloud php occ maintenance:repair --include-expensive +docker compose exec -u www-data nextcloud php occ maintenance:mode --off +docker compose exec -u www-data nextcloud php occ status +``` + +Read the exact error if it persists: + +```bash +docker compose exec nextcloud tail -n 50 /var/www/html/data/nextcloud.log +``` + +## FAQ + +- **"I edit and don't see the change."** Compiled mode ([§ rebuild](hot-reload.md)): + you missed the `ds-cache.conf` re-tag or a hard reload. Source mode: check Network + serves unversioned modules with `no-store`. +- **"It doesn't reload on its own."** Is the live-reload server alive? + `docker compose exec eo cat /tmp/livereload.log` should log `change:` on save. +- **"`Asc is not defined` / version prefix in URLs."** `api.js` is being served compiled — + make sure source mode is active and clear the service worker + site data. +- **"+ New buttons gone in Nextcloud."** The integration app isn't enabled, or the + handshake with the document server failed — see + [Nextcloud integration](../integration/nextcloud.md#local-development). diff --git a/docs/integration/nextcloud.md b/docs/integration/nextcloud.md index 2d18b1d..539cd20 100644 --- a/docs/integration/nextcloud.md +++ b/docs/integration/nextcloud.md @@ -27,3 +27,52 @@ Nextcloud app that connects to a running {{ brand.name }} document server. The Nextcloud integration app lives at [`{{ brand.repo }}/{{ brand.product_slug }}-nextcloud`]({{ brand.repo }}/{{ brand.product_slug }}-nextcloud). + +## Local development + +In the [dev environment](../development/setup.md), the integration app is mounted into +Nextcloud from a sibling checkout (`~/repos/{{ brand.product_slug }}-nextcloud`), not +installed from the App Store. If that directory is empty the app can't be enabled and +the **+ New** buttons disappear. + +**Set it up once:** + +```bash +# 1) clone WITH submodules (it needs document-formats / document-templates) +cd ~/repos +git clone --recurse-submodules {{ brand.repo }}/{{ brand.product_slug }}-nextcloud.git + +# 2) build: frontend (vite → js/) + PHP deps (firebase/php-jwt → vendor/) +cd {{ brand.product_slug }}-nextcloud +npm install && npm run build +composer install --no-dev + +# 3) enable it (as www-data) and align URLs/JWT +cd ~/repos/DocumentServer/develop +docker compose exec -u www-data nextcloud php occ app:enable {{ brand.product_slug }} +make refresh-urls +``` + +!!! warning "App Store / permissions" + Nextcloud requires the `apps_paths` entry marked `writable: true` to be writable. + The dev mount creates `custom_apps` as `root:root`, so `occ app:enable` fails with + *"Cannot write into apps directory"*. Give it to `www-data` and disable the App + Store (not needed in dev): + + ```bash + docker compose exec -u root nextcloud chown www-data:www-data /var/www/html/custom_apps + docker compose exec -u root nextcloud chmod 775 /var/www/html/custom_apps + docker compose exec -u www-data nextcloud php occ config:system:set appstoreenabled --type=boolean --value=false + ``` + +!!! note "`/apps/files/` shows a 500 (`getFormats(): … null`)" + You cloned the app **without** its submodules. Fix it: + + ```bash + cd ~/repos/{{ brand.product_slug }}-nextcloud && git submodule update --init --recursive + docker compose -f ~/repos/DocumentServer/develop/docker-compose.yml restart nextcloud + ``` + +**Verify:** `occ app:list | grep {{ brand.product_slug }}` lists the app; in +`http://localhost:8081` the **+ New** menu offers Document / Spreadsheet / Presentation. +See [Troubleshooting](../development/troubleshooting.md) for `occ`/permissions recovery. diff --git a/docs/introduction/architecture.md b/docs/introduction/architecture.md index b9680aa..67892dd 100644 --- a/docs/introduction/architecture.md +++ b/docs/introduction/architecture.md @@ -51,6 +51,44 @@ built per document type and served as static assets. See A C++ rendering and conversion engine. Output artifacts are consumed by FileConverter at runtime. See [components → core](components.md). +## Editing session lifecycle + +Opening and saving a document is a fully decoupled, asynchronous flow between the +storage host (e.g. Nextcloud), the user's browser, and the document server: + +```mermaid +sequenceDiagram + participant NC as Storage host
(+ integration app) + participant B as Browser (editor) + participant DS as DocService + participant CV as FileConverter + NC->>B: 1. Open doc — signed JWT (permissions, metadata, CallbackUrl) in an iframe + B->>DS: 2. Load editor assets (web-apps) + engine (sdkjs) + DS->>NC: 3a. Download source file (JWT-authorized) + B-->>DS: 3b. Edits stream over WebSocket (Socket.io) + Note over DS: changesets held in Redis;
inter-service coordination via RabbitMQ + Note over DS: last client leaves → grace timer + DS->>CV: 4a. Apply changesets, repack to native format + DS->>NC: 4b. POST to CallbackUrl + Note over NC: validate signature → write new file version +``` + +1. **Initialization & JWT.** When a user opens a document, the storage host's + integration app issues a **JWT** signed with the shared secret, carrying the user's + permissions, file metadata and a `CallbackUrl`, and embeds an `iframe` pointing at the + document server. +2. **Asset loading.** The browser loads the editor UI (`web-apps`) and the document + engine (`sdkjs`) from the document server; the editor initializes under the global + `Asc` namespace. +3. **Live editing.** DocService downloads the source file from storage using the JWT, + then exchanges edits with the browser as **changesets** over a WebSocket + (Socket.io). Session state is kept in **Redis** and inter-service messaging goes + through **RabbitMQ**, so collaborators see each other's edits without disk round-trips. +4. **Save & persistence.** After the last client disconnects (plus a short grace timer), + **FileConverter** applies the accumulated changesets and repacks the native binary; + DocService then POSTs it to the `CallbackUrl`, and the storage host validates the + signature and stores a new file version. + ## Scaling out For multi-node deployments DocService, FileConverter, and storage scale diff --git a/mkdocs.yml b/mkdocs.yml index f2f3126..c3849bf 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -110,5 +110,7 @@ nav: - Plugins: integration/plugins.md - Development: - development/setup.md + - Front hot-reload: development/hot-reload.md - Building from source: development/building.md + - Troubleshooting: development/troubleshooting.md - development/contributing.md