Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ coverage:
target: 90
patch:
default:
informational: true
target: 100
codecov:
branch: main
Expand Down
189 changes: 189 additions & 0 deletions docs/admin/boost-weblate.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
.. _boost-weblate:

Boost Weblate additions
=======================

This repository extends upstream Weblate with capabilities used for translating
`Boost C++ Libraries <https://www.boost.org/>`_ documentation: QuickBook and
AsciiDoc handling tailored for Boost workflows, optional OpenRouter-based batch
machine translation, and a REST surface for CI-driven component maintenance.

The sections below document fork-specific **configuration**, **dependencies**,
and **HTTP endpoints** that are not covered by generic Weblate documentation.

.. seealso::

Standard administrator guides still apply: :doc:`install/docker`,
:ref:`docker-environment`, :doc:`machine`, and :doc:`config`.

Python packages
---------------

OpenRouter batch translation uses the `OpenAI Python SDK <https://pypi.org/project/openai/>`_
(`OpenAI Client`) against the OpenRouter HTTP API. The SDK is **not** part of
core Weblate dependencies; install it explicitly:

.. code-block:: sh

pip install 'weblate[openai]'
# or
pip install 'openai>=2.0,<3.0'

If the SDK is missing when OpenRouter translation runs, Weblate raises
``django.core.exceptions.ImproperlyConfigured`` with an installation hint.

Docker images built from :file:`weblate-docker/Dockerfile` use
``WEBLATE_EXTRAS=all`` so the ``openai`` extra is included in the container.

System commands and packages
----------------------------

The following executables must be available on the server **PATH** where the
relevant code paths execute (web workers, Celery workers):

.. list-table::
:header-rows: 1
:widths: 22 78

* - Executable
- Used by
* - ``git``
- Boost endpoint service: clone repositories, commit and push translation
changes (see ``weblate.boost_endpoint.services``).
* - ``po4a-gettextize``, ``po4a-translate``
- AsciiDoc format pipeline (``weblate.formats.asciidoc``).
* - ``msgattrib``, ``msgfmt``
- gettext toolchain for AsciiDoc save path; ``msgattrib`` is optional (the
code falls back if absent).

The official Docker image for this fork installs **po4a** from source during the
image build (see comments in :file:`weblate-docker/Dockerfile`). Custom or
bare-metal installs must provide **po4a** and **gettext** separately (for
example distribution packages for ``po4a`` and ``gettext``).

Environment variables
---------------------

These variables apply to **Boost fork** behaviour. They do **not** use the
``WEBLATE_`` prefix. Standard Docker variables remain documented under
:ref:`docker-environment`.

.. envvar:: OPENROUTER_API_KEY

API key used when OpenRouter batch translation cannot read credentials from
Weblate’s machinery configuration (see :ref:`boost-weblate-openrouter-config`).
Read by ``weblate.trans.autobatchtranslate``.

.. envvar:: OPENROUTER_MODEL

Model identifier passed to OpenRouter (for example ``deepseek/deepseek-chat``).
Default if unset: ``deepseek/deepseek-chat``. Used together with
:envvar:`OPENROUTER_API_KEY` as an environment fallback.

.. envvar:: AUTO_BATCH_TRANSLATE_VIA_OPENROUTER

Boolean interpreted by :file:`weblate/settings_docker.py`. When ``true``
(Docker default), components may trigger automatic batch translation via
OpenRouter according to internal workflows. When ``false``, that behaviour is
disabled. For non-Docker installs, set ``AUTO_BATCH_TRANSLATE_VIA_OPENROUTER``
in :file:`settings.py`.

.. envvar:: BOOST_ENDPOINT_ADD_TRANSLATION_SECONDS

Integer seconds to wait when the Boost endpoint waits for a component or
translation to become ready before adding a language (polling interval is
derived from this setting in ``weblate.boost_endpoint.services``).
Default in Docker: ``300``. Override per deployment if repositories are slow
or fast to sync.

.. _boost-weblate-openrouter-config:

OpenRouter credentials (batch translation)
------------------------------------------

Batch OpenRouter translation resolves configuration in this order:

#. **Weblate machinery settings** — category MT, ``openai`` entry with ``key``
(API key) and ``custom_model`` (model id). This mirrors fields used for the
generic OpenAI-compatible machinery documented under :ref:`mt-openai`.
#. **Environment variables** — :envvar:`OPENROUTER_API_KEY` and
:envvar:`OPENROUTER_MODEL` when the database configuration does not supply both
values.

If no usable key and model are found, auto-translation is skipped and a warning
is logged.

REST API: ``/boost-endpoint/``
-------------------------------

These endpoints are **not** part of the ``/api/`` namespace and are **not**
included in the OpenAPI schema served at ``/api/schema/``. They require an
authenticated user (same token mechanism as :ref:`api-tokens`).

Base path (relative to your site root): ``/boost-endpoint/``.

.. http:get:: /boost-endpoint/

Returns a short JSON description of the Boost endpoint module.

:reqheader Authorization: ``Token …`` (required)

:status 200:

.. code-block:: json

{
"module": "boost-endpoint",
"description": "Boost documentation translation API"
}

.. http:post:: /boost-endpoint/add-or-update/

Accepts a job description and enqueues asynchronous work on a Celery worker.
The HTTP response returns immediately with a task identifier.

:reqheader Authorization: ``Token …`` (required)
:reqheader Content-Type: ``application/json``

:<json string organization: GitHub organisation hosting Boost library repos (example: ``CppDigest``).
:<json object add_or_update: Mapping from Weblate language code to a list of repository names (submodules) to process for that language. Must be non-empty.
:<json string version: Boost branch or tag name (example: ``boost-1.90.0``).
:<json array extensions: Optional list of file extensions to scan (example: ``[".adoc", ".md"]``). Only extensions Weblate recognises are used; omit or leave empty to allow all supported extensions.

:status 202: Job accepted; processing continues in Celery.

.. code-block:: json

{
"status": "accepted",
"task_id": "<celery-task-uuid>",
"detail": "Boost add-or-update is running in the background; check Celery logs or task result for completion."
}

:status 400: Validation error.

.. code-block:: json

{ "errors": { "...": ["..."] } }

Related Django settings
-----------------------

The following settings appear in :file:`weblate/settings_example.py` for
non-Docker deployments:

``AUTO_BATCH_TRANSLATE_VIA_OPENROUTER``
Enables or disables OpenRouter batch translation hooks. Defaults to
``False`` in the example settings file; Docker defaults differ via
:envvar:`AUTO_BATCH_TRANSLATE_VIA_OPENROUTER`.

``BOOST_ENDPOINT_ADD_TRANSLATION_SECONDS``
Delay used when waiting for components during Boost endpoint processing.
Example file sets ``150`` seconds; Docker overrides via
:envvar:`BOOST_ENDPOINT_ADD_TRANSLATION_SECONDS` unless customised.

File formats
------------

* :doc:`../formats/quickbook` — QuickBook ``.qbk`` (fork-specific).
* :doc:`../formats/asciidoc` — AsciiDoc (implementation notes including **po4a**).
12 changes: 6 additions & 6 deletions docs/admin/machine.rst
Original file line number Diff line number Diff line change
Expand Up @@ -369,15 +369,15 @@ Weblate supports DeepL formality, it will choose matching one based on the
language (for example, there is ``de@formal`` and ``de@informal``).

The translation context can optionally be specified to improve translations quality. Read more on that in
`DeepL translation context documentation <https://developers.deepl.com/docs/best-practices/working-with-context>`_.
`DeepL translation context documentation <https://developers.deepl.com/docs/learning-how-tos/examples-and-guides/how-to-use-context-parameter>`_.

The service automatically uses :ref:`glossary`, see :ref:`glossary-mt`.

.. seealso::

* `DeepL translator <https://www.deepl.com/translator>`_
* `DeepL pricing <https://www.deepl.com/pro>`_
* `DeepL API documentation <https://developers.deepl.com/docs/getting-started/intro>`_
* `DeepL translator <https://www.deepl.com/en/translator>`_
* `DeepL pricing <https://www.deepl.com/en/pro>`_
* `DeepL API documentation <https://developers.deepl.com/docs/getting-started/quickstart>`_

.. _mt-glosbe:

Expand Down Expand Up @@ -622,7 +622,7 @@ You can also specify a custom category to use `custom translator <https://learn.
.. seealso::

* `Cognitive Services - Text Translation API <https://azure.microsoft.com/en-us/products/ai-services/ai-translator>`_
* `Microsoft Azure Portal <https://portal.azure.com/>`_
* `Microsoft Azure Portal <https://learn.microsoft.com/en-us/azure/azure-portal/>`_
* `Base URLs <https://learn.microsoft.com/en-us/azure/ai-services/translator/text-translation/reference/v3/reference#base-urls>`_
* `"Authenticating with a Multi-service resource" <https://learn.microsoft.com/en-us/azure/ai-services/translator/text-translation/reference/authentication#authenticating-with-a-multi-service-resource>`_
* `"Authenticating with an access token" section <https://learn.microsoft.com/en-us/azure/ai-services/translator/text-translation/reference/authentication#authenticating-with-an-access-token>`_
Expand Down Expand Up @@ -1076,7 +1076,7 @@ This service uses an API, and you need to obtain an ID and an API key from Youda

.. seealso::

`Youdao Zhiyun Natural Language Translation Service <https://ai.youdao.com/product-fanyi-text.s>`_
`Youdao Zhiyun Natural Language Translation Service <https://ai.youdao.com/new/product-fanyi-text.s>`_

.. _custom-machinery:

Expand Down
5 changes: 5 additions & 0 deletions docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,11 @@ can browse at ``/api/docs/``.
incomplete at this point and subject to change. Please consult the
documentation below for more detailed information on the API.

.. note::

**Boost Weblate fork:** authenticated endpoints under ``/boost-endpoint/``
(outside ``/api/``) are documented in :doc:`admin/boost-weblate`.

.. _api-generic:

Authentication and generic parameters
Expand Down
53 changes: 52 additions & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,53 @@ def setup(app) -> None:

# Number of retries and timeout for linkcheck
linkcheck_retries = 10
linkcheck_timeout = 10
# Default 10s is tight for some CDNs from GitHub Actions (e.g. contributor-covenant.org).
linkcheck_timeout = 45

# Treat these redirects as working. Linkcheck emits redirects as warnings; CI runs
# sphinx with --fail-on-warning, so expected HTTP redirects must be allowed here
# (or links updated in RST). Patterns use re.match() from the URL start.
linkcheck_allowed_redirects = {
(
r"https://support\.okta\.com/help/s/article/"
r"How-to-send-a-custom-relaystate-to-application-through-idp-initiated-authentication-urls"
): (
r"https://support\.okta\.com/help/s/article/"
r"How-to-send-a-custom-relaystate-to-application-through-idp-initiated-authentication-urls(\?.*)?"
),
r"https://docs\.djangoproject\.com/en/stable/.*": (
r"https://docs\.djangoproject\.com/en/[0-9]+\.[0-9]+/.*"
),
r"https://weblate\.org/?.*": r"https://weblate\.org/.*",
r"https://docs\.weblate\.org/?.*": r"https://docs\.weblate\.org/.*",
r"https://hosted\.weblate\.org.*": r"https://hosted\.weblate\.org.*",
r"https://www\.sphinx-doc\.org/?$": r"https://www\.sphinx-doc\.org/en/master/?",
r"https://angular\.io/.*": r"https://.*\.angular\.io/.*",
r"https://babel\.pocoo\.org/?$": r"https://babel\.pocoo\.org/en/latest/.*",
r"https://cryptography\.io/?$": r"https://cryptography\.io/en/latest/.*",
r"https://docs\.celeryq\.dev/?$": r"https://docs\.celeryq\.dev/en/stable/.*",
r"https://docs\.phpmyadmin\.net/?$": r"https://docs\.phpmyadmin\.net/en/latest/.*",
r"https://doc\.galette\.eu/?$": r"https://doc\.galette\.eu/en/master/.*",
r"https://pytest\.org/?$": r"https://docs\.pytest\.org/.*",
r"https://python-social-auth\.readthedocs\.io/?$": (
r"https://python-social-auth\.readthedocs\.io/en/latest/.*"
),
r"https://sentry\.io/?$": r"https://sentry\.io/.*",
r"https://ruby-doc\.org/current/.*": r"https://ruby-doc\.org/[0-9.]+/.*",
r"https://docs\.anthropic\.com/.*": r"https://.*\.claude\.com/.*",
r"https://console\.anthropic\.com/.*": r"https://.*\.claude\.com/.*",
r"https://console\.cloud\.google\.com/.*": r"https://accounts\.google\.com/.*",
r"https://console\.developers\.google\.com/.*": r"https://accounts\.google\.com/.*",
r"https://gitee\.com/help/.*": r"https://help\.gitee\.com.*",
r"https://git\.cloudron\.io/.*": r"https://git\.cloudron\.io/.*",
r"https://github\.com/[^/]+/[^/]+/security/advisories/new(\?.*)?$": (
r"https://github\.com/login.*"
),
r"https://www\.bestpractices\.dev/en/projects/[0-9]+/?$": (
r"https://www\.bestpractices\.dev/en/projects/[0-9]+/passing"
),
}

linkcheck_ignore = [
# Local URL to Weblate
"http://127.0.0.1:8080/",
Expand Down Expand Up @@ -402,6 +448,11 @@ def setup(app) -> None:
"https://dev.mysql.com/",
# Responds with HTTP 418 I'm a teapot
"https://www.freedesktop.org/",
# 403 to automated clients (URLs remain valid in browsers)
"https://mymemory\\.translated\\.net/.*",
"https://docs\\.oasis-open\\.org/.*",
# Captcha / bot wall in CI; human documentation links remain valid
"https://cloud\\.yandex\\.com/.*",
]

# HTTP docs
Expand Down
18 changes: 17 additions & 1 deletion docs/formats.rst
Original file line number Diff line number Diff line change
Expand Up @@ -346,6 +346,22 @@ Translation types capabilities
- no
- no
-
* - :ref:`asciidoc`
- mono
- no
- no
- no
- no
- no
-
* - :ref:`quickbook`
- mono
- no
- no
- no
- no
- no
-

.. [#m] See :ref:`bimono`
.. [#p] See :ref:`format-plurals`
Expand All @@ -359,7 +375,7 @@ Translation types capabilities
.. [#lp] The plurals are supported only for Laravel which uses in string syntax to define them, see `Localization in Laravel`_.
.. [#fp] Plurals are handled in the syntax of the strings and not exposed as plurals in Weblate.

.. _Localization in Laravel: https://laravel.com/docs/localization
.. _Localization in Laravel: https://laravel.com/docs/13.x/localization

.. _bimono:

Expand Down
11 changes: 11 additions & 0 deletions docs/formats/asciidoc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,17 @@ The translatable content is extracted from the AsciiDoc files and offered for th

.. include:: /snippets/format-database-backed.rst

System dependencies (Boost Weblate)
+++++++++++++++++++++++++++++++++++

This implementation extracts and merges translations using **po4a**
(``po4a-gettextize``, ``po4a-translate``) and the gettext utilities ``msgattrib``
and ``msgfmt``. Install the corresponding system packages on application and
Celery hosts, or use the Docker image built from this repository (po4a is
installed during the image build—see :file:`weblate-docker/Dockerfile`).

Full operational notes: :doc:`../admin/boost-weblate`.

.. seealso::

:doc:`tt:formats/asciidoc`
Expand Down
2 changes: 1 addition & 1 deletion docs/formats/laravel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The Laravel PHP localization files are supported as well with plurals:
* :doc:`tt:formats/php`
* `Localization in Laravel`_

.. _Localization in Laravel: https://laravel.com/docs/localization
.. _Localization in Laravel: https://laravel.com/docs/13.x/localization

Weblate configuration
+++++++++++++++++++++
Expand Down
Loading
Loading