Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ Provide a cookbook showing how to prompt LLMs to draft block maps/entity decorat

> Possible changes that require R&D, and high-risk ideas that could bring large benefits but with likely trade-offs.

### Markdown importer

`markdown_to_content_state(markdown, options)` parses Markdown back into Draft.js ContentState, enabling round-trip workflows (ContentState → Markdown → ContentState). The parser is dependency-free and mirrors the exporter's configurable markers. It supports the Markdown subset the exporter produces, plus common CommonMark edge cases (escapes, HR variants, link titles, code fence info strings, multi-line blockquotes, `)` ordered-list delimiters, `~~~` fences). Inline HTML tags (`<u>`, `<sup>`, etc.) emitted by the exporter for non-Markdown styles are parsed back into `inlineStyleRanges`.

### Rust-backed DOM engine

Prototype a Rust extension or PyO3 module for DOM construction to outperform the current string/lxml/html5lib engines on large documents.
Expand Down
16 changes: 15 additions & 1 deletion docs/contributing/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ draftjs_exporter/
html5lib.py # BeautifulSoup / html5lib engine.
lxml.py # lxml engine.
markdown.py # Non-escaping string engine for Markdown output.
markdown/ # Markdown-specific components, config builder, and helpers.
markdown/ # Markdown-specific components, config builder, importer, and helpers.
importer.py # Markdown importer – parses Markdown into ContentState.
utils/
module_loading.py # import_string() for dotted-path class resolution.
```
Expand All @@ -46,6 +47,19 @@ The core flow lives in `HTML.render()` and proceeds as follows:
5. **Wrapper resolution** – `wrapper_state.element_for()` resolves each block to a DOM element, managing nesting of wrapper elements (e.g. `<ul>`/`<ol>` for list items) based on block depth.
6. **Final rendering** – All block elements are appended to the document fragment, then `DOM.render()` serialises the virtual DOM tree to the output string (HTML or Markdown).

## Importer pipeline

The reverse direction — Markdown → Draft.js ContentState — lives in `markdown/importer.py` and is exposed as `markdown_to_content_state(markdown, options=None)`. It is a dependency-free parser that does not reuse the export pipeline's `Command`/`EntityState`/`StyleState` abstractions (those consume ranges; the importer must _produce_ ranges from a different source representation).

The flow:

1. **Block splitting** – `_split_blocks()` splits Markdown into raw block strings, handling code fences, paragraph joining, and continuation lines.
2. **Block parsing** – `_parse_block()` dispatches each raw block to the right handler (heading, blockquote, list, code fence, HR, image, or unstyled fallback) via regex matching.
3. **Inline parsing** – `_InlineParser` walks block text character-by-character, tracking opening/closing of style markers, inline HTML tags, and link syntax. It produces plain text, `inlineStyleRanges`, `entityRanges`, and entity definitions.
4. **Style merging** – `_merge_style_ranges()` merges adjacent or overlapping ranges of the same style. This compensates for the exporter's inline style nesting behavior: when styles partially overlap (e.g. bold 0-5, italic 3-8), the exporter must close and reopen the outer marker to produce valid Markdown (`**Bold ****_Italic_**`), producing two BOLD ranges in the imported output that should be a single continuous range. This is an intentional round-trip trade-off, coupling the importer to the exporter's marker-emission quirks.

The importer accepts `MarkdownImporterOptions` with the three inline style markers (`bold`, `italic`, `strikethrough`) and a custom `html_style_tags` mapping. Block-level syntax (list markers, horizontal rule variants, code fence variants) is recognized polymorphically rather than configured – the importer accepts any valid variant the exporter might produce. See [Markdown support](../markdown.md#importer) for the user-facing options reference.

## Engine system

The exporter uses a **Strategy pattern** for output generation. All engines implement the `DOMEngine` interface (five static methods: `create_tag`, `parse_html`, `append_child`, `render`, `render_debug`). The `DOM` class delegates to the active engine, selected at runtime via dotted-path strings stored as class constants (`DOM.STRING`, `DOM.HTML5LIB`, `DOM.LXML`, `DOM.MARKDOWN`, `DOM.STRING_COMPAT`). Engine selection is thread-safe through a `ContextVar`.
Expand Down
110 changes: 105 additions & 5 deletions docs/markdown.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,12 +150,112 @@ config = {

## Unsupported

The Markdown exporter has inherent limitations compared to the HTML exporter.
The Markdown exporter has inherent limitations compared to the HTML exporter. Where the importer can repair or absorb these limitations for round-trip workflows, that's noted inline; see [Importer](#importer) for the full picture.

- **No underline, subscript, or other HTML-only styles**: The exporters default configuration falls through to inline HTML like `<sup>text</sup>`.
- **No underline, subscript, or other HTML-only styles**: The exporter's default configuration falls through to inline HTML like `<sup>text</sup>`. The importer parses these back into inline style ranges.
- **No reference-style links**: Links are always rendered inline as `[text](url)`.
- **No table support**: Draft.js has no built-in table block type, and the exporter does not attempt to generate Markdown tables from custom block types.
- **No Setext-style headings**: Headings always use ATX style (`# Heading`).
- **Entity data fidelity**: The Markdown link syntax `[text](url)` only preserves the URL.
- **No HTML escaping in text**: If your Draft.js content contains literal HTML characters like like `<` or `>`, the Markdown output will include them unescaped, which may be interpreted as HTML by Markdown renderers.
- **Inline style nesting edge cases**: When bold and italic styles partially overlap, the exporter may produce markers that some strict Markdown parsers reject (e.g. `**Bold **_Italic_**`). Most renderers handle this correctly, but it is not guaranteed by the CommonMark spec.
- **Entity data fidelity**: The Markdown link syntax `[text](url)` preserves the URL and optional title, but other entity data (e.g. `rel`, `data-*` attributes) is lost.
- **No HTML escaping in text**: If your Draft.js content contains literal HTML characters like `<` or `>`, the Markdown output will include them unescaped, which may be interpreted as HTML by Markdown renderers.
- **Inline style nesting edge cases**: When bold and italic styles partially overlap, the exporter may produce markers that some strict Markdown parsers reject (e.g. `**Bold **_Italic_**`). Most renderers handle this correctly, but it is not guaranteed by the CommonMark spec. The importer repairs these by merging adjacent ranges of the same style.

## Importer

> ⚠️ The Markdown importer is **experimental**. There is no guarantee of API stability at this time.

The importer parses Markdown back into Draft.js ContentState, enabling round-trip workflows. It is dependency-free and understands the same subset of Markdown that the exporter produces, including the limitations documented above.

### Quick start

```python
from draftjs_exporter import markdown_to_content_state

content_state = markdown_to_content_state(markdown_text)
```

### Configuring input markers

Pass `MarkdownImporterOptions` matching the options used with `build_markdown_config` to round-trip customized output:

```python
from draftjs_exporter import build_markdown_config, HTML, markdown_to_content_state

options = {"bold": "__", "italic": "*"}
config = build_markdown_config(options)
exporter = HTML(config)

markdown = exporter.render(content_state)
# Pass the same options to the importer.
content_state = markdown_to_content_state(markdown, options)
```

### Importer options

| Option | Choices | Default |
| ----------------- | ------------------------------ | -------------------- |
| `bold` | `**`, `__` | `**` |
| `italic` | `*`, `_` | `_` |
| `strikethrough` | `~`, `~~` | `~` |
| `html_style_tags` | `dict[str, str]` (tag → style) | Inverse of STYLE_MAP |

Only inline style markers are configurable. Block-level syntax (list markers, horizontal rule variants, code fence variants, ordered list delimiters) is **recognized polymorphically** – the importer accepts any of `-`/`*`/`+` for unordered lists, `.`/`)` for ordered lists, `---`/`***`/`___` for horizontal rules, and ` ``` `/`~~~` for code fences. Any block-level keys you pass through from `MarkdownOptions` (`unordered_list_marker`, `ordered_list_delimiter`, `horizontal_rule`, `code_fence`) are simply ignored.

The `html_style_tags` option extends the default HTML tag → style mapping (e.g. `<u>` → `UNDERLINE`). Custom tags are merged with the defaults, so built-in tags still work alongside user-defined ones. This lets you round-trip non-default inline styles configured on the exporter side:

```python
from draftjs_exporter import HTML, build_markdown_config, markdown_to_content_state
from draftjs_exporter.defaults import STYLE_MAP
from draftjs_exporter.dom import DOM

# Register a custom "KBD" inline style on the exporter side.
# (defaults.py ships "KEYBOARD" -> <kbd>; here we re-map it to a custom name.)
STYLE_MAP["KBD"] = lambda props: DOM.create_element("kbd", {}, props["children"])

options = {"html_style_tags": {"kbd": "KBD"}}
config = build_markdown_config(options)
exporter = HTML(config)

markdown = exporter.render(content_state)
# The importer parses <kbd> back as "KBD" rather than the default "KEYBOARD".
content_state = markdown_to_content_state(markdown, options)
```

### Fallback behavior

If the importer encounters a block-level construct it doesn't recognize (anything that doesn't match the heading, list, blockquote, code fence, horizontal rule, or image patterns), the block falls through to `unstyled` and is logged at debug level. This is the expected path for hand-written Markdown (plain paragraphs don't match any of the block-level patterns), so the importer doesn't emit warnings on normal input. Inline syntax that isn't recognized (unknown HTML tags, mismatched markers, unterminated links) is emitted as literal text rather than dropping content.

### Round-trip guarantees

`ContentState → Markdown → ContentState` preserves the following when the default exporter config is used:

- Block types (`unstyled`, headings 1-6, blockquote, list items with depth, code-block, atomic).
- Block text, including soft line breaks within block-level elements.
- `inlineStyleRanges` (offsets, lengths, and style names) for the four Markdown markers (`BOLD`, `ITALIC`, `CODE`, `STRIKETHROUGH`) and the ten inline HTML tag styles (`UNDERLINE`, `SUPERSCRIPT`, `SUBSCRIPT`, `MARK`, `QUOTATION`, `SMALL`, `SAMPLE`, `INSERT`, `DELETE`, `KEYBOARD`).
- `entityRanges` (offsets, lengths, and key mappings) for `LINK`, `IMAGE`, and `HORIZONTAL_RULE` entities.
- Entity `data.url` and, for links, an optional `data.title`.

The following are **not** preserved, because Markdown's link syntax cannot represent them:

- Entity data beyond `url` and `title` (e.g. `rel`, `target`, `data-*` attributes on links).
- Structural entities added by composite decorators during export (e.g. linkified URLs). The importer parses them back as `LINK` entities, but they didn't exist in the original ContentState.
- Atomic block internal structure (e.g. embed payloads).
- Block keys (the importer generates random 5-character keys; Draft.js reassigns keys client-side when loading content into an editor anyway).

### Supported Markdown features

- ATX headings (`#` through `######`)
- Blockquotes (single-line and multi-line)
- Unordered lists (`-`, `*`, `+` with 2-space indent per depth)
- Ordered lists (`.` and `)` delimiters)
- Fenced code blocks (` ``` ` and `~~~`, with optional info strings)
- Horizontal rules (`---`, `***`, `___`)
- Images (`![alt](src)`)
- Links (`[text](url)`, with optional `"title"`)
- Inline styles: bold, italic, code, strikethrough
- Inline HTML tags: `<u>`, `<sup>`, `<sub>`, `<mark>`, `<q>`, `<small>`, `<samp>`, `<ins>`, `<del>`, `<kbd>`
- Backslash escapes for all CommonMark punctuation
- Nested links and styles within link text
- Soft line breaks within block-level elements

The importer does not validate URL schemes. Custom protocols such as `wagtail://core.Page.89` round-trip unchanged in the entity `data.url`. Whether to accept or reject dangerous schemes (e.g. `javascript:`) is the integrating application's responsibility, matching the exporter's behavior – see [SECURITY.md](SECURITY.md).
9 changes: 9 additions & 0 deletions draftjs_exporter/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@
from draftjs_exporter.markdown import CONFIG as MARKDOWN_CONFIG
from draftjs_exporter.markdown import MarkdownOptions as MarkdownOptions
from draftjs_exporter.markdown import build_markdown_config as build_markdown_config
from draftjs_exporter.markdown.importer import (
MarkdownImporterOptions as MarkdownImporterOptions,
)
from draftjs_exporter.markdown.importer import (
markdown_to_content_state as markdown_to_content_state,
)
from draftjs_exporter.types import Block as Block
from draftjs_exporter.types import Component as Component
from draftjs_exporter.types import CompositeDecorators as CompositeDecorators
Expand Down Expand Up @@ -60,6 +66,9 @@
"MARKDOWN_CONFIG",
"MarkdownOptions",
"build_markdown_config",
# Importer
"MarkdownImporterOptions",
"markdown_to_content_state",
# Constants
"BLOCK_TYPES",
"ENTITY_TYPES",
Expand Down
Loading
Loading