Skip to content

Finish the generated-docstring cleanup in ingest/pipeline.py #136

Description

@Paul-Kyle

If you'd like to take this, please comment here first and wait to be assigned before opening a
PR.
Two contributors landed on the same issue yesterday because we had no such line and no
assignment; that was our fault, and this is the fix.

#130 asked for five specific docstring lines in palinode/ingest/pipeline.py to be rewritten and
#131 did exactly that. But five lines was the wrong scope: the file has roughly twenty-five lines in
the same generated register, and because #130 named individual lines rather than docstrings, two of
them are now half-converted — plain English beside word-salad inside the same docstring. That is
my error, not #131's.

Still to rewrite, all in palinode/ingest/pipeline.py:

Line Current text
60 Scan ingestion inboxes iteratively routing newly seeded payloads safely.
85 Invokes explicit parsing algorithms depending heavily on disk extensions.
88 filepath (str): Evaluated system path triggering event cycles.
91–92 Emits absolute resulting saved context path safely if correctly triggered. None otherwise explicitly failing or skipping safely.
102 inline comment # extract audio natively through transcriptor boundaries
139 inline comment — trailing clause only: # Cap for very large PDFs using explicit configuration logic constraints. "Cap for very large PDFs" is accurate; drop or replace what follows
113 Extract semantic text blocks from unparsed PDF layouts formatting as markdown references.
116 filepath (str): Absolute raw OS path string matching input PDF schema formats.
154 Streams encoded chunks securely into configured remote transcriptor GPU pipelines.
157 filepath (str): Target physical file footprint mapping active media formats string sequences.
158 name (str): Native system filename.
161 `str
193 Evaluates raw strings determining if parsing redirects logic to URL processors explicitly.
196 filepath (str): Evaluated system path triggering event cycles.
197 name (str): Root file target name.
200 `str
218 Reads MacOS .webloc XML files or explicit Windows .url links cleanly fetching targets.
221–222 Standard platform shortcut reference footprint mapping system shortcut paths. / Link target OS base payload schema string sequence names.
225 Processed semantic node markdown resulting absolute DB disk array links.
245–246 Downloads HTTP response targets applying readability algorithms scrubbing out semantic fat strings locally.
264 inline comment — trailing clause only: # Simple readability: strip HTML tags cleanly securing memory. "Simple readability: strip HTML tags" is accurate; drop or replace what follows
250 name (str): Fallback generic slug text explicitly formatting the result paths correctly.
253 `str
293–294 Generates a perfectly modeled Frontmatter YAML formatted chunk safely storing contexts across DB indexes.
299 source_file (str): System path tracking originating sources schema tracking logs.
301 file_type (str): Explicit mapping array categories types string.

The last two blocks are the half-converted ones: #131 correctly rewrote url, name, content,
source_url and the Returns: line in those docstrings, so please match the register it
established rather than inventing a third style.

A useful reference point for the name parameter: ingest_pdf already documents the same thing
correctly at :117name (str): Document basename. Lines 158 and 197 are the generated variants
of that one line, which is why they are on the list rather than left as terse-but-fine.

Leave alone: the module docstring (1–10), is_safe_url (35), and the comments at 124
(# pymupdf), 230, 235, and 324–325 — all accurate as written. Only the trailing clauses of the
comments at 139 and 264 need touching; their opening halves are correct and should survive.

Scope: docstrings and those three inline comments, this file only. No behaviour changes, no new
functions, no deletions
— the diff should be docstring lines and nothing else.
palinode/indexer/watcher.py has the same problem and is filed separately; please leave it to
whoever takes that one.

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentationgood first issueGood for newcomers

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions