Bump unstructured from 0.10.4 to 0.11.6

[dependabot]

Bumps unstructured from 0.10.4 to 0.11.6.

Release notes

Sourced from unstructured's releases.

0.11.6

Enhancements

• Update the layout analysis script. The previous script only supported annotating final elements. The updated script also supports annotating inferred and extracted elements.
• AWS Marketplace API documentation: Added the user guide, including setting up VPC and CloudFormation, to deploy Unstructured API on AWS platform.
• Azure Marketplace API documentation: Improved the user guide to deploy Azure Marketplace API by adding references to Azure documentation.
• Integration documentation: Updated URLs for the staging_for bricks

Features

• Partition emails with base64-encoded text. Automatically handles and decodes base64 encoded text in emails with content type text/plain and text/html.
• Add Chroma destination connector Chroma database connector added to ingest CLI. Users may now use unstructured-ingest to write partitioned/embedded data to a Chroma vector database.
• Add Elasticsearch destination connector. Problem: After ingesting data from a source, users might want to move their data into a destination. Elasticsearch is a popular storage solution for various functionality such as search, or providing intermediary caches within data pipelines. Feature: Added Elasticsearch destination connector to be able to ingest documents from any supported source, embed them and write the embeddings / documents into Elasticsearch.

Fixes

• Enable --fields argument omission for elasticsearch connector Solves two bugs where removing the optional parameter --fields broke the connector due to an integer processing error and using an elasticsearch config for a destination connector resulted in a serialization issue when optional parameter --fields was not provided.

0.11.5

Enhancements

Features

Fixes

• Fix partition_pdf() and partition_image() importation issue. Reorganize pdf.py and image.py modules to be consistent with other types of document import code.

0.11.4

Enhancements

• Refactor image extraction code. The image extraction code is moved from unstructured-inference to unstructured.
• Refactor pdfminer code. The pdfminer code is moved from unstructured-inference to unstructured.
• Improve handling of auth data for fsspec connectors. Leverage an extension of the dataclass paradigm to support a sensitive annotation for fields related to auth (i.e. passwords, tokens). Refactor all fsspec connectors to use explicit access configs rather than a generic dictionary.
• Add glob support for fsspec connectors Similar to the glob support in the ingest local source connector, similar filters are now enabled on all fsspec based source connectors to limit files being partitioned.
• Define a constant for the splitter "+" used in tesseract ocr languages.

Features

• Save tables in PDF's separately as images. The "table" elements are saved as table-<pageN>-<tableN>.jpg. This filename is presented in the image_path metadata field for the Table element. The default would be to not do this.
• Add Weaviate destination connector Weaviate connector added to ingest CLI. Users may now use unstructured-ingest to write partitioned data from over 20 data sources (so far) to a Weaviate object collection.
• Sftp Source Connector. New source connector added to support downloading/partitioning files from Sftp.

Fixes

• Fix pdf hi_res partitioning failure when pdfminer fails. Implemented logic to fall back to the "inferred_layout + OCR" if pdfminer fails in the hi_res strategy.
• Fix a bug where image can be scaled too large for tesseract Adds a limit to prevent auto-scaling an image beyond the maximum size tesseract can handle for ocr layout detection
• Update partition_csv to handle different delimiters CSV files containing both non-comma delimiters and commas in the data were throwing an error in Pandas. partition_csv now identifies the correct delimiter before the file is processed.
• partition returning cid code in hi_res occasionally pdfminer can fail to decode the text in an pdf file and return cid code as text. Now when this happens the text from OCR is used.

... (truncated)

Changelog

Sourced from unstructured's changelog.

0.11.6

Enhancements

• Update the layout analysis script. The previous script only supported annotating final elements. The updated script also supports annotating inferred and extracted elements.
• AWS Marketplace API documentation: Added the user guide, including setting up VPC and CloudFormation, to deploy Unstructured API on AWS platform.
• Azure Marketplace API documentation: Improved the user guide to deploy Azure Marketplace API by adding references to Azure documentation.
• Integration documentation: Updated URLs for the staging_for bricks

Features

• Partition emails with base64-encoded text. Automatically handles and decodes base64 encoded text in emails with content type text/plain and text/html.
• Add Chroma destination connector Chroma database connector added to ingest CLI. Users may now use unstructured-ingest to write partitioned/embedded data to a Chroma vector database.
• Add Elasticsearch destination connector. Problem: After ingesting data from a source, users might want to move their data into a destination. Elasticsearch is a popular storage solution for various functionality such as search, or providing intermediary caches within data pipelines. Feature: Added Elasticsearch destination connector to be able to ingest documents from any supported source, embed them and write the embeddings / documents into Elasticsearch.

Fixes

• Enable --fields argument omission for elasticsearch connector Solves two bugs where removing the optional parameter --fields broke the connector due to an integer processing error and using an elasticsearch config for a destination connector resulted in a serialization issue when optional parameter --fields was not provided.
• Add hi_res_model_name Adds kwarg to relevant functions and add comments that model_name is to be deprecated.

0.11.5

Enhancements

Features

Fixes

• Fix partition_pdf() and partition_image() importation issue. Reorganize pdf.py and image.py modules to be consistent with other types of document import code.

0.11.4

Enhancements

• Refactor image extraction code. The image extraction code is moved from unstructured-inference to unstructured.
• Refactor pdfminer code. The pdfminer code is moved from unstructured-inference to unstructured.
• Improve handling of auth data for fsspec connectors. Leverage an extension of the dataclass paradigm to support a sensitive annotation for fields related to auth (i.e. passwords, tokens). Refactor all fsspec connectors to use explicit access configs rather than a generic dictionary.
• Add glob support for fsspec connectors Similar to the glob support in the ingest local source connector, similar filters are now enabled on all fsspec based source connectors to limit files being partitioned.
• Define a constant for the splitter "+" used in tesseract ocr languages.

Features

• Save tables in PDF's separately as images. The "table" elements are saved as table-<pageN>-<tableN>.jpg. This filename is presented in the image_path metadata field for the Table element. The default would be to not do this.
• Add Weaviate destination connector Weaviate connector added to ingest CLI. Users may now use unstructured-ingest to write partitioned data from over 20 data sources (so far) to a Weaviate object collection.
• Sftp Source Connector. New source connector added to support downloading/partitioning files from Sftp.

Fixes

• Fix pdf hi_res partitioning failure when pdfminer fails. Implemented logic to fall back to the "inferred_layout + OCR" if pdfminer fails in the hi_res strategy.
• Fix a bug where image can be scaled too large for tesseract Adds a limit to prevent auto-scaling an image beyond the maximum size tesseract can handle for ocr layout detection

... (truncated)

Commits

• 4533bda build: release commit for 0.11.6 (#2304)
• ac380ce Added AWS Marketplace docs and improved Azure Marketplace docs (#2248)
• 04f4c3a create teardown fixture for tests (#2269)
• 4ae4941 feat: support base64-encoded text in partition_email (#2277)
• 82714ca rfctr(chunking): extract BasePreChunker (#2294)
• fd293b3 feat: add elasticsearch destination connector (#2152)
• 4e2ba2c rfctr(chunking): extract boundary predicates (#2284)
• 4b8352e feat: add chroma destination connector (#2240)
• bd8a74d chore: shell scripts default indent of 2 instead of 4 (#2287)
• 096d23b Refactor: support layout analysis (#2273)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)