From aabb3b8ac94887b19b33478aa5625a6c38b27fd2 Mon Sep 17 00:00:00 2001 From: sonam-pankaj95 Date: Mon, 15 Sep 2025 23:58:23 +0200 Subject: [PATCH] textembedconfig blog --- README.md | 8 +- docs/blog/.authors.yml | 2 +- docs/blog/posts/config.md | 164 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 169 insertions(+), 5 deletions(-) create mode 100644 docs/blog/posts/config.md diff --git a/README.md b/README.md index 33363338..66585ff6 100644 --- a/README.md +++ b/README.md @@ -80,6 +80,7 @@ EmbedAnything is a minimalist, yet highly performant, modular, lightning-fast, l - **Cloud Embedding Models:**: Supports OpenAI, Cohere, and Gemini. - **MultiModality** : Works with text sources like PDFs, txt, md, Images JPG and Audio, .WAV - **GPU support** : Hardware acceleration on GPU as well. +- **Chunking** : In-built chunking methods like semantic, late-chunking - **Vector Streaming:** Separate file processing, Indexing and Inferencing on different threads, reduces latency. ## 💡What is Vector Streaming @@ -96,12 +97,11 @@ The embedding process happens separetly from the main process, so as to maintain ➡️Faster execution.
➡️No Pytorch Dependency, thus low-memory footprint and easy to deploy on cloud.
-➡️Memory Management: Rust enforces memory management simultaneously, preventing memory leaks and crashes that can plague other languages
➡️True multithreading
➡️Running embedding models locally and efficiently
-➡️Candle allows inferences on CUDA-enabled GPUs right out of the box.
-➡️Decrease the memory usage.
-➡️Supports range of models, Dense, Sparse, Late-interaction, ReRanker, ModernBert. +➡️In-built chunking methods like semantic, late-chunking
+➡️Supports range of models, Dense, Sparse, Late-interaction, ReRanker, ModernBert.
+➡️Memory Management: Rust enforces memory management simultaneously, preventing memory leaks and crashes that can plague other languages
## 🍓 Our Past Collaborations: diff --git a/docs/blog/.authors.yml b/docs/blog/.authors.yml index 77d10d14..410d4ebf 100644 --- a/docs/blog/.authors.yml +++ b/docs/blog/.authors.yml @@ -6,4 +6,4 @@ authors: sonam: name: Sonam Pankaj description: Creator of EmbedAnything - avatar: https://pbs.twimg.com/profile_images/1798985783292125184/L6YQmg1Q_400x400.jpg \ No newline at end of file + avatar: https://pbs.twimg.com/profile_images/1961327674044907520/-bgQKKlr_400x400.jpg \ No newline at end of file diff --git a/docs/blog/posts/config.md b/docs/blog/posts/config.md new file mode 100644 index 00000000..a0cc6c26 --- /dev/null +++ b/docs/blog/posts/config.md @@ -0,0 +1,164 @@ +--- +draft: false +date: 2025-09-15 +authors: + - sonam +slug: semantic-late-chunking +title: How to write textembedconfig for chunking +--- +# How to Configure TextEmbedConfig in EmbedAnything + +After presenting at Google, PyCon DE, Berlin Buzzwords, and GDG Berlin, I was surprised by how many people approached me with questions about writing configurations, chunk sizes, and batch sizes for EmbedAnything. Since I had never specifically covered this topic in my talks or blog posts, I decided to create this comprehensive guide to clarify these concepts and explain how we handle your chunking strategy with vector streaming. + +## Understanding TextEmbedConfig + +TextEmbedConfig consists of three essential components that work together to optimize your text embedding process: + +1. **The embedding model** - defines how text is converted to vectors +2. **Splitting strategy with chunk size** - determines how documents are divided +3. **Batch size** - controls vector streaming performance + +Let's explore each component in detail. + +## Setting Up the Embedding Model + +The foundation of any embedding configuration is the model itself. Here's how to initialize an embedding model: + +```python +model = EmbeddingModel.from_pretrained_hf( + WhichModel.Bert, + model_id="sentence-transformers/all-MiniLM-L12-v2" +) +``` + +This example uses a BERT-based model from Hugging Face, but you can choose from various architectures depending on your specific needs. + +## Splitting Strategies and Chunk Size + +EmbedAnything offers multiple splitting strategies, each designed for different use cases. The two primary approaches are semantic chunking and sentence-based splitting. + +### Semantic Chunking + +Semantic chunking groups similar content together based on meaning rather than arbitrary boundaries. This approach requires a semantic encoder to determine content similarity. + +First, set up your semantic encoder: + +```python +semantic_encoder = EmbeddingModel.from_pretrained_hf( + WhichModel.Jina, + model_id="jinaai/jina-embeddings-v2-small-en" +) +``` + +Then configure TextEmbedConfig with semantic chunking: + +```python +config = TextEmbedConfig( + chunk_size=1000, + batch_size=32, + splitting_strategy="semantic", + semantic_encoder=semantic_encoder +) +``` + +### Late Chunking Strategy + +Late chunking is an advanced technique that preserves contextual relationships throughout the entire document during the embedding process. The document is embedded as a whole first, then divided into chunks while maintaining rich contextual information. + +```python +config = TextEmbedConfig( + chunk_size=1000, + batch_size=8, + splitting_strategy="sentence", + late_chunking=True +) +``` + +Key benefits of late chunking: +- Maintains contextual relationships across the entire document +- Produces more meaningful embeddings for each chunk +- Particularly effective for longer documents with complex relationships + +## Understanding Key Configuration Parameters + +### Chunk Size +The `chunk_size` parameter defines the maximum number of characters (or tokens, depending on the model) allowed in each chunk. Consider these factors when setting chunk size: + +- **Smaller chunks (256-512)**: Better for precise retrieval, more granular search results +- **Larger chunks (1000-2000)**: Better for maintaining context, fewer total chunks to process +- **Model limitations**: Ensure chunk size doesn't exceed your embedding model's maximum input length + +### Batch Size for Vector Streaming +Batch size controls how many chunks the embedding model processes simultaneously. This directly impacts performance and memory usage: + +```python +# Conservative approach for limited resources +config = TextEmbedConfig(chunk_size=1000, batch_size=8, splitting_strategy="sentence") + +# Aggressive approach for high-performance systems +config = TextEmbedConfig(chunk_size=1000, batch_size=32, splitting_strategy="semantic") +``` + +**Choosing the right batch size:** +- **Smaller batches (4-8)**: Lower memory usage, more stable processing +- **Larger batches (16-32)**: Faster processing, higher memory requirements +- **Experimentation is key**: Test different batch sizes with your specific documents and hardware + +### Splitting Strategy Options + +EmbedAnything supports several splitting strategies: + +- **"semantic"**: Groups content by meaning (requires semantic encoder) +- **"sentence"**: Splits at sentence boundaries +- **Custom strategies**: Can be implemented for specialized use cases + +## Putting It All Together + +Here's a complete example showing different configuration approaches: + +```python +# Basic semantic chunking configuration +semantic_encoder = EmbeddingModel.from_pretrained_hf( + WhichModel.Jina, + model_id="jinaai/jina-embeddings-v2-small-en" +) + +config_semantic = TextEmbedConfig( + chunk_size=1000, + batch_size=16, + splitting_strategy="semantic", + semantic_encoder=semantic_encoder +) + +# Late chunking configuration for complex documents +config_late_chunking = TextEmbedConfig( + chunk_size=1500, + batch_size=8, + splitting_strategy="sentence", + late_chunking=True +) + +# Simple sentence-based chunking +config_simple = TextEmbedConfig( + chunk_size=800, + batch_size=24, + splitting_strategy="sentence" +) +``` + +## Performance Optimization Tips + +1. **Start with default values** (chunk_size=1000, batch_size=8) and adjust based on your specific use case +2. **Monitor memory usage** when increasing batch size +3. **Consider your documents' structure** when choosing splitting strategy +4. **Test retrieval quality** with different chunk sizes +5. **Profile your pipeline** to find the optimal batch size for your hardware + +## Best Practices + +- Use semantic chunking for documents where meaning preservation is crucial +- Implement late chunking for complex documents with intricate relationships +- Adjust batch size based on your available memory and processing requirements +- Regularly evaluate your configuration's impact on both performance and retrieval quality + +By understanding these configuration options and their trade-offs, you can optimize EmbedAnything for your specific use case, whether you're processing technical documentation, literary texts, or any other type of content that requires intelligent chunking and embedding. \ No newline at end of file