From 3bf5570961504eca1f86742edf5847e94bb8adec Mon Sep 17 00:00:00 2001 From: sonam-pankaj95 Date: Tue, 16 Sep 2025 09:52:58 +0200 Subject: [PATCH 1/2] rectified --- README.md | 4 +++- docs/blog/posts/config.md | 4 ++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 66585ff6..f2823832 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,9 @@

- Highly Performant and Modular - Inference, Ingestion, and Indexing in Rust 🦀 + Highly Performant, Modular and Memory Safe +
+ Ingestion, Inference and Indexing in Rust 🦀
Python docs »
diff --git a/docs/blog/posts/config.md b/docs/blog/posts/config.md index a0cc6c26..4775009f 100644 --- a/docs/blog/posts/config.md +++ b/docs/blog/posts/config.md @@ -84,8 +84,8 @@ Key benefits of late chunking: ### Chunk Size The `chunk_size` parameter defines the maximum number of characters (or tokens, depending on the model) allowed in each chunk. Consider these factors when setting chunk size: -- **Smaller chunks (256-512)**: Better for precise retrieval, more granular search results -- **Larger chunks (1000-2000)**: Better for maintaining context, fewer total chunks to process +- **Smaller chunks**: Better for precise retrieval, more granular search results +- **Larger chunks**: Better for maintaining context, fewer total chunks to process - **Model limitations**: Ensure chunk size doesn't exceed your embedding model's maximum input length ### Batch Size for Vector Streaming From a8191f4906c3b87bb7c8f02c476ef726f18bdb73 Mon Sep 17 00:00:00 2001 From: sonam-pankaj95 Date: Tue, 16 Sep 2025 10:00:06 +0200 Subject: [PATCH 2/2] red --- docs/blog/posts/config.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/blog/posts/config.md b/docs/blog/posts/config.md index 4775009f..fc0284d3 100644 --- a/docs/blog/posts/config.md +++ b/docs/blog/posts/config.md @@ -10,6 +10,9 @@ title: How to write textembedconfig for chunking After presenting at Google, PyCon DE, Berlin Buzzwords, and GDG Berlin, I was surprised by how many people approached me with questions about writing configurations, chunk sizes, and batch sizes for EmbedAnything. Since I had never specifically covered this topic in my talks or blog posts, I decided to create this comprehensive guide to clarify these concepts and explain how we handle your chunking strategy with vector streaming. + + + ## Understanding TextEmbedConfig TextEmbedConfig consists of three essential components that work together to optimize your text embedding process: