refactor(#148): Fix MongoDB 16MB BSON limit by separating messages collection#274
Open
anshul23102 wants to merge 1 commit into
Open
refactor(#148): Fix MongoDB 16MB BSON limit by separating messages collection#274anshul23102 wants to merge 1 commit into
anshul23102 wants to merge 1 commit into
Conversation
… fix BSON 16MB limit - Move embedded message arrays from Chat documents to Message collection - Add chat_id reference in Message model for document relationships - Create compound index on chat_id and timestamp for efficient pagination - Preserve all message fields: sender info, timestamp, reactions, metadata - Enable scalability for chats with thousands of messages - Improve query performance with targeted indexing Fixes chthonn#148
Contributor
|
@anshul23102 is attempting to deploy a commit to the Sunil Kumar's projects Team on Vercel. A member of the Team first needs to authorize it. |
Author
|
Please add labels:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem Statement
CRITICAL DATABASE LIMIT: Embedded Message Arrays Hit BSON 16MB Limit
MongoDB BSON documents have a hard 16MB size limit. Embedding all messages as arrays in Chat documents means any chat with more than ~1000 messages will fail - guaranteed production outage.
Vulnerability Details
Current approach: Store messages as embedded array in Chat document
Problem: Message array growth → Document size → 16MB limit exceeded → Database errors
Impact:
Solution Implemented
1. Message Collection
Created dedicated Message model with proper schema:
2. Database Schema
Message structure:
3. Scalability
Files Changed
Benefits
Fixes #148