Skip to content

Resource leak: gRPC channels and HTTP clients not closed in batch uploaders #1172

@Shiven0504

Description

@Shiven0504

What's happening

When using upload_collection() or upload_points() with parallel > 1, each worker creates its own gRPC channel or HTTP client — but neither is ever closed after the upload finishes.

gRPC uploader

In qdrant_client/uploader/grpc_uploader.py, process_upload() calls get_channel() to create a new gRPC channel, but there's no corresponding channel.close():

def process_upload(self, items):
    channel = get_channel(host=self._host, port=self._port, ...)  # opened here
    points_client = grpc.PointsStub(channel)
    for batch in items:
        yield upload_batch_grpc(...)
    # channel is never closed

REST uploader

In qdrant_client/uploader/rest_uploader.py, __init__() creates a SyncApis client that's also never cleaned up:

class RestBatchUploader(BaseUploader):
    def __init__(self, ...):
        self.openapi_client: SyncApis = SyncApis(host=uri, **kwargs)
        # no close() or __del__ anywhere

Impact

Each parallel worker leaks a connection. If you're doing repeated uploads with parallel=4, that's 4 orphaned connections per call that pile up over time. In long-running services this leads to file descriptor exhaustion and connection pool issues.

Steps to reproduce

from qdrant_client import QdrantClient
from qdrant_client.models import PointStruct, VectorParams, Distance

client = QdrantClient("http://localhost:6333")

client.create_collection("test", vectors_config=VectorParams(size=4, distance=Distance.COSINE))

points = [PointStruct(id=i, vector=[0.1]*4, payload={"x": i}) for i in range(1000)]

# Each call leaks gRPC/HTTP connections from worker processes
for _ in range(100):
    client.upload_points("test", points, parallel=4)

Suggested fix

  • GrpcBatchUploader.process_upload(): close the channel after the upload loop (e.g. in a finally block)
  • RestBatchUploader: add a cleanup method and call self.openapi_client.close() when the worker is done

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions