[bug] Gitea sync fails with ERR_STREAM_PREMATURE_CLOSE, then crashes on null full_name

[HueskenNiels-hue]

Describe the bug

Bug description

When syncing repositories from a self-hosted Gitea instance, Sourcebot fails the connection sync job with:

Cannot read properties of null (reading 'full_name')

After debugging, the root cause appears to be that gitea-js / cross-fetch fails while reading the Gitea API response body with:

ERR_STREAM_PREMATURE_CLOSE

This causes api.repos.repoGet() to return:

{
  "size": 0,
  "timeout": 0,
  "data": null,
  "error": {
    "message": "Invalid response body while trying to fetch http://xxxxx:3001/api/v1/repos/xxxx/xxx.xxxx: Premature close",
    "type": "system",
    "errno": "ERR_STREAM_PREMATURE_CLOSE",
    "code": "ERR_STREAM_PREMATURE_CLOSE"
  }
}

Sourcebot then executes this code in /app/packages/backend/dist/gitea.js:

return {
    type: 'valid',
    data: [response.data]
};

Since response.data is null, allRepos contains null. Later this crashes:

allRepos = allRepos.filter(repo => repo.full_name !== undefined);

To reproduce

Inside the Sourcebot container:

import { giteaApi } from "gitea-js";
import fetch from "cross-fetch";

const api = giteaApi("http://gitea:3001", {
  token: process.env.GITEA_TOKEN,
  customFetch: fetch,
});

const response = await api.repos.repoGet("Org", "Repo");

console.log(JSON.stringify(response, null, 2));

{
  "size": 0,
  "timeout": 0,
  "data": null,
  "error": {
    "message": "Invalid response body while trying to fetch http://gitea:3001/api/v1/repos/Fairparken/Fairparken.Worker: Premature close",
    "type": "system",
    "errno": "ERR_STREAM_PREMATURE_CLOSE",
    "code": "ERR_STREAM_PREMATURE_CLOSE"
  }
}

Sourcebot deployment information

Sourcebot version (e.g. v3.0.1): V5.0.4
Gittea image version: docker.gitea.com/gitea:1.26.1-rootless

Additional information

Temporary workaround tested:

I patched /app/packages/backend/dist/gitea.js inside the running Sourcebot container to use a custom fetch:

const customFetch = (url, options = {}) => {
    const headers = {
        ...(options.headers ?? {}),
        'Accept-Encoding': 'identity',
        'Connection': 'close',
    };

    return fetch(url, {
        ...options,
        headers,
    });
};

const api = giteaApi(config.url ?? 'https://gitea.com', {
    token: token,
    customFetch,
});

I also added a null guard:

allRepos = allRepos.filter((repo, index) => {
    if (repo === null || repo === undefined) {
        logger.warn(`Skipping null/undefined Gitea repo at index ${index}`);
        return false;
    }
    return repo.full_name !== undefined;
});

Relevant code locations

/app/packages/backend/dist/gitea.js:42
/app/packages/backend/dist/gitea.js:161-166
/app/packages/backend/dist/repoCompileUtils

[rachit367]

Picking this up. Root cause confirmed in packages/backend/src/gitea.ts: when repoGet fails mid-body (ERR_STREAM_PREMATURE_CLOSE) it resolves with { data: null, error } rather than throwing, and getRepos wraps that as data: [response.data] — pushing null into allRepos. The top-level allRepos.filter(repo => repo.full_name !== undefined) then dereferences null.full_name and crashes the whole sync. Fixing by treating a null response body as a per-repo warning (like the 404 path) so one failed fetch is skipped instead of failing the connection, and hardening the filter against null. Not attempting the underlying stream/Accept-Encoding workaround here — that's a separate networking concern.

[HueskenNiels-hue]

Picking this up. Root cause confirmed in packages/backend/src/gitea.ts: when repoGet fails mid-body (ERR_STREAM_PREMATURE_CLOSE) it resolves with { data: null, error } rather than throwing, and getRepos wraps that as data: [response.data] — pushing null into allRepos. The top-level allRepos.filter(repo => repo.full_name !== undefined) then dereferences null.full_name and crashes the whole sync. Fixing by treating a null response body as a per-repo warning (like the 404 path) so one failed fetch is skipped instead of failing the connection, and hardening the filter against null. Not attempting the underlying stream/Accept-Encoding workaround here — that's a separate networking concern.

Hi Rachit,

Thank you for taking care of this problem. Please note, that the main issue is not a networking problem. The issue is related to the way the gittea client requests data from the server. The gittea client can't handle chunked server responses. You can reproduce the actual problem by running the gitea-server image I meantioned.

My suggested fix is to ask the server for a non-chuncked response by sending the ""Connection": "close"" - Header, like I meantioned in my patch.

Kind Regards,
Niels

[brendan-kellam]

@HueskenNiels-hue thanks for raising. I shipped a change in #1405. Can you let me know if this fixes your issue? thanks

[HueskenNiels-hue]

@HueskenNiels-hue thanks for raising. I shipped a change in #1405. Can you let me know if this fixes your issue? thanks

@brendan-kellam Thanks for the quick response.

I checked against

ghcr.io/sourcebot-dev/sourcebot                 main              sha256:10ff005727b8d9cbb7368cae88de9fd4b3b682970efbdbc90e1272f44a795ec7   10ff005727b8   4 hours ago    2.85GB

Sync works flawless now. Thanks!