make inference endpoint sync

[hunterp]

the main change here is making the get_detections http endpoint be a synchronous endpoint rather than async.

Because the get_detections method was marked as async, but contained no actual async code, when it ran it blocked the entire asyncio event loop. Meaning that the server could only handle a single request at a time effectively.

In production we run a healthcheck that ensures the api is up and responding to requests, in order to have a worker respond to those healthchecks while we also ran inference, we had to run fastapi in multi-process mode with WEB_CONCURRENCY=2

However, that change caused uvicorn to become a multi-worker supervisor: a parent process that spawns N worker subprocesses and manages their lifecycle. That supervisor path was crash-looping in production:

The logs showed:

  Started parent process [1]
  Waiting for child process [26]
  Child process [26] died      ← worker dies during bootstrap, before binding :5555
  ... (repeats hundreds of times, never reaching "Started server process")

I have built this image and am running it in production ;we're seeing about 2x the throughput already with just this change.

[favyen2]

should this be applied to all the modalities or it's only needed for sentinel-2?

[hunterp]

should this be applied to all the modalities or it's only needed for sentinel-2?

My plan is to get this fully rolled out and get us through the S2 backlog; and then apply this to the other modalities.