Generating tests with HuggingFace leads to 404

[PhilippStraubinger]

Affected Modules

• UI
• EvoSuite
• LLM
• Kex
• Other (please specify)

Bug Description

Attempting to generate tests using a HuggingFace token (with either of the two supported models) results in a 404 error.

Steps to Reproduce

1. Open any Java class
2. Right-click a method and select TestSpark
3. Choose HuggingFace and select one of the two Ollama models
4. Click Generate → an IntelliJ error notification appears with a 404 error (no additional details provided)

Expected Behavior

The test generation process completes successfully.

Additional Information

According to the API reference of HuggingFace, the issue could be a missing api/ in the URL

[vsantele]

According to the API reference you mentioned, the Hub API doesn't provide an endpoint to make request to a model.

It seems that a lot of users are getting the same 404 errors. (1), (2)

Since January 2025, there has been a new API for calling LLMs using HF: https://huggingface.co/blog/inference-providers

There are now several providers for calling LLMs via HF.
The python SDK allows us to choose provider=auto (docs). The SDK makes a call to the API to select the provider (here and here). Then the api call seems to be the same for all providers and compatible with the openai spec using the url router.huggingface.co/{provider}/models/{providerModelId}/v1/chat/completions.

I guess TestSpark could support more models from HF using this new API while fixing the issue?