Allow /api/v1/project/lookup to return project with no version

[rkg-mm]

Current Behavior

Using the /api/v1/project/lookup?name=x&version=y allows looking up a project with a version.
However, if a project does have an empty version field, this API cannot be used (as version is required parameter) and there is no equivalent API to return projects without version.

Proposed Behavior

Allow the lookup API to look up projects with no version.

Consideration:
It might be misleading if only searching for name does not return all projects with that name, so docs would have to make this clear, that not providing version is only returning a project with this name and no version.

Checklist

• I have read and understand the contributing guidelines
• I have checked the existing issues for whether this enhancement was already requested

[rkg-mm]

Oh interesting - despite the API docs saying different, you can just completely omit the version parameter. If you provide it empty it fails with 404, not providing it works correctly.

[jakub-bochenski]

I'm also using the lookup endpoint without version parameter, and it works for me.

In general I've found the API documentation to be a mess.
There is just too much false information there, as seen in: #4938, #4929, #4926, #4899

I've taken to trying to understand the API by observing what the Web UI does.
This isn't how "API first" is supposed to work :(

[rkg-mm]

@nscuro do you have any activity going on in v5 to improve on API consistency and API docs?

[stohrendorf]

One more issue related to the docs: #4804

[nscuro]

The underlying issue is that API docs are generated from code, and that model classes are reused across endpoints. This gives the wrong impression of all endpoints always returning all the data, even deeply nested relationships. Which is simply impractical to do because it's horrendously expensive.

What I'd love us to do is turn the whole thing around: Manually author the openapi.yaml spec, and then generate Java interfaces and model classes from that. This would be possible with openapi-generator. There is a module that generates JAX-RS compliant interfaces.

That way the YAML becomes the source of truth, and we get much more control over what we "promise" to clients.

We don't have anyone working on this yet, simply because there is so much else to do. But I agree v5 would be a good place to finally wipe this issue clean.

[stohrendorf]

@nscuro In my own experience (both commercial and open source), the OpenAPI spec can't be used to precisely generate REST interfaces with validation. You can do it, but it will never be perfect. This is why I created my own generator in one of my projects, which generates validating interfaces for both my frontend and my backend, but it is very incomplete and lacks a lot of features that would be necessary for DT. I'm just mentioning this because you should be aware that generating interfaces from OpenAPI specs will never be complete and always carry their own risks, and it's a really challenging task. Generators like these get you 70-80% there, but it's generally just not enough.

[nscuro]

@stohrendorf That is good to know. But we shouldn't let perfect be the enemy of good IMO. What we have right now is quite frankly unsatisfactory. And fixing it in the current setup would require a lot of manual effort to ensure any kind of consistency.

Validation may not be 100% accurate, but at least clients would have more certainty as to what the API expects as input, and what they can expect as output. Is it mainly validation that you have found to be lacking with such an approach?

We're of course open to alternatives, but I think OpenAPI has emerged as de-facto standard.

[rkg-mm]

Yea I see this as problem, too, I faced this at other APIs too. Having specific models per API would be best, generated or not.
I am just not sure how big the risk of breaking API changes would be, since different APIs currently might handle the models differently than intended and people might depend on this. Maybe this should be a v2 API then?

[stohrendorf]

I bet this would introduce breaking changes, since the API currently ignores some fields it deosn't process on consumption. Some customers may provide these additional fields, which would produce an error in the future, meaning a breaking change.

[jakub-bochenski]

I think having specific models per API endpoint is the way to go.
This doesn't have to involve a lot of boilerplate. AFAIK D-T is using Jackson, so you can use Jackson @JsonView to use a single Java class for multiple endpoints (e.g. defining views like "create", "read", "update", "search")

[nscuro]

Going forward, for v2 of the REST API, we'll follow a spec-first approach: https://github.com/DependencyTrack/hyades/blob/main/docs/architecture/decisions/007-spec-first-rest-api-v2.md

The idea of using @JsonView proposed by @jakub-bochenski can still be applied to improve the accuracy of the v1 API.