Skip to content

Refactor auth filtering to be more efficient#354

Merged
mbklein merged 1 commit intodeploy/stagingfrom
visibility-refactor
Mar 19, 2026
Merged

Refactor auth filtering to be more efficient#354
mbklein merged 1 commit intodeploy/stagingfrom
visibility-refactor

Conversation

@mbklein
Copy link
Copy Markdown
Contributor

@mbklein mbklein commented Mar 19, 2026

This PR refactors the way auth filters are applied to search queries. Currently, an auth filter is calculated based on what should be excluded, then the entire query (regardless of type) is wrapped in a { bool: { must: [query], must_not: [filters] } } structure. (There is a slightly different method for hybrid queries, but it just involves applying the above struct to every query in hybrid.queries[].)

This refactor calculates the auth filter based on what should be included, and then applies it depending on the query type:

  • For regular queries not covered by the following special cases, it wraps the query in bool: { must: [query], filter: [filters] }
  • For bool queries, it leaves the existing structure alone and adds the necessary filters to bool.filter
  • For neural queries, it adds the filters to neural[field].filter as a new bool filter
  • For hybrid queries, it applies the above rules to each query in hybrid.queries[].

The result is a much more efficient and focused query.

The PR also adds an optional visibility query parameter to all search routes, both GET and POST. If present, it will limit the auth filter based on the values provided, but never expand the scope beyond what the user's token allows. For example:

  • Anonymous user: ["Institution", "Public"] + Query param: visibility=private,public = { terms: { visibility: ["Public"] } }
  • Reading room user: ["Private", "Institution", "Public"] + Query param: visibility=private,institution = { terms: { visibility: ["Private", "Institution"] } }

If the main query also includes visibility constraints, the most restrictive filter will apply.

In addition to increasing efficiency, this will allow us to build simpler queries downstream (DC, MCP) and leave the heavy lifting to the API where it belongs.

Add `visibility` query param
Update OpenAPI spec
Copy link
Copy Markdown
Contributor

@bmquinn bmquinn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Searches looking good in my dev env: Image

@mbklein mbklein merged commit 64ee859 into deploy/staging Mar 19, 2026
1 check passed
@mbklein mbklein deleted the visibility-refactor branch March 19, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants