fix(create,update): standardize duplicate key error handling across endpoints#64
fix(create,update): standardize duplicate key error handling across endpoints#64Bodeayman wants to merge 1 commit into
Conversation
- Added [handle_duplicate_error](cci:1://file:///c:/Myfiles/OSGeo/istSOS4/api/app/utils/utils.py:274:0-294:5) utility to reliably catch `UniqueViolationError`. - Implemented regex parsing of database error details to provide developer-friendly 409 Conflict responses including specific conflicting columns and values. - Integrated exception handling across all CREATE and UPDATE endpoints for: Thing, Sensor, Location, ObservedProperty, FeatureOfInterest, and Network. - Prevented internal schema/constraint leakage by sanitizing database error messages.
| from urllib.parse import parse_qs, quote, urlencode, urlparse, urlunparse | ||
|
|
||
| from app import EPSG, HOSTNAME, TOP_VALUE | ||
| from asyncpg.exceptions import UniqueViolationError |
There was a problem hiding this comment.
why do we need to import 'UniqueViolationError' if we are not using it
| if match: | ||
| column = match.group(1) | ||
| value = match.group(2) | ||
| message = f"An entity with {column} '{value}' already exists." |
There was a problem hiding this comment.
Im a bit unsure about parsing e.detail and returning the extracted column/value to the client. even if we no longer expose the constraint name, we are still deriving the response from internal database error details. would it be safer to return a fixed application-level message like with this name already exists. and keep the detailed exception only for logs?
| from app import AUTHORIZATION, POSTGRES_PORT_WRITE, VERSIONING | ||
| from app.db.asyncpg_db import get_pool, get_pool_w | ||
| from app.utils.utils import validate_payload_keys | ||
| from app.utils.utils import handle_duplicate_error, validate_payload_keys |
There was a problem hiding this comment.
could you import from asyncpg.exceptions import InsufficientPrivilegeError, UniqueViolationError as it was used but never import.
| "code": 400, | ||
| "type": "error", | ||
| "message": str(e), | ||
| "message": f"[{type(e).__name__}] {str(e)}" |
There was a problem hiding this comment.
I don’t think we should include type(e).name or raw str(e) in the client response here. That still exposes internal implementation details and can leak database/backend information. For this case, it would be safer to return a sanitized conflict message and keep the original exception only in server-side logs
|
if this PR only checks for duplicates before the DB write, i think it is still incomplete for concurrent requests. two requests can pass the pre check at the same time and one will still hit @ClaudioPrimerano @danistrigaro @massimiliano-cannata, what do you guys think about the concurrent request problem? |
Related Issue
Closes #63
Problem
When creating or updating an entity with a duplicate unique key (such as
name),the API bubbled up a raw
UniqueViolationErrordirectly to the client, resultingin
400 Bad Requestresponses that leaked internal database details includingtable names, constraint names, and row data from the
DETAILfield.Solution
Introduced a centralized error handling mechanism for uniqueness violations:
utils.handle_duplicate_error— a new utility function that catchesUniqueViolationErrorexceptions from asyncpgand value from the database error, returning a clean message:
"An entity with [column] '[value]' already exists."409 Conflictinstead of generic
400 Bad RequestChanges Made
api/app/utils/utils.py— addedhandle_duplicate_errorwith regex parsingapi/app/v1/endpoints/create/— Thing, Sensor, Location, ObservedProperty,FeaturesOfInterest, Network
api/app/v1/endpoints/update/— Thing, Sensor, Location, ObservedProperty,FeaturesOfInterest, Network
Verification
400+ raw DB error409+ clean message400+ constraint details409+ clean messageBefore
{ "code": 400, "type": "error", "message": "duplicate key value violates unique constraint \"unique_location_name\"\nDETAIL: Key (name)=(location name 1) already exists." }After
{ "code": 409, "type": "error", "message": "An entity with name 'location name 1' already exists." }