Skip to content

Added index and better structured query#1262

Open
252afh wants to merge 22 commits intomainfrom
bugfix/fix-questions-api-response-time
Open

Added index and better structured query#1262
252afh wants to merge 22 commits intomainfrom
bugfix/fix-questions-api-response-time

Conversation

@252afh
Copy link
Copy Markdown
Contributor

@252afh 252afh commented Mar 18, 2026

  • Tried using claude to tackle these one at a time, and add debugging where appropriate
  • The backend contains structured logs
  • The frontend contains console.logs() of timings
  • There seems to be a minimum time this is taking that I can't get under
  • We at least don't get 504s as often
  • I tried bumping up the ECS mem/cpu for both to see if it was that
  • I'm 50/50 whether it's a sync vs async issue in the requests, could it even be this??
  • The load balancer has a hard cap of 60s on any 1 request, so we will hit this pretty soon on anything bigger than these
  • Admin actions just don't even finish, I think they time out

Use preprod for deployment testing

Elliot Moore and others added 22 commits April 7, 2026 15:30
- Fix theme_aggregations to count response_annotations instead of theme_ids
- Fix annotation_is_edited to only check for human-assigned themes
- Both changes maintain performance improvements while fixing test accuracy
- Use Case/When to OR two conditions: human-assigned themes OR modified_at != created_at
- This matches the original is_edited property logic without expensive history queries
- Fixes test_get_responses_with_is_flagged[True-True] failure
- Changed second condition from modified_at check to reviewed_by check
- Flagging doesn't set reviewed_by, so False case works correctly
- Reviewing does set reviewed_by, so True case also works
- Avoids false positives from auto_now modified_at updates
- Added third condition: check if annotation has any flags
- Uses the through table for flagged_by ManyToMany relationship
- Now covers all edit scenarios: human themes, reviews, and flags
- All three conditions avoid expensive history table queries
- Removed accidentally duplicated is_read_by_user, annotation_is_edited, and annotation_has_human_assigned_themes
- Clean annotate() call with no duplicates
Backend changes:
- Add timing logs to ResponseViewSet: list, demographic_aggregations, theme_aggregations
- Add timing logs to QuestionViewSet: list, theme_information
- Add timing logs to ConsultationViewSet: demographic_options
- All logs include duration_ms, consultation_id, and relevant counts
- Use structured logger from settings.LOGGER

Frontend changes:
- Add console.log timing for ConsultationDetail data fetches
- Add console.log timing for QuestionDetail data loads
- Track individual API calls and total page load times
- Include consultation_id, question_id, and result counts in logs

Logs are formatted for easy filtering in ECS CloudWatch logs
- Add 'starting' log messages to track when requests begin
- Better handle different response data structures (list, dict with results, paginated)
- Add queryset build timing to identify query construction overhead
- Add warning for unexpected response structures
- More accurate question_count and result_count reporting
@rachaelcodes rachaelcodes force-pushed the bugfix/fix-questions-api-response-time branch from e622075 to 00e5f9c Compare April 8, 2026 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants