Skip to content

Comments

Add new Scout interface#1407

Draft
talister wants to merge 19 commits intoTOMToolkit:devfrom
talister:1326-new-scout-interface
Draft

Add new Scout interface#1407
talister wants to merge 19 commits intoTOMToolkit:devfrom
talister:1326-new-scout-interface

Conversation

@talister
Copy link
Collaborator

This (draft) PR creates an interface to the JPL Scout service via its API endpoint. This is implemented using the new(ish) tom_dataservices. It is substantially functional in terms of fetching, filtering and creating targets but some questions before it's mergeable:

  1. Having always struggled with Mocking and monkeypatching of requests-based functionality, it's not clear how best to test this without hitting live services
  2. I've not seen a way to get access to the request to add a message to through the Django messages infrastructure, if e.g. the user searches for a target that is removed
  3. The Targets can be created in the initial run of fetching targets but will need to be updated with new values for existing Targets if Scout reruns on the same data producing different results or the number of observations changes. I think generally Targets in the TOM Toolkit once created are relatively static and only checking on the name is done to avoid creating duplicates. I could sub-class the match_target() to match on things other than name but the NEOx experience has taught me that checking for changes in e.g. the floating point orbital elements can be tricky. Any thoughts on how to do robust updating of changing Targets?

Blank/unfilled tdes field is coming back as null string not None.
Don't ask for orbits when no querying for specific object as this isn't allowed in the Scout API
Add custom table template for Scout results.
…rvices). Change BaseDataService->DataService from upstream changes
Save a copy of input form parameters so we can do the NEO score
filtering (and other filters in future).
Check for expected signature.
Add get_additional_context_data() for a few extra things for the
template.
Update template to show total number of results and no. filtered from
(if applicable)
…certainty. Add CA distance into results table.
@talister talister added enhancement New feature or request Data Services Data Services Rubin SSSC Solar System Science Collaboration Work labels Feb 11, 2026
@jchate6 jchate6 moved this to Needs Review in TOM Toolkit Feb 11, 2026
@jchate6
Copy link
Contributor

jchate6 commented Feb 11, 2026

  1. Including some canary tests that actually touch the services is not a bad idea. These (hopefully) will fail and alert us when those services inevitably and without warning change their API. Also, please see tom_Antare:Tests (specifically factories.py and tests.py) for examples of how to mock these functions.
  2. What you can do is raise a MissingDataException with a message attached. That will end up being displayed for the user. That may be limiting, let me know if that doesn't work for you and I'll come up with another solution.
  3. This is actually represented in this Sprint (Users should have the ability to update targets via dataservices #1205) if I can ever actually have a chance to start working on sprint work. I'm hopeful that this will not be an issue in a few weeks, and I'll keep you updated so you can make sure it's behaving in a way that makes sense for you.

Copy link
Contributor

@jchate6 jchate6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint fixes.

# print(result['neoScore'] >= neo_score_min, result['phaScore'] >= pha_score_min, \
# result['geocentricScore'] < geo_score_max, ((result['rating'] is not None and \
# impact_rating_min is not None and result['rating'] >= impact_rating_min) or \
# impact_rating_min is None), (ca_dist_min is None or (ca_dist_min is not None and result['caDist'] is not None and\
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint: For readability, we restrict individual lines to 120 characters. This line is too long, and should be truncated somewhere. (or just removed if this code isn't needed.)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is currently needed due to the hard-to-test mess that is the later if- clause to end all if-clauses, as you so correctly note next... 🙃

# float(result['caDist']) <= ca_dist_min)), pos_unc >= pos_unc_min , pos_unc <= pos_unc_max)
if result['neoScore'] >= neo_score_min and result['phaScore'] >= pha_score_min and \
result['geocentricScore'] < geo_score_max and ((result['rating'] is not None and
impact_rating_min is not None and result['rating'] >= impact_rating_min) or
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint: Replace this with something like:

                if result['neoScore'] >= neo_score_min and result['phaScore'] >= pha_score_min and \
                        result['geocentricScore'] < geo_score_max and \
                        ((result['rating'] is not None and
                          impact_rating_min is not None and
                          result['rating'] >= impact_rating_min
                          ) or
                         impact_rating_min is None) and \
                        (ca_dist_min is None or
                         (ca_dist_min is not None and ca_dist is not None and ca_dist <= ca_dist_min)) and \
                        pos_unc >= pos_unc_min and pos_unc <= pos_unc_max:

This maintains the proper indentation for line overflows with all of the parentheticals, and is a little easier to read. Realistically though, this is an awful if statement and I don't really know what we are filtering on. It feels like we should pull this out into a test function where we can step through each of these logic conditions and understand them individually and in context...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I agree and an incorrect logic condition has already bitten me twice... I think the best plan is to:

  1. Apply your fixes to stop lint yelling
  2. Add some test coverage, possibly by pulling this code into a method like a is_valid_target() which (lots) of various result dictionary combos can be thrown at
  3. Potentially refactor this into a Table/QTable and iterate over the various conditions making and applying row/bitmasks

@jchate6 jchate6 added this to the FOMO milestone Feb 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Data Services Data Services enhancement New feature or request Rubin SSSC Solar System Science Collaboration Work

Projects

Status: Needs Review

Development

Successfully merging this pull request may close these issues.

2 participants