Skip to content

CSCL: Add ThinLION#2246

Draft
alexrichey wants to merge 13 commits intomainfrom
ar-thinlion
Draft

CSCL: Add ThinLION#2246
alexrichey wants to merge 13 commits intomainfrom
ar-thinlion

Conversation

@alexrichey
Copy link
Contributor

@alexrichey alexrichey commented Feb 24, 2026

Research

In 26a, these atomic polys have an interesting type
atomicid's
--2020501318
--2020501317
image

@alexrichey alexrichey changed the title Create text formatting CSV for thinlion CSCL: Add ThinLION Feb 24, 2026
TL33,police_sector,Police Sector,4,91,94,RJSF,FALSE
TL34,census_tract_2010_basic,2010 Census Tract Basic,4,95,98,RJSF,FALSE
TL34_1,census_tract_2010_suffix,2010 Census Tract Suffix,2,99,100,RJZF,TRUE
TL35,census_block_2010,2010 Census Block,4,101,104,RJZF,FALSE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that it'd be too crazy to do again, but there was some special formatting stuff around census blocks I believe in LION already, so if something along those lines is needed for THINLION definitely check out existing code

@codecov
Copy link

codecov bot commented Feb 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.82%. Comparing base (4ba9e9b) to head (0854cb6).

Additional details and impacted files

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@alexrichey alexrichey linked an issue Feb 26, 2026 that may be closed by this pull request
@alexrichey alexrichey force-pushed the ar-thinlion branch 2 times, most recently from 1877b7a to d130936 Compare February 26, 2026 18:48
- Added text_formatting__thinlion_dat.csv with 40 field specifications
- Field numbers use TL1-TL40 format from spec
- Mapped justify_and_fill from format field (RJZF default if blank)
- Set blank_if_none=TRUE for fields with 'blank if none/empty' format
- Includes position, length, and label for each field

Closes data-engineering-vwb.3
- Add 5 missing FGDB layers to recipe.yml (CensusTract2010, CensusTract2020, HealthArea, NYPDPrecinct, NYPDPatrolBorough)
- Create staging models for new layers
- Update stg__atomicpolygons with ThinLION-required fields (fire company, sanborn, water flag, etc)
- Implement thinlion_by_field.sql with census/health/NYPD joins and spatial lookups
- Create 5 borough-specific export files (Manhattan, Bronx, Brooklyn, Queens, Staten Island)
- Add _thinlion.yml with column tests

Closes de-lju.1, de-lju.2, de-lju.3, de-lju.4, de-lju.5, de-lju.6
- Create borough-specific _by_field models (manhattan, bronx, brooklyn, queens, statenisland)
- Update export models to reference borough-specific models instead of using unsupported 'where' parameter
- Add 5 new sources to sources.yml (censustract2010, censustract2020, healtharea, nypdprecinct, nypdpatrolborough)
- Update _thinlion.yml to include new borough-specific models

Fixes de-hlf
Quote field labels containing commas (Sanborn fields 'if any' labels).
This fixes 'Row 19 has 9 values, but Table only has 8 columns' error.

Fixes de-713
Add underscores to census/block column aliases to match seed field names:
- censustract_XXXX -> census_tract_XXXX
- censusblock_XXXX -> census_block_XXXX

This allows apply_text_formatting_from_seed macro to find the columns.

Fixes de-k9n
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ThinLION

2 participants