Skip to content

[codex] Harden place extractor fixtures and preview enrichment #48

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
michaelmwu merged 1 commit into from
Jul 5, 2026
Merged

[codex] Harden place extractor fixtures and preview enrichment #48

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
896 changes: 896 additions & 0 deletions src/gmaps_scraper/data/place_extractor.js
View file
Open in desktop

Large diffs are not rendered by default.

836 changes: 80 additions & 756 deletions src/gmaps_scraper/place_scraper.py
View file
Open in desktop

Large diffs are not rendered by default.

41 changes: 41 additions & 0 deletions tests/fixtures/place_pages/about.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Fixture About</title>
<style>
body { margin: 0; font-family: Arial, sans-serif; }
[role="main"] { box-sizing: border-box; width: 720px; min-height: 980px; padding: 24px; }
h1 { margin: 0 0 12px; font-size: 32px; }
.F7nice, [data-item-id], section { margin-top: 12px; }
[data-item-id] { display: block; min-height: 32px; }
</style>
</head>
<body>
<main role="main" aria-label="Fixture About Bakery">
<h1 class="DUwDvf">Fixture About Bakery</h1>
<div class="F7nice">
<span><span aria-hidden="true">4.3</span></span>
<span aria-label="678 reviews">(678)</span>
</div>
<button jsaction="category.click">Bakery</button>
<button data-item-id="address"><span class="Io6YTe">9 About Avenue, Fixture City</span></button>
<a data-item-id="authority" href="https://about.fixture.example/">Website</a>
<button data-item-id="phone:tel:+15550102"><span class="Io6YTe">+1 555-0102</span></button>
<div role="tablist">
<button role="tab">Overview</button>
<button role="tab">Reviews</button>
<button role="tab">About</button>
</div>
<div aria-label="About Fixture About Bakery">
<section class="iP2t7d">
<h2>Service options</h2>
<ul>
<li><span aria-label="Offers takeaway">Takeaway</span></li>
<li><span aria-label="Offers delivery">Delivery</span></li>
</ul>
</section>
</div>
</main>
</body>
</html>
18 changes: 18 additions & 0 deletions tests/fixtures/place_pages/limited_view.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Limited View Fixture</title>
<style>
body { margin: 0; font-family: Arial, sans-serif; }
[role="main"] { box-sizing: border-box; width: 720px; min-height: 520px; padding: 24px; }
h1 { margin: 0 0 12px; font-size: 32px; }
</style>
</head>
<body>
<main role="main" aria-label="Limited Fixture Place">
<h1 class="DUwDvf">Limited Fixture Place</h1>
<p>This is a limited view of Google Maps.</p>
</main>
</body>
</html>
41 changes: 41 additions & 0 deletions tests/fixtures/place_pages/overview.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta property="og:image" content="https://example.test/fallback-og.jpg">
<title>Fixture Place</title>
<style>
body { margin: 0; font-family: Arial, sans-serif; }
[role="main"] { box-sizing: border-box; width: 720px; min-height: 900px; padding: 24px; }
h1 { margin: 0 0 12px; font-size: 32px; }
.F7nice, .WeS02d, [data-item-id], .tabs { margin-top: 12px; }
[data-item-id] { display: block; min-height: 32px; }
button, a { font: inherit; }
</style>
</head>
<body>
<main role="main" aria-label="Fixture Cafe">
<div class="RZ66Rb">
<button jsaction="heroHeaderImage.open">
<img src="https://lh3.googleusercontent.com/place-overview.jpg" alt="Fixture Cafe">
</button>
</div>
<h1 class="DUwDvf">Fixture Cafe</h1>
<div class="F7nice" aria-label="4.6 stars, 1,234 reviews">
<span><span aria-hidden="true">4.6</span></span>
<span aria-label="1,234 reviews">(1,234)</span>
</div>
<button jsaction="category.click">Coffee shop</button>
<div class="WeS02d">Compact fixture description for the place overview.</div>
<button data-item-id="address"><span class="Io6YTe">123 Test Street, Fixture City</span></button>
<a data-item-id="authority" href="https://fixture.example/">Website</a>
<button data-item-id="phone:tel:+15550100"><span class="Io6YTe">+1 555-0100</span></button>
<button data-item-id="oloc"><span class="Io6YTe">7FG8+22 Fixture City</span></button>
<div class="tabs" role="tablist">
<button role="tab">Overview</button>
<button role="tab">Reviews</button>
<button role="tab">About</button>
</div>
</main>
</body>
</html>
51 changes: 51 additions & 0 deletions tests/fixtures/place_pages/reviews.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Fixture Reviews</title>
<style>
body { margin: 0; font-family: Arial, sans-serif; }
[role="main"] { box-sizing: border-box; width: 720px; min-height: 980px; padding: 24px; }
h1 { margin: 0 0 12px; font-size: 32px; }
.F7nice, [data-item-id], [data-review-id], .topic-row { margin-top: 12px; }
[data-item-id] { display: block; min-height: 32px; }
[data-review-id] { border-top: 1px solid #ddd; padding-top: 12px; }
</style>
</head>
<body>
<main role="main" aria-label="Fixture Reviews Cafe">
<div class="RZ66Rb">
<button jsaction="heroHeaderImage.open">
<img src="https://lh3.googleusercontent.com/place-review-hero.jpg" alt="">
</button>
</div>
<h1 class="DUwDvf">Fixture Reviews Cafe</h1>
<div class="F7nice">
<span><span aria-hidden="true">4.8</span></span>
<span aria-label="2,345 reviews">(2,345)</span>
</div>
<button jsaction="category.click">Restaurant</button>
<button data-item-id="address"><span class="Io6YTe">55 Review Road, Fixture City</span></button>
<a data-item-id="authority" href="https://reviews.fixture.example/">Website</a>
<button data-item-id="phone:tel:+15550101"><span class="Io6YTe">+1 555-0101</span></button>
<div role="tablist">
<button role="tab">Overview</button>
<button role="tab">Reviews</button>
<button role="tab">About</button>
</div>
<div class="topic-row">
<button role="radio" aria-label="coffee, mentioned in 19 reviews">coffee 19</button>
<button role="radio" aria-label="service, mentioned in 12 reviews">service 12</button>
</div>
<article data-review-id="review-placeholder-1">
<button jsaction="image.open">
<img src="https://lh3.googleusercontent.com/reviewer-placeholder.jpg" alt="Reviewer avatar">
</button>
<div class="d4r55">Reviewer Placeholder</div>
<span role="img" aria-label="5 stars"></span>
<span class="rsqaWe">2 months ago</span>
<div class="MyEned"><span class="wiI7pd">Useful visible review snippet.</span></div>
</article>
</main>
</body>
</html>
32 changes: 32 additions & 0 deletions tests/fixtures/place_pages/search_result.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Search Result Fixture</title>
<style>
body { margin: 0; font-family: Arial, sans-serif; }
[role="main"] { box-sizing: border-box; width: 840px; min-height: 700px; padding: 24px; }
[role="article"] { width: 420px; min-height: 160px; margin-top: 16px; padding: 12px; }
a { display: block; font-size: 20px; }
</style>
</head>
<body>
<main role="main" aria-label="Results for Fixture Search">
<h1 class="DUwDvf">Results</h1>
<div role="feed">
<div role="article">
<a class="hfpxzc" aria-label="Fixture Search Cafe"
href="https://www.google.com/maps/place/Fixture+Search+Cafe/data=!4m2!3m1!1sfixture-place-id">
Fixture Search Cafe
</a>
<div class="W4Efsd">
<span class="AJB7ye" role="img" aria-label="4.7 stars 321 reviews"></span>
4.7 (321)
</div>
<div class="W4Efsd">Cafe · 10 Search Street</div>
<div class="W4Efsd">Small local search-result description.</div>
</div>
</div>
</main>
</body>
</html>
32 changes: 32 additions & 0 deletions tests/fixtures/preview_payloads/ambiguous_rating_review_count.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
)]}'
[
[
"price-summary",
4.8,
101,
"NT$101"
],
[
"history-summary",
4.6,
2024,
"Founded 2024"
],
[
"non-adjacent-summary",
4.7,
null,
null,
912
],
[
"conflicting-summary-a",
4.5,
120
],
[
"conflicting-summary-b",
4.6,
130
]
]
17 changes: 17 additions & 0 deletions tests/fixtures/preview_payloads/rating_review_count.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
)]}'
[
[
"preview-place",
[
"rating-summary",
4.7,
832
],
[
null,
null,
25.2048,
55.2708
]
]
]
Loading