Skip to content

feat(band): add Band.us adapter — bands, posts, mentions, post commands#532

Open
kanghouchao wants to merge 16 commits intojackwener:mainfrom
kanghouchao:feat/band
Open

feat(band): add Band.us adapter — bands, posts, mentions, post commands#532
kanghouchao wants to merge 16 commits intojackwener:mainfrom
kanghouchao:feat/band

Conversation

@kanghouchao
Copy link
Copy Markdown

Summary

Adds a Band.us (band.us) adapter with 4 commands:

  • band bands — list all Bands you belong to
  • band posts <band_no> — list posts from a Band (with --limit)
  • band mentions — show notifications where you were @mentioned (supports --filter, --limit, --unread)
  • band post <band_no> <post_no> — export full post content including nested comments, with optional --output <dir> for photo download and --comments false to skip comment fetch

Implementation notes

Band.us signs every API request with a per-request HMAC (md header) generated by its own JavaScript — the header cannot be replicated externally. The initial implementation used Strategy.INTERCEPT (XHR interception) with an indirect home→SPA-navigate flow to capture the API responses.

After further investigation, all required data is available in the DOM for logged-in users. The final implementation uses Strategy.COOKIE with navigateBefore: false and navigates directly to the target URL:

  • bands: framework pre-nav to band.us home, extract sidebar band list from DOM
  • posts: direct goto to /band/{band_no}/post, extract post list from DOM
  • post: direct goto to /band/{band_no}/post/{post_no}, extract post body + nested replies from DOM

Comment hierarchy is handled by recursing into .sReplyList .sCommentList — replies appear as type: reply with a prefix in table output.

Test plan

  • opencli band bands — lists your bands (requires Band login)
  • opencli band posts <band_no> --limit 5 — lists posts
  • opencli band mentions — shows @mention notifications
  • opencli band post <band_no> <post_no> — exports post with comments
  • opencli band post <band_no> <post_no> --comments false — skips comment fetch
  • opencli band post <band_no> <post_no> --output /tmp/photos — downloads photos
  • Auth failure tests pass: npx vitest run tests/e2e/browser-auth.test.ts

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings March 28, 2026 02:51
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new band.us adapter to opencli, providing Band membership and content export commands using browser automation and DOM/XHR extraction.

Changes:

  • Introduces band bands, band posts, band post, and band mentions commands under src/clis/band/.
  • Implements DOM-based extraction for bands/posts/post, and XHR interception for mentions.
  • Extends E2E “graceful auth failure” coverage to include the new Band commands.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/e2e/browser-auth.test.ts Adds auth-required graceful-failure tests for the new band commands.
src/clis/band/bands.ts New command to list bands from the Band.us home sidebar DOM.
src/clis/band/posts.ts New command to list posts for a band via direct navigation + DOM parsing.
src/clis/band/post.ts New command to export a full post (plus optional photo download) via DOM parsing.
src/clis/band/mentions.ts New command to fetch @mention notifications via UI actions + XHR interception.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@kanghouchao
Copy link
Copy Markdown
Author

All 6 review comments have been addressed in d269b5e:

  • Removed redundant ArgumentError guards in posts.ts and post.ts (framework validates required: true args before func() is called)
  • Fixed photo URL resolution to use new URL(src, location.href) for correct handling of relative/protocol-relative URLs
  • Replaced manual http(s).get download with shared downloadMedia utility (handles redirects, timeouts, stream errors); all bare Node built-in imports removed as a result
  • band mentions keeps Strategy.INTERCEPT intentionally — notification data is not in the DOM, replied inline to explain

Replied to each comment individually.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

kanghouchao and others added 8 commits March 28, 2026 13:47
- bands: lists all Bands via get_band_list_with_filter intercept
- posts: lists posts from a Band via get_posts_and_announcements intercept
- mentions: shows @mention notifications via get_news intercept

All use Strategy.INTERCEPT since band.us API requires an HMAC md header
generated by its own JS. SPA navigation to /band/{no}/post triggers the
band list and posts APIs; bell + @メンション tab click triggers mentions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix doc comments: Band uses XHR not fetch; clarify INTERCEPT rationale
- bands: replace for-loop with flatMap; explain why band page nav is needed
- posts: remove item.post ?? item fallback (API always wraps in post); rename
  finalRequests → requests for consistency; extract stripBandTags helper
- mentions: remove redundant ?? defaults (args have defaults defined); fix
  unreadOnly bug (was not applied to post/comment modes); consolidate Band tag
  stripping to single regex; cast kwargs types directly instead of converting;
  add comments explaining last-response strategy and 'referred' filter flag

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
get_posts_and_announcements returns both regular posts and announcements
that have different shapes — some lack post_no and wrap differently.
Restore item.post ?? item fallback and filter out items with no resolvable
identifier to prevent undefined in URLs and empty rows in output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…to download

Exports the complete content of a single Band post:
- Post body (with Band markup tags stripped)
- All comments in chronological order
- Photo URLs shown inline, or downloaded with --output <dir>

Uses Strategy.INTERCEPT with a broad 'band.us' pattern to capture both the
batch request (embedding get_post) and get_comments in one SPA navigation.
Responses are identified client-side by shape: batch_result array vs items
array with comment_id fields.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- bands, posts, post: navigate directly to target URL instead of home→SPA detour
- All three switch from Strategy.INTERCEPT to Strategy.COOKIE with navigateBefore: false
  (bands uses framework pre-nav to home; posts/post disable it and goto target directly)
- DOM extraction polls for specific content elements rather than fixed waits
- post: confirm selectors via browser inspection (a.text, time.time, .sCommentList,
  .sReplyList for nested replies); add --comments flag to skip comment fetch
- posts: extract from rendered post list DOM; correct comment item selector (div.cComment)
- Fix: post empty-result guard changed from && to handle null data safely
- Fix: photo download now checks HTTP status code before piping to avoid writing
  redirect HTML into image files
- Fix: mentions unread client-side filter skipped for 'mentioned' mode since
  server already filtered via 未確認のみ button click

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- post: replace manual http/https download with shared downloadMedia utility
  (handles redirects, timeouts, stream errors correctly)
- post: fix photo URL resolution to use location.href as base, handling
  protocol-relative and relative URLs without throwing
- post: switch to node:-prefixed imports per repo convention
- post/posts: remove redundant ArgumentError guards — framework already
  validates required args before func() is called
- mentions: INTERCEPT strategy is intentional (Band HMAC prevents DOM-only
  approach for notifications; update PR description to clarify)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- bands: tighten href selector to /band/{id}(?:/post)?$ so feed/post-detail
  links are excluded; only sidebar navigation links match
- mentions: replace fixed page.wait(2) sleeps with polling on
  getInterceptedRequests() — waits up to 8 s per action, exits as soon
  as the expected number of captures arrives (avoids flakiness on slow XHR)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- bands: use a.bandCover._link + p.uriText + span.member em selectors
  (previous a[href*="/band/"] + .bandName combo leaked "メンバー" text)
- posts: use article.cContentsCard._postMainWrap + span.count selectors
  (previous li._postListItem selector matched nothing; DOM changed)
- mentions: fix page.wait(500) → page.wait(0.5) (was waiting 500s not ms);
  use timestamp-suffixed URL to force fresh page load each run so the
  notification panel is closed; fix get_news vs get_news_count capture
  ambiguity with result_data.news check; replace cumulative waitForCaptures
  with waitForOneCapture (getInterceptedRequests clears array on each call)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@kanghouchao
Copy link
Copy Markdown
Author

Following up on the two issues flagged in the second review round — both are resolved, but the actual fixes landed differently than my earlier inline replies described (the code was further improved after live testing):

1. bands.ts — selector scoped to the sidebar (earlier reply said: regex tightened)

After testing, the regex approach still matched non-sidebar links. The final fix uses a.bandCover._link — the sidebar band card's actual CSS class — as the primary selector, so the query is inherently sidebar-scoped without relying on URL pattern matching. Band name and member count are extracted from p.uriText and span.member em inside each card.

2. mentions.ts — eliminate flaky fixed waits (earlier reply said: replaced with waitForCaptures(n) polling)

The waitForCaptures(n) approach had a subtle bug: getInterceptedRequests() clears the captured array on each call, so a cumulative count target of 2+ was unreachable. The final fix uses waitForOneCapture(), which polls every 0.5 s and returns as soon as 1 new capture arrives (per UI action), up to 8 s. Additionally, navigating with ?_=${Date.now()} forces a fresh page load so a stale notification panel from a previous run doesn't suppress the XHR trigger.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

kanghouchao and others added 2 commits March 28, 2026 14:06
… locale-dependent text match

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- post: pass browser cookies to downloadMedia so Band's login-protected
  photo URLs don't fail with 401/403
- post: include photos.length in empty-result guard so photo-only posts
  are not falsely reported as not found
- mentions: accumulate captures across poll iterations so get_news_count
  responses don't cause early exit before the real get_news arrives
- mentions: update docstring to match actual implementation (client-side
  filtering, no tab-click)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@kanghouchao
Copy link
Copy Markdown
Author

All 4 issues from the third review round have been addressed in 071b647:

  • post.ts — cookies not passed to downloadMedia: Fixed — page.getCookies({ domain: 'band.us' }) is now called before the download, and the resulting cookie string is passed as cookies to downloadMedia() so Band's login-protected photo URLs don't fail with 401/403.

  • mentions.tswaitForOneCapture exits early on get_news_count: Fixed — the helper now accumulates captures across poll iterations and keeps polling until a capture containing result_data.news is found (or the 8 s limit is reached). A get_news_count response arriving first no longer causes an early exit.

  • mentions.ts — docstring describes tab-click flow that isn't implemented: Fixed — updated the docstring to match the actual behavior: click the bell to trigger the get_news XHR, then apply client-side filtering.

  • post.ts — empty-result guard ignores photos: Fixed — the guard is now !data?.text && !data?.comments?.length && !data?.photos?.length, so photo-only posts are no longer incorrectly treated as not found.

@kanghouchao
Copy link
Copy Markdown
Author

@copilot review

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- mentions: fail fast with a clear error when bell button is not found,
  instead of silently no-op and waiting 8s before EmptyResultError
- post: use shared formatCookieHeader() instead of manual cookie string
  construction

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- mentions: replace fixed page.wait(2) with polling for bell button
  readiness (up to 10s), eliminating the fixed sleep and fail-fast
  when the selector is missing
- mentions: add explicit !newsReq guard with a clear error message when
  get_news capture times out, instead of falling through to a misleading
  "No notifications found"
- posts: skip posts with no permalink href instead of emitting a bogus
  'https://www.band.us' URL

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- post: only send Band cookies to *.band.us photo URLs; third-party CDN
  URLs are downloaded without cookies to avoid cross-domain cookie leakage
- bands: strip non-digit chars before parseInt so member counts like
  "1,234" parse correctly
- posts: same fix for comment counts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- posts: check limit before push so --limit 0 returns empty result
- post: indent replies proportionally by depth ('  '.repeat(depth))
  so multi-level threads remain readable in table output

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Pattern now requires /band/{id} or /band/{id}/post (with optional trailing
slash) so deeper paths like /band/{id}/post/{postNo} are excluded.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- mentions: guard bell click with a boolean return so a disappearing
  element throws a clear EmptyResultError instead of a raw TypeError
- post: wait for comment list container instead of first .cComment so
  posts with zero comments don't incur a fixed 6s delay

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

columns: ['band_no', 'name', 'members'],

func: async (page, _kwargs) => {
const isLoggedIn = await page.evaluate(`() => document.cookie.includes('band_session')`);
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Login detection uses document.cookie.includes('band_session'), which won’t see HttpOnly cookies. If Band’s session cookie is HttpOnly (common for session cookies), this will incorrectly throw AuthRequiredError even when the user is logged in. Prefer checking await page.getCookies({ domain: 'band.us' }) for a cookie named band_session (or another reliable logged-in signal in the DOM).

Suggested change
const isLoggedIn = await page.evaluate(`() => document.cookie.includes('band_session')`);
const cookies = await page.getCookies();
const isLoggedIn = cookies.some((cookie: { name?: string }) => cookie.name === 'band_session');

Copilot uses AI. Check for mistakes.
// Navigate directly to the band's post page — no home-page detour needed.
await page.goto(`https://www.band.us/band/${bandNo}/post`);

const isLoggedIn = await page.evaluate(`() => document.cookie.includes('band_session')`);
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Login detection uses document.cookie.includes('band_session'), which won’t see HttpOnly cookies. If Band’s session cookie is HttpOnly, this command will always fail with AuthRequiredError even for logged-in users. Prefer checking await page.getCookies({ domain: 'band.us' }) for the band_session cookie (or another DOM-based logged-in indicator).

Suggested change
const isLoggedIn = await page.evaluate(`() => document.cookie.includes('band_session')`);
const cookies = await page.getCookies({ domain: 'band.us' });
const isLoggedIn = cookies?.some(cookie => cookie.name === 'band_session');

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants