Skip to content

feat: 使用 curl_cffi 模拟浏览器会话#594

Open
MoYeRanqianzhi wants to merge 2 commits intoSamueli924:mainfrom
MoYeRanqianzhi:feat/curl-cffi-browser-session
Open

feat: 使用 curl_cffi 模拟浏览器会话#594
MoYeRanqianzhi wants to merge 2 commits intoSamueli924:mainfrom
MoYeRanqianzhi:feat/curl-cffi-browser-session

Conversation

@MoYeRanqianzhi
Copy link
Copy Markdown

@MoYeRanqianzhi MoYeRanqianzhi commented Mar 24, 2026

将超星核心请求链路切换为 curl_cffi 浏览器仿真会话,减少默认 requests 指纹暴露。

主要改动:

  • 新增 api/http.py 统一创建浏览器会话
  • 将登录、课程、任务点、视频状态和验证码请求切换到 curl_cffi
  • api/config.py 中集中管理浏览器指纹、超时和重试参数
  • 更新 requirements.txt,加入 curl_cffi

范围控制:

  • 仅调整超星核心请求链路
  • 未修改题库、通知等外部服务逻辑

验证:

  • 已完成实际测试,功能正常
  • uv run main.py -c config.ini
  • uv run python -m compileall api main.py app.py

Summary by CodeRabbit

  • New Features

    • Enhanced HTTP session handling with improved browser impersonation for better compatibility.
    • Automatic retry mechanism with smart backoff for handling network interruptions and transient failures.
  • Bug Fixes

    • Improved session management timeout configuration and error handling.
  • Chores

    • Updated HTTP client dependencies.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 24, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5fd2894f-d78f-4882-904d-e33bdd3bf370

📥 Commits

Reviewing files that changed from the base of the PR and between eecf679 and f94286f.

📒 Files selected for processing (3)
  • api/base.py
  • api/cookies.py
  • api/http.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • api/cookies.py

📝 Walkthrough

Walkthrough

Replaced the requests-based HTTP layer with a curl_cffi-backed BrowserSession that applies browser impersonation, default headers, configurable retries with exponential backoff and jitter, and changes call sites to use BrowserSession and BrowserRequestException; added related config constants and cookie/session typing protocols.

Changes

Cohort / File(s) Summary
HTTP client & callsite migration
api/http.py, api/base.py, api/captcha.py, api/...study* (work submission changes referenced), requirements.txt
Added BrowserSession and create_browser_session() (curl_cffi-based) with retryable override and backoff. Replaced direct requests.Session usage with BrowserSession at call sites, switched exception handling to BrowserRequestException, and set retryable=False for login/cookie validation/work POSTs. Added curl_cffi>=0.14.0,<0.15.0.
Configuration updates
api/config.py
Added BROWSER_IMPERSONATE, REQUEST_TIMEOUT, REQUEST_RETRIES, REQUEST_RETRY_BACKOFF. Replaced HEADERS with Chrome/142 macOS client hints, added Accept-Language, and introduced WORK_HEADERS for XHR/work submissions with Sec-Fetch/Origin fields.
Cookie & typing adjustments
api/cookies.py
Removed direct requests typing; introduced Protocol types CookieContainer and CookieSession and changed save_cookies() to accept a CookieSession-compatible object while preserving serialization logic.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 I swapped my paws for browser hands today,
Curl-cffi hops where requests used to play.
Retries wobble with jittered spin,
Cookies tucked, headers gleam like linen,
A rabbit's hop: secure, retry, and sway. 🥕✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: migrating from requests to curl_cffi for browser-like session management across the codebase.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
api/base.py (1)

406-406: ⚠️ Potential issue | 🟡 Minor

Fix string formatting syntax error.

Line 406 uses % formatting but the logger expects {} placeholders (loguru style), causing the status code to not be interpolated correctly.

🐛 Proposed fix
-            logger.debug("刷新视频状态返回码异常: {}"% resp.status_code)
+            logger.debug("刷新视频状态返回码异常: {}", resp.status_code)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@api/base.py` at line 406, The logger.debug call that currently uses "%"
interpolation is incorrect for the logging API being used; update the
logger.debug invocation (the line calling logger.debug with resp.status_code) to
use the logger's placeholder style by passing the message with a {} placeholder
and the resp.status_code as an argument (or alternatively use an f-string), so
the status code is interpolated correctly.
🧹 Nitpick comments (1)
api/cookies.py (1)

7-7: Consider adding a Protocol type hint for better documentation.

The type annotation was removed to decouple from requests.Session. While this works via duck typing, consider adding a Protocol type or a union type hint to document the expected interface (session.cookies.items()).

💡 Optional: Add type hint for clarity
+from typing import Protocol
+
+class CookieSession(Protocol):
+    `@property`
+    def cookies(self): ...
+
-def save_cookies(session):
+def save_cookies(session: CookieSession):

Alternatively, use a simple comment or Any type if Protocol feels heavy.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@api/cookies.py` at line 7, Add a lightweight Protocol to document the
expected interface for the save_cookies(session) parameter: define a
CookieSession Protocol (import Protocol from typing) that requires a cookies
attribute exposing an items() method returning an iterable of (str, str) (or use
typing.Iterable[tuple[str, str]]), then type the function as def
save_cookies(session: CookieSession) to make the duck-typed expectation
explicit; alternatively, if you prefer minimal change, annotate session as Any
or include a one-line comment referencing the required session.cookies.items()
behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@api/http.py`:
- Around line 32-46: The final "raise last_exc" in the request method is
unreachable because the except block re-raises when attempts exhausted; remove
the dead code at the end of request in api/http.py (the unreachable "raise
last_exc") so control flow is clear; reference: method request, exception
BrowserRequestException, constants SAFE_RETRY_METHODS, and attributes _retries
and _retry_backoff when locating the logic to edit.

---

Outside diff comments:
In `@api/base.py`:
- Line 406: The logger.debug call that currently uses "%" interpolation is
incorrect for the logging API being used; update the logger.debug invocation
(the line calling logger.debug with resp.status_code) to use the logger's
placeholder style by passing the message with a {} placeholder and the
resp.status_code as an argument (or alternatively use an f-string), so the
status code is interpolated correctly.

---

Nitpick comments:
In `@api/cookies.py`:
- Line 7: Add a lightweight Protocol to document the expected interface for the
save_cookies(session) parameter: define a CookieSession Protocol (import
Protocol from typing) that requires a cookies attribute exposing an items()
method returning an iterable of (str, str) (or use typing.Iterable[tuple[str,
str]]), then type the function as def save_cookies(session: CookieSession) to
make the duck-typed expectation explicit; alternatively, if you prefer minimal
change, annotate session as Any or include a one-line comment referencing the
required session.cookies.items() behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a789d625-924b-4a77-8eaa-4990ff53b11b

📥 Commits

Reviewing files that changed from the base of the PR and between d4816a5 and eecf679.

📒 Files selected for processing (6)
  • api/base.py
  • api/captcha.py
  • api/config.py
  • api/cookies.py
  • api/http.py
  • requirements.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant