Skip to content

Smarter prod-run health summary and resilient systemd/logrotate handling - #49

Merged
gamblecodezcom merged 2 commits into
mainfrom
codex/main
Feb 21, 2026
Merged

Smarter prod-run health summary and resilient systemd/logrotate handling#49
gamblecodezcom merged 2 commits into
mainfrom
codex/main

Conversation

@gamblecodezcom

@gamblecodezcom gamblecodezcom commented Feb 21, 2026

Copy link
Copy Markdown
Owner

User description

Motivation

  • Provide operators clear, actionable health output instead of vague "running" lines so they can tell if the bot is actually making progress.
  • Avoid failures on hosts that lack a dedicated runewager user or have legacy log locations by falling back to valid users/groups and legacy log files.
  • Make logrotate validation and service unit generation more diagnostic so root causes (dry-run stderr, stale unit directives) are visible and corrected automatically.

Description

  • Added health tooling: HEALTH_LOG_MAX_AGE_MINUTES, get_bot_pid(), log_age_seconds() and print_health_summary() to report ✅ Bot is running (pid: ...), log freshness, and a concrete restart suggestion when /start may be unresponsive.
  • Replaced simple is_bot_running/status prints with print_health_summary calls so health checks evaluate both process and log activity.
  • Strengthened log and legacy-log handling by keeping ensure_log_paths() and using the legacy runewager.log when the primary log is empty.
  • Made ensure_logrotate_config() emit a create line using a fallback owner/group when runewager user is missing and include logrotate dry-run stderr in warnings for clearer diagnostics.
  • Made ensure_systemd_service() dynamically select a valid service User/Group when runewager is absent and validate/replace stale directives (including StandardOutput) with $PROJECT_DIR/$SYSTEMD_STDOUT_LOG values.
  • Reordered startup flow so health summaries run around launch checks and print_human_errors() runs at the end to provide context after status checks.

Testing

  • Performed a shell syntax check with bash -n prod-run.sh, which passed.
  • Reviewed the produced diff for prod-run.sh to verify added functions, reordered flow, and updated systemd/logrotate templates.

Codex Task


CodeAnt-AI Description

Provide clearer bot health summary and make systemd/logrotate resilient to missing users and legacy logs

What Changed

  • Health summary now reports whether the bot process is running (pid shown), detects which log file is active (primary or legacy), reports log age, and suggests an exact restart command if /start appears unresponsive.
  • When the primary log is empty, the script will use legacy log files for health checks so operators see recent activity when present.
  • Systemd service and logrotate generation now fall back to the system's valid user/group when the dedicated runewager user is absent, creating service/logrotate files with usable ownership and avoiding failures.
  • Logrotate dry-run errors are captured and printed so configuration problems surface in warnings instead of failing silently.
  • System health checks are run around startup steps and human-friendly error suggestions are printed after status checks, giving operators actionable diagnostics in the expected order.

Impact

✅ Clearer bot health messages with pid and log freshness
✅ Fewer service failures on hosts without runewager user
✅ More actionable logrotate diagnostics

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

@codeant-ai

codeant-ai Bot commented Feb 21, 2026

Copy link
Copy Markdown

CodeAnt AI is reviewing your PR.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@coderabbitai

coderabbitai Bot commented Feb 21, 2026

Copy link
Copy Markdown

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

The prod-run.sh script introduces health monitoring functionality with new helper functions to check bot process status and log freshness. It adds dynamic user/group resolution for systemd and logrotate configurations, refactors process detection logic, and integrates error reporting into the main startup flow for improved visibility and service management.

Changes

Cohort / File(s) Summary
Health Monitoring Infrastructure
prod-run.sh
Added constants (HEALTH_LOG_MAX_AGE_MINUTES=10) and three new functions: get_bot_pid() to detect running bot process, log_age_seconds() to compute log file age, and print_health_summary() to report bot status with restart guidance.
Process Status and Logging
prod-run.sh
Refactored is_bot_running() to use get_bot_pid() output; updated check_bot_status_twice() to invoke print_health_summary() twice with pause; integrated print_human_errors() into main flow for error visibility post-start.
Configuration and Service Management
prod-run.sh
Enhanced ensure_logrotate_config() with dynamic log ownership computation; augmented ensure_systemd_service() to resolve and use service_user/service_group at runtime in unit directives and required configurations; updated systemd unit template with dynamic user/group substitution.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Poem

🐰 Health checks bloom like clover in spring,
Bot status and logs, what good things they bring!
Dynamic users dance in the scripts,
Fresh logs, keen health—not a fault ever slips! 🌱


Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

Comment @coderabbitai help to get the list of available commands and usage tips.

@gamblecodezcom
gamblecodezcom merged commit 5af975f into main Feb 21, 2026
1 of 2 checks passed
@gamblecodezcom
gamblecodezcom deleted the codex/main branch February 21, 2026 01:03
@codeant-ai codeant-ai Bot added the size:M This PR changes 30-99 lines, ignoring generated files label Feb 21, 2026
@codeant-ai

codeant-ai Bot commented Feb 21, 2026

Copy link
Copy Markdown

Nitpicks 🔍

🔒 No security issues identified
⚡ Recommended areas for review

  • Systemd directive matching
    The required systemd directives are validated using exact line matches (grep -Fqx). Slight formatting differences, additional whitespace, or reordering in existing unit files could cause the script to consider the service unit "missing directives" and overwrite it. This replacement behavior can unexpectedly discard custom settings.

  • Process match fragility
    get_bot_pid() uses a broad pgrep pattern that can match unintended processes (other node instances, different index.js files, or command lines with similar fragments). This can yield false positives/negatives when detecting whether the bot is running and affect restart decisions.

  • Stat portability
    log_age_seconds() relies on stat -c %Y, which works on GNU stat but fails on BSD/macOS where the option is different. On non-GNU hosts this will fallback to mtime 0 and report stale logs incorrectly. This impacts health checks and alerts.

Comment thread prod-run.sh


get_bot_pid() {
pgrep -f "node .*index\.js" | head -n 1 || true

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The bot PID detection uses a very generic pgrep -f "node .*index\.js" pattern, which can match any node index.js process on the host, so on systems running multiple Node apps it may incorrectly treat an unrelated process as this bot, causing health summaries and launch logic to report the bot as running and skip starting it when it is actually down. [logic error]

Severity Level: Major ⚠️
- ❌ Bot restart skipped when another node index.js is running.
- ⚠️ Health summary misreports status using wrong PID and logs.
- ⚠️ Operators may trust false positives from prod-run.sh checks.
Suggested change
pgrep -f "node .*index\.js" | head -n 1 || true
# Restrict match to this project's index.js to avoid grabbing unrelated node processes
pgrep -f "node .*${PROJECT_DIR}/index\.js" | head -n 1 || true
Steps of Reproduction ✅
1. Deploy the app using the documented flow so that systemd or an operator runs
`/var/www/html/Runewager/prod-run.sh` as described in `README.md:102-171` and
`runewager.service:8-9`.

2. Start a different Node service on the same host with a generic entrypoint, e.g. `node
/opt/otherapp/index.js`, so that `pgrep -f "node .*index\.js"` matches this unrelated
process.

3. Stop the actual Runewager bot so there is no `node $PROJECT_DIR/index.js` process, then
invoke `./prod-run.sh` (or let systemd restart it), which calls `check_bot_status_twice()`
at `prod-run.sh:487-490`.

4. Observe that `get_bot_pid()` at `prod-run.sh:50-52` returns the PID of the other app,
causing `print_health_summary()` at `prod-run.sh:66-75` and `is_bot_running()` at
`prod-run.sh:483-485` to treat the bot as running, so `start_background_if_needed()` at
`prod-run.sh:493-513` skips launching the real bot even though it is down.
Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** prod-run.sh
**Line:** 51:51
**Comment:**
	*Logic Error: The bot PID detection uses a very generic `pgrep -f "node .*index\.js"` pattern, which can match any `node index.js` process on the host, so on systems running multiple Node apps it may incorrectly treat an unrelated process as this bot, causing health summaries and launch logic to report the bot as running and skip starting it when it is actually down.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
👍 | 👎

@codeant-ai

codeant-ai Bot commented Feb 21, 2026

Copy link
Copy Markdown

CodeAnt AI finished reviewing your PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex size:M This PR changes 30-99 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant