Add domain allowlist and private IP blocking to prevent SSRF attacks

[Copilot]

The folder URL validation only checked HTTPS protocol, allowing SSRF attacks against internal services via URLs like https://localhost:8080/admin or https://192.168.1.1/.

Changes

• Domain allowlist: Restrict URLs to github.com and githubusercontent.com (including subdomains)
• IP address blocking: Reject direct IP access to private ranges (RFC 1918), loopback, link-local, and reserved addresses for both IPv4 and IPv6
• Validation order: Check IP addresses before domain allowlist to prevent bypass attempts
• Enhanced logging: Provide specific rejection reasons for debugging

Implementation

def validate_folder_url(url: str) -> bool:
    """
    Validate that the folder URL is safe (HTTPS only, trusted domains).
    Blocks SSRF attacks by allowing only trusted domains and blocking private IPs.
    """
    if not url.startswith("https://"):
        return False
    
    parsed = urlparse(url)
    host = parsed.hostname
    
    # Block private/loopback IPs
    try:
        ip = ipaddress.ip_address(host)
        if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
            return False
    except ValueError:
        pass  # Not an IP, continue to domain check
    
    # Allowlist domains
    ALLOWED_DOMAINS = {"github.com", "githubusercontent.com"}
    host_lower = host.lower()
    return any(host_lower == d or host_lower.endswith(f".{d}") for d in ALLOWED_DOMAINS)

Attack Vectors Mitigated

• Localhost: https://localhost:8080/admin, https://127.0.0.1/
• Private networks: https://192.168.1.1/, https://10.0.0.1/, https://172.16.0.1/
• IPv6: https://[::1]/, https://[fd00::1]/
• Untrusted domains: https://evil.com/malicious

Original prompt

Pull Request: #1

The folder URL validation only checks for the HTTPS protocol but doesn't validate the host. This could still allow SSRF attacks against internal services accessible via HTTPS (e.g., "https://localhost:8080/admin" or "https://192.168.1.1/"). Consider adding an allowlist of trusted domains (e.g., only allowing github.com and githubusercontent.com) or at minimum blocking private IP ranges and localhost.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[Copilot]

Pull request overview

This pull request enhances the folder URL validation function to prevent Server-Side Request Forgery (SSRF) attacks by implementing domain allowlisting and blocking private IP addresses. The changes address a security vulnerability where the previous implementation only validated the HTTPS protocol without checking the target host.

Key changes:

• Added domain allowlist validation restricting URLs to github.com and githubusercontent.com (including subdomains)
• Implemented IP address validation to block private, loopback, link-local, and reserved IP ranges for both IPv4 and IPv6
• Added comprehensive error logging for different validation failure scenarios

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

This validation only checks the hostname at URL parsing time, which doesn't protect against DNS rebinding attacks. An attacker could register a subdomain under github.com or githubusercontent.com (if possible) or exploit Time-of-Check-Time-of-Use (TOCTOU) issues where a domain resolves to a safe IP during validation but to a private IP during the actual HTTP request. Consider implementing additional protections such as: 1) Resolving the hostname to IP addresses before making requests and validating those IPs, or 2) Configuring the httpx client with connection restrictions to block private IP ranges at the connection level.

[Copilot]

The ALLOWED_DOMAINS constant is defined inside the function, which means it's recreated on every function call. Consider moving this to a module-level constant (e.g., after the imports or with other constants like API_BASE) to improve performance and make it easier to maintain and test. This would also make it more discoverable for future configuration needs.