Add allow/deny feature for forward URLs

[UiP9AV6Y]

we operate the application in kubernetes and want to avoid access to internal ressources via request-baskets, as it can be turned into an open proxy. the most flexible use is probably to provide the ability to allow/deny domains and ip ranges.

example use cases:

• request-baskets --allow-domain example.com --deny-domain dev.example.com
  baskets can only be configured with URLs with example.com
  as domain (e.g. https://api.prod.example.com),
  except URLs pointing to dev.example.com
  (e.g. https://api.dev.example.com)

• request-baskets --deny-domain cluster.local
  baskets can forward any URL except to URLS under the
  cluster.local domain

• request-baskets --deny-address 127.0.0.0/8
  baskets can not forward URLs pointing to the loopback
  interface

• request-baskets --deny-private-address
  shortcut for denying forwards to all addresses designated
  for private use (192.168.0.0/16, 172.16.0.0/12,
  10.0.0.0/8) by the IANA

• request-baskets --allow-address 192.168.1.0/24 --deny-address 192.168.1.1/32 --deny-address 192.168.1.254/32
  only URLs resolving to a class B private net are allowed,
  except 192.168.1.1 and 192.168.1.254

by default, everything is allowed and nothing is denied (just as
before). denials are applied after allowances, so that you
can allow on a broad scale and deny granular.

given that the validation only has to be performed on basket
creation/updates, the resource overhead for domain resolution
should be relatively low.

[eliliam]

Any update on this? This would be a great feature to add

[darklynx]

PRs are welcome

[darklynx]

we operate the application in kubernetes and want to avoid access to internal ressources via request-baskets, as it can be turned into an open proxy.

That is a very valid concern 👍

The easiest solution is an option to disable forwarding in general.

The proposed solution sounds like a reasonable amount of development with additional testing of corner cases like resolving a domain into IP and sub-network. Maybe there are already libraries in Go that solves that problem and they can be easily applied here. But I'm not aware of them, and not that familiar with such kind of libraries. Also I have a limited free time at the moment.

Hence, PRs for improvements are welcome and very much appreciated.

Thank you! 🙏

[darklynx]

I think that the unrestricted forwarding feature made the Request Baskets "famous" and got us to the CVE database 😅:
https://nvd.nist.gov/vuln/detail/CVE-2023-27163

And some samples of Request Baskets misuse, when running the service non-isolated from the rest of your network 😰:
https://cyb3rc4t.hashnode.dev/htb-writeupsau