我用这个工具，能正常得到一些代理ip，类似：
http://127.0.0.1:8080/get?type=HTTP&count=2&anonymity=all
[
{
"Ip": "117.160.250.133",
"Port": "9999",
"Country": "中国",
"Province": "河南",
"City": "安阳",
"Isp": "移动",
"Type": "HTTP",
"Anonymity": "透明",
"Time": "2023-03-21 20:25:57",
"Speed": "85.7072ms",
"SuccessNum": 5,
"RequestNum": 5,
"Source": "scrape代理"
},
{
"Ip": "117.159.15.99",
"Port": "9091",
"Country": "中国",
"Province": "河南",
"City": "郑州",
"Isp": "移动",
"Type": "HTTP",
"Anonymity": "普匿",
"Time": "2023-03-21 20:25:57",
"Speed": "130.7455ms",
"SuccessNum": 5,
"RequestNum": 5,
"Source": "scrape代理"
}
]
我将得到的这些ip 和 port 放到我的程序中，我使用了colly 这个工具, 代码类似：
...
// 获取可被抓取的域名
domains := strings.Split(spiders.Domains, ",")
// 使用go_proxy_pool 得到的代理ip和port
proxyUrls := []string{
"http://117.160.250.133:9999",
"http://117.159.15.99:9091",
"http://116.113.68.130:9091",
"http://221.6.215.202:9091",
"http://117.160.250.138:8080",
"http://117.160.250.134:8081",
"http://117.160.250.131:8899",
"http://27.15.232.197:9091",
"http://222.139.221.185:9091",
"http://42.228.61.245:9091",
}
rp, err := proxy.RoundRobinProxySwitcher(proxyUrls...)
if err != nil {
log.Fatal(err)
}
// Collector
collector := colly.NewCollector(
colly.AllowedDomains(domains...),
colly.AllowURLRevisit(),
)
collector.SetRequestTimeout(120 * time.Second)
collector.SetProxyFunc(rp)
...
但是启动后，总是报下面Access Denied的错误，类似：
2023/03/22 15:10:36 (D:/work/aaa/main.go:133 main.Run.func1) [info] OnRequest: Req.ID: 2, urlType:listPage, Req.URL: https://www.baidu.com/list/, q.IsEmpty: true, q.Size: 0, q.Threads: 20
2023/03/22 15:10:46 (D:/work/aaa/main.go:300 main.Run.func4) [info] OnError: [2]listPage, https://www.baidu.com/list/, Get "https://www.baidu.com/list/": Access Denied
请问是哪里，搞错了？
我用这个工具，能正常得到一些代理ip，类似：
http://127.0.0.1:8080/get?type=HTTP&count=2&anonymity=all
我将得到的这些ip 和 port 放到我的程序中，我使用了colly 这个工具, 代码类似：
但是启动后，总是报下面Access Denied的错误，类似：
请问是哪里，搞错了？