通过squid轮循代理IP来采集网站
爬虫使用过程中经常会遇到反爬限制的网站，爬的快一点就会被封ＩＰ，那有没有一种简单的办法突破IP速度限制呢？
当然有一点爬虫经验的朋友肯定会想到使用代理突破防御．网络上有很多免费代理随便扫描一批就可以使用．每增加一个代理ＩＰ爬取速度就会增加一倍，所以使用代理爬网站是最经济实用的．
虽然pyspider支持代理配置，但是有很多代理那应该怎么配置呢？
其实作者已经想到了这个问题，并且在问答模块里也回复过多次网友的回复，使用squid就可以方便的实现代理ip的轮循．下面我们来看看squid怎么配置来轮循代理IP．
首先是先安装squid.ubuntu系统直接使用下面的命令安装squid：
sudo apt-get install squid3
如果是centos系统就使用下面的命令安装：
yum -y install squid
安装成功后修改配置文件为：下载配置文件
acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access deny manager http_access allow all http_access allow localhost http_access deny all http_port 6666 ＃代理端口，可以自己指定 coredump_dir /var/spool/squid3 refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880 refresh_pattern . 0 20% 4320 #visible_hostname haha via off forwarded_for delete #forwarded_for off #follow_x_forwarded_for deny all #request_header_access X-Forwarded-For deny all #request_header_access From deny all #request_header_access Referer deny all #request_header_access User-Agent deny all ###cache_peer start 以下是代理ＩＰ列表 cache_peer 119.122.93.112 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=119.122.93.112820 cache_peer 219.128.136.242 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=219.128.136.242712 cache_peer 125.33.188.32 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=125.33.188.32867 cache_peer 58.100.106.139 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=58.100.106.139828 cache_peer 121.207.6.126 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=121.207.6.126574 cache_peer 14.209.24.31 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=14.209.24.31685 cache_peer 118.144.176.6 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=118.144.176.6621 cache_peer 104.236.31.8 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=104.236.31.8937 cache_peer 36.82.133.42 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=36.82.133.42776 cache_peer 110.246.104.84 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=110.246.104.84296 cache_peer 107.151.142.126 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=107.151.142.126894 cache_peer 50.81.91.68 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=50.81.91.68944 cache_peer 118.69.66.63 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=118.69.66.63818 cache_peer 177.130.59.66 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=177.130.59.66807 cache_peer 123.7.82.20 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=123.7.82.2025 cache_peer 59.104.194.23 parent 8088 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=59.104.194.23238 cache_peer 114.27.132.112 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=114.27.132.11270 cache_peer 122.226.141.67 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=122.226.141.67352 cache_peer 120.52.72.23 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=120.52.72.23762 cache_peer 58.242.248.6 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=58.242.248.6635 cache_peer 117.41.144.21 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=117.41.144.21791 cache_peer 218.10.97.115 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=218.10.97.115503 cache_peer 123.121.108.86 parent 9999 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=123.121.108.86916 cache_peer 117.168.73.233 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=117.168.73.233367 cache_peer 180.183.159.82 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=180.183.159.82713 cache_peer 162.243.133.13 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=162.243.133.13678 cache_peer 115.221.233.172 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=115.221.233.172856 cache_peer 112.92.206.113 parent 9999 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=112.92.206.113380 cache_peer 104.236.54.196 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=104.236.54.196892 cache_peer 121.238.231.97 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=121.238.231.9752 cache_peer 121.139.156.58 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=121.139.156.58256 cache_peer 124.202.180.6 parent 8118 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=124.202.180.6757 cache_peer 106.58.57.116 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=106.58.57.116650 cache_peer 119.18.234.60 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=119.18.234.6072 cache_peer 192.232.243.59 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=192.232.243.59959 cache_peer 61.166.56.177 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=61.166.56.177170 cache_peer 60.11.41.69 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=60.11.41.69848 cache_peer 107.151.142.117 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=107.151.142.117371 cache_peer 203.88.172.70 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=203.88.172.70197 cache_peer 60.191.158.211 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=60.191.158.211928 cache_peer 179.177.223.114 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=179.177.223.114454 cache_peer 171.5.136.68 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=171.5.136.68656 cache_peer 118.69.168.101 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=118.69.168.10157 cache_peer 1.9.166.132 parent 800 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=1.9.166.132855 cache_peer 218.7.170.190 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=218.7.170.190295 cache_peer 124.193.51.249 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=124.193.51.249165 cache_peer 223.199.45.13 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=223.199.45.13135 cache_peer 138.19.5.196 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=138.19.5.196957 cache_peer 1.82.216.134 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=1.82.216.134996 cache_peer 118.144.154.253 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=118.144.154.25328 cache_peer 104.236.124.96 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=104.236.124.96253 cache_peer 120.52.72.22 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=120.52.72.2261 cache_peer 128.199.132.138 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=128.199.132.138986 cache_peer 139.196.233.180 parent 21193 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=139.196.233.180406 cache_peer 125.113.132.132 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=125.113.132.1326 cache_peer 59.108.201.239 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=59.108.201.239180 cache_peer 111.206.81.248 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=111.206.81.248794 cache_peer 101.201.235.141 parent 8000 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=101.201.235.141452 cache_peer 58.242.248.5 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=58.242.248.5811 cache_peer 110.231.144.72 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=110.231.144.72172 cache_peer 107.151.152.212 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=107.151.152.212335 cache_peer 179.242.95.178 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=179.242.95.178439 cache_peer 180.175.240.83 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=180.175.240.83748 cache_peer 202.171.253.72 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=202.171.253.7293 cache_peer 113.106.213.162 parent 9797 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=113.106.213.162513 cache_peer 60.191.163.147 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=60.191.163.147256 cache_peer 182.42.216.120 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=182.42.216.120329 cache_peer 219.155.96.170 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=219.155.96.170321 cache_peer 118.244.239.2 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=118.244.239.2456 cache_peer 223.14.217.189 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=223.14.217.189243 cache_peer 123.57.190.51 parent 7777 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=123.57.190.51441 cache_peer 54.169.237.120 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=54.169.237.120300 cache_peer 58.20.249.157 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=58.20.249.157390 cache_peer 107.151.152.222 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=107.151.152.222473 cache_peer 123.161.70.5 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=123.161.70.512 cache_peer 111.252.221.24 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=111.252.221.24286 cache_peer 104.233.88.225 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=104.233.88.225846 cache_peer 178.49.228.101 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=178.49.228.101601 cache_peer 58.19.222.139 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=58.19.222.139438 cache_peer 60.191.147.202 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=60.191.147.20257 cache_peer 45.58.54.233 parent 25 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=45.58.54.233900 cache_peer 124.193.58.10 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=124.193.58.1080 cache_peer 182.48.113.11 parent 8088 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=182.48.113.1120 cache_peer 210.91.48.121 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=210.91.48.121353 cache_peer 61.53.65.54 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=61.53.65.54116 cache_peer 66.186.2.163 parent 443 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=66.186.2.163185 cache_peer 93.174.55.82 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=93.174.55.82398 cache_peer 115.226.215.24 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=115.226.215.24641 cache_peer 5.196.61.125 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=5.196.61.125734 cache_peer 218.103.42.105 parent 1080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=218.103.42.10528 cache_peer 183.19.46.57 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=183.19.46.5744 cache_peer 182.116.253.167 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=182.116.253.167414 cache_peer 111.67.202.83 parent 808 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=111.67.202.83193 cache_peer 220.248.229.45 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=220.248.229.45677 cache_peer 61.223.127.152 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=61.223.127.152617 cache_peer 112.112.70.115 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=112.112.70.115412 cache_peer 23.94.244.247 parent 25 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=23.94.244.24718 cache_peer 182.253.201.77 parent 10000 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=182.253.201.77678 cache_peer 46.101.81.85 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=46.101.81.85330 cache_peer 61.19.250.51 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=61.19.250.51113 cache_peer 121.228.20.236 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=121.228.20.236444 cache_peer 5.189.142.160 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=5.189.142.160801 cache_peer 41.33.22.186 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=41.33.22.186764 cache_peer 219.131.111.235 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=219.131.111.235674 cache_peer 113.16.186.245 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=113.16.186.245192 cache_peer 60.194.100.51 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=60.194.100.51572 cache_peer 113.107.57.76 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=113.107.57.7655 cache_peer 121.14.36.38 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=121.14.36.38320 cache_peer 124.133.240.88 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=124.133.240.88846 cache_peer 1.255.53.81 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=1.255.53.81913 cache_peer 123.139.59.85 parent 9999 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=123.139.59.8563 cache_peer 101.30.208.216 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=101.30.208.216105 cache_peer 123.56.45.102 parent 82 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=123.56.45.102974 cache_peer 119.180.92.124 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=119.180.92.124917 cache_peer 84.119.63.243 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=84.119.63.243707 cache_peer 202.103.215.199 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=202.103.215.199797 cache_peer 200.29.191.149 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=200.29.191.149412 cache_peer 178.33.11.87 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=178.33.11.87157 cache_peer 107.151.136.222 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=107.151.136.222351 cache_peer 124.120.30.10 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=124.120.30.10511 cache_peer 116.25.80.164 parent 8118 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=116.25.80.164416 cache_peer 36.7.172.18 parent 82 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=36.7.172.18561 cache_peer 107.151.152.211 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=107.151.152.21171 cache_peer 60.191.159.86 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=60.191.159.86462 cache_peer 107.151.136.219 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=107.151.136.21963 cache_peer 124.16.70.11 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=124.16.70.1185 cache_peer 202.77.57.124 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=202.77.57.12412 cache_peer 178.18.25.184 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=178.18.25.184670 cache_peer 123.30.238.16 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=123.30.238.16646 cache_peer 120.28.45.202 parent 8090 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=120.28.45.202230 cache_peer 1.82.216.135 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=1.82.216.135556 cache_peer 113.106.213.162 parent 9797 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=113.106.213.162531 cache_peer 103.54.147.132 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=103.54.147.132130 cache_peer 36.72.22.93 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=36.72.22.93296 cache_peer 107.151.142.122 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=107.151.142.122667 cache_peer 60.191.164.83 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=60.191.164.8330 cache_peer 183.19.48.88 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=183.19.48.88741 cache_peer 123.120.21.229 parent 9999 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=123.120.21.229516 cache_peer 188.166.194.193 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=188.166.194.193340 cache_peer 177.17.231.81 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=177.17.231.81204 cache_peer 82.196.10.29 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=82.196.10.29451 cache_peer 221.226.67.202 parent 8118 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=221.226.67.202404 cache_peer 120.132.71.212 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=120.132.71.212125 cache_peer 106.38.251.62 parent 8088 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=106.38.251.62671 cache_peer 183.88.73.178 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=183.88.73.178311 cache_peer 106.75.128.89 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=106.75.128.89490 cache_peer 61.75.2.124 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=61.75.2.124394 cache_peer 120.52.72.55 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=120.52.72.55753 cache_peer 27.131.47.132 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=27.131.47.132988 cache_peer 124.192.106.247 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=124.192.106.247426 cache_peer 110.179.89.175 parent 81 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=110.179.89.175194 cache_peer 113.107.57.76 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=113.107.57.76804 cache_peer 180.107.84.131 parent 8888 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=180.107.84.131113 cache_peer 46.101.23.211 parent 3129 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=46.101.23.211958 cache_peer 202.100.167.142 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=202.100.167.142917 cache_peer 124.126.118.223 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=124.126.118.223828 cache_peer 118.253.82.126 parent 9797 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=118.253.82.1263 cache_peer 110.7.106.143 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=110.7.106.143872 cache_peer 180.250.32.66 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=180.250.32.66706 cache_peer 120.27.210.131 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=120.27.210.131656 cache_peer 183.7.88.110 parent 8118 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=183.7.88.110224 cache_peer 139.196.73.101 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=139.196.73.101952 cache_peer 61.160.212.74 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=61.160.212.74367 cache_peer 120.236.148.199 parent 2226 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=120.236.148.19917 cache_peer 61.166.56.177 parent 3128 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=61.166.56.17748 cache_peer 171.116.209.27 parent 9797 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=171.116.209.27409 cache_peer 182.253.123.32 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=182.253.123.32619 cache_peer 107.151.136.202 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=107.151.136.20278 cache_peer 114.130.13.60 parent 8080 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=114.130.13.60764 cache_peer 101.231.250.102 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=101.231.250.102390 cache_peer 107.151.136.194 parent 80 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=107.151.136.194798 cache_peer 139.196.234.11 parent 21193 0 no-query weighted-round-robin weight=1 connect-fail-limit=2 allow-miss max-conn=5 name=139.196.234.11270 never_direct allow all
配置文件一般是在/etc/squid3/下的squid.conf文件，请对应修改．ubuntu系统可以直接全部替换，centos对应修改就可以了．重启后就可以生效了．可以把浏览器的ＨＴＴＰ代理设置成１２７．０．０．１：６６６６或是服务器的ＩＰ就可以,设置好后就可以访问能显示ＩＰ的网站查看了．如：http://httpbin.org/get
如果代理ＩＰ是免费采集的就需要写个脚本自动采集验证自动生成配置文件，然后重新载入配置文件或重启squid．
pyspider使用的时候只需要设置代理服务器为你配置好的服务器的ＩＰ就可以了，如下面项目代码．是不是很简单呢？
class Handler(BaseHandler):
crawl_config = {
'proxy': '127.0.0.1:6666'
}
感谢＠开水泡脚丫提供配置文件
转载自：http://www.pyspider.cn/jiaocheng/squid%EF%BC%BFproxy-13.html
