SRL Archive - Python/Perl Help and Tutorials

To use, put the source in a .py and in the same folder include proxies_1.txt (an example). There can be multiple proxies_X.txt files (X being numbers).
Also include a working_proxies.txt in the same directory. It should look something like this, for those of you that cannot read and or understand the directions.

You can specify the amount of threads to use when calling the script, or after running the script.

For example: `python proxy_tester.py 6', or `python proxy_tester.py' it will then ask for how many threads you would like to use.

Example files are attached, source code is below.
Edit: Yea well it looks like you can attach .py files, so I'll harass Wizzup about that.

# To change this template, choose Tools | Templates
# and open the template in the editor.

__author__="lucas"
__date__ ="$Nov 14, 2010 12:14:30 PM$"

import math
import os
import os.path
import urllib2
import threading
import sys

global good_list
good_list = []


class Proxish(threading.Thread):
    def __init__(self, proxy_list, threadnum, timeout=5):
        threading.Thread.__init__(self)
        self.good_list = []
        self.thread_number = threadnum
        self.timeout = timeout
        self.good_list = []
        self.list = proxy_list

    def verify_list(self, proxy_list):
        working_list = []
        for prox in proxy_list:
            try:
                proxy = urllib2.ProxyHandler({'http':'http://'+prox+'/'})
                opener = urllib2.build_opener(proxy)
                site_code = opener.open("http://ipchicken.com/", timeout=self.timeout).readlines()
                ip = site_code[site_code.index('      <p align="center"><font face="Verdana, Arial, Helvetica, sans-serif" size="5" color="#0000FF"><b> \r\n')+1].strip(' <br>\r\n')
                print '[Thread:', self.thread_number, '] Current IP:', ip
                print '[Thread:', self.thread_number, '] Proxy works:', prox
                print '[Thread:', self.thread_number, '] match:', True if ip == prox.split(':')[0] else False
                working_list.append(prox)
            except Exception, e:#urllib2.URLError:
                print '[Thread:', self.thread_number, '] Proxy failed', prox
                print '[Thread:', self.thread_number, '] Proxy failed', e
        print '[Thread:', self.thread_number, '] Working Proxies:', working_list
        return working_list

    def run(self):
        global good_list
        good_list += self.verify_list(self.list)
        print '[All       ] Working Proxies:', good_list
        f = open('working_proxies.txt', 'r+')
        to_write = ''
        for i in good_list:
            to_write+= i+'\n'
        f.write(to_write)
        f.close()


def get_proxy_list():
    #get the directory
    file = os.path.abspath(__file__)
    for i in sorted(range(len(file)), reverse=True):
        if '/' in file[i] or '\\' in file[i]:
            directory = file[:i+1]
            break
    file_list = os.listdir(directory)
    proxy_list = []
    for file in file_list:
        if len(file) > 12:
            if file[:8] == 'proxies_':
                if not file[len(file)-1]== '~': # Gedit kept making temp files and it would read those
                    proxy_list.append(directory+file)
    return proxy_list

def get_proxies(files):
    proxy_list = []
    for file in files:
        for prox in open(file, 'r+').readlines():
            proxy_list.append(prox.strip())
    return proxy_list

def setup(number_threads):
    thread_amount = float(number_threads) # how many threads do you want the program to have?
    proxy_list = get_proxies(get_proxy_list()) # load the lists
    # get the amount to div the list into
    amount = int(math.ceil(len(proxy_list)/thread_amount))
    # split the large list into multiple smaller lists
    proxy_lists = [proxy_list[x:x+amount] for x in xrange(0, len(proxy_list), amount)]
    # if there is one left over, add ito the end of the last list
    if len(proxy_list) % thread_amount > 0.0:
        proxy_lists[len(proxy_lists)-1].append(proxy_list[len(proxy_list)-1])
    return proxy_lists

def start(threads):
    lists = setup(threads)
    thread_list = []
    good_list = []
    count = 0
    for list in lists:
        thread_list.append(Proxish(list, count))
        thread_list[len(thread_list)-1].start() # the above creates, but this starts
        count+=1

if __name__ == "__main__":
    if len(sys.argv) > 1:
        start(sys.argv[1])
    else:
        start(raw_input('How many threads you would like to test proxies with: '))

This is what it looks like while working.

This is what it looks like finished and after writing the good proxies to working_proxies.txt

so this is the thing you told me you working on and i honestly had no idea wtf you were talking about…

:P

You should take a look at NiX Proxy Checkers features, found at http://myproxylists.com/. Thought its coded with PHP + ZSH. Its pretty neat tool. I don't understand much python, but looks cool.

At the top of that site it says (based on all proxy checks in the past 8 hours).
So if you've got a large list of files and want to test them quickly, use the script :p
Using many threads works best with a multicore cpu. I blasted through >3k proxies in less than an hour.

Update soon: forgot to add in the option of changing the timeout parameter when starting when clearly the code supports it.