[3SidedSquare]

The idea of a rate-based is that you screen and filter out high-rate traffic. That screening and filtering mechanism is the very thing you're trying to innovate.


1148 B = ~0.001MB.

Let's say we have 750 users on the whitelist( about what sinister.ly has, and keep in mind this is a relatively small form ), and we'll be nice and say only 500 use the full amount of bytes. That's half a MB. Still insignificant on a server, but important to recognize it's scalability issues.

Take ha-ck-forums as an example. Their most active users in 1 day was ~2000. If we assume the same percent of people use the full memory, we get ~1332 using the full memory requirement, totaling at 1.5 MB, and that assumes we only care about those 2000, 0.5% of the total registered users.

A blacklist would be easier, and would be a true rate-based system. When a certain type of IP( maybe an IP range ) is detected as an attacker, create a regular expression and add it to an array to test IPs against.

Alright, I did a thing, and here are the results:

Useing the code:

Code:

class node(): def __init__(self): self.trust = 1 self.children={} def setChild(self, num, node): self.children[num] = node def hasChild(self,num): return num in self.children def get(self, num): return self.children[num] head = node() HEADER = '\033[95m' OKBLUE = '\033[94m' OKGREEN = '\033[92m' WARNING = '\033[93m' FAIL = '\033[91m' ENDC = '\033[0m' end = None class setup(): def __init__(self): pass def makeIP(self,parent,ipLeft): if(ipLeft%10 == ipLeft): ##If we've reached the begining, make a node and quit. n = node() n.setChild(ipLeft,1) parent.setChild(ipLeft,n) return n else: ##If we've not reached the begining nodeNumber = ipLeft%10 nextIpLeft = int(ipLeft/10) if(parent.hasChild(nodeNumber)): ##If the parrent already has this child return self.makeIP(parent.get(nodeNumber),nextIpLeft) else: ##The parrent does not have the node child = node() self.makeIP(child,nextIpLeft) parent.setChild(nodeNumber,child) return parent def printTree(self,parent): for i in range(0,9): if(parent.hasChild(i)): if(parent.get(i) != 1): self.printTree(parent.get(i)) else: print(parent.get(i)) def searchIP(self,parent,ipLeft): if(ipLeft%10 == ipLeft): ##if we've reached the end if(parent.hasChild(ipLeft)): return True else: return False else: nodeNumber = ipLeft%10 nextIPLeft = int(ipLeft/10) if(parent.hasChild(nodeNumber)): return self.searchIP(parent.get(nodeNumber),nextIPLeft) setup().makeIP(head,255255255255) setup().makeIP(head,255255255254) setup().makeIP(head,255255155255) setup().searchIP(head,255255255255)

Searching for the ip 100,000 times takes 1.0253695160000689 seconds, slightly more then opening, reading, and closing a small text file, effectively doubling the time it takes to serve .html files.

Additionally, non-added addresses take less time (xbar of 0.512684768, with a standard deviation of 0.002628457 per 100,000) to realize they are not in the list.

To find memory taken, I used pympler, replaced the code at the end with

Code:

import random from pympler import tracker memory_tracker = tracker.SummaryTracker() memory_tracker.print_diff() l = 0 while(l < 400000): randIP = random.normalvariate(113113113113, 50) if(setup().searchIP(head, randIP)): pass else: setup().makeIP(head,randIP) l = l+1 print("Generated " + str(l) + " ip's") memory_tracker.print_diff()

Memory taken for a normal distribution of 400,000(H-F) unique IP address around 113.113.113.113 with a standard deviation of 50 yields the following:

Code:

types | # objects | total size ========== | =========== | ============ dict | 8995841 | 2.36 GB instance | 4697920 | 322.58 MB float | 303177 | 6.94 MB str | 2 | 97 B list | 1 | 96 B int | 2 | 48 B

I'm a little on edge to even try to look at it right now, I'll figure it out after a little dota2. Just looking at 2.36 gb I assume it means my method is worse than other methods out there. Thanks to everyone, and to w00t for his persistence.