+- Sinisterly (https://sinister.ly)
+-- Forum: Coding (https://sinister.ly/Forum-Coding)
+--- Forum: Coding (https://sinister.ly/Forum-Coding--71)
+--- Thread: Create a wordlist generator (i.e. for bruteforcing) (/Thread-Create-a-wordlist-generator-i-e-for-bruteforcing)
Create a wordlist generator (i.e. for bruteforcing) - Deque - 11-01-2012
Hello guys,
this is a little guide for making a wordlist generator. It will give you some programming practice as well as some understanding in combinatorics. I will use Java, but the way should be understandable for coders who don't know Java. But I take as a given that you understand numeral systems (at least the binary one).
The wordlist generator shall be fast and generate all possible words for a given alphabet and a wordlength. So we can begin with defining them. We keep it simple, so we can easily prove the results later. We generate words with a length of 3 and the alphabet will contain only 0 and 1. It is important to prepare it in a way that we can change these later without any problems.
The sceleton of our generator with the first local variables defined looks like this:
Code:
public class WordListGen {
public static void main(String[] args) {
generate();
}
private static void generate() {
int wordlength = 3;
char[] alphabet = { '1', '0' };
}
}Before we really start with programming let's make sure you understand what the result has to be in the end.
A wordlength of three means we have three positions for a word to fill with characters: _ _ _
The alphabet says which characters can fill the positions. Doing this manually we have the following possibilities to arrange the characters:
000
001
010
011
100
101
110
111
Those are eight possible words. As I said you should have an understanding of numeral systems. If so, you can easily see that the results are the numbers from 0 to 7 (decimal) in the binary system. Getting all results for this particular generation just needs us to count from 0 to 7 and translate this to binary.
But we want to make the generator flexible so the alphabet can be easily changed. So we need a solution that works for everything.
How do we get the number of results?
Now we dive into combinatorics. We can compare this situation to an urn that contains the characters '1' and '0'. We have three turns and every single turn we take a character, write it on a paper and put it back to the urn. The order of the results is important, because "001" is another word than "100". That means our case is a variation with repetition (<- because we put the characters back every turn). We can get the number of results by using the formula for variation with repetition: N^k
k is the number of positions or the wordlength.
N is the number of characters in the alphabet.
Applied to our current situation we have characters^wordlength = 2^3 = 8 results and that is correct.
Let's compute this number in our program. For Java there is Math.pow to do exponentiation.
Code:
final long MAX_WORDS = (long) Math.pow(alphabet.length,
wordlength);What we do next is just counting from 0 to (MAX_WORDS - 1) which is 0 to 7. This produces our words as decimal numbers:
Code:
for (long i = 0; i < MAX_WORDS; i++) {
}But we don't want the decimals. We want the numbers in our numeral system. The radix of the numeral system is the number of characters, our case 2 which is the binary system. We compute the radix like that:
Code:
final int RADIX = alphabet.length;Most languages provide out-of-the-box functions to translate numbers to a given numeral system. However, we only need integer representations and no letters. I.e. the hexadecimal system (radix 16) uses 0-9 and the letters A-F. We use 0-15 instead for this example. This way we make sure that we can use every radix and thus large alphabets that may contain more than 35 characters. Also making our own method for that is much more efficient.
Code:
private static int[] convertToRadix(int radix, long number, int wordlength) {
int[] result = new int[wordlength];
for (int i = wordlength - 1; i >= 0; i--) {
if (number > 0) {
int rest = (int) (number % radix);
number /= radix;
result[i] = rest;
} else {
result[i] = 0;
}
}
return result;
}To explain how this conversion method works I use an example: number = 5, radix = 2, wordlength = 3
First we create our array of the length 3. So we provide three places to put numbers in: _ _ _
We run through this array backwards. Our number is not 0, so we divide the number by the radix. We put the rest of this operation (computed via the modul operator %) into the array:
number / radix = 5 / 2 = 2 rest 1
Our array: _ _ 1
The new number is the result of the division = 2
Now we repeat that:
number / radix = 2 / 2 = 1 rest 0
Our array: _ 0 1
The new number is the result of the division = 1
Last turn:
number / radix = 1 / 2 = 0 rest 1
Our array: 1 0 1
And this is the correct result for our decimal to binary conversion.
The whole code by now:
Code:
private static void generate() {
int wordlength = 3;
char[] alphabet = { '0', '1' };
final long MAX_WORDS = (long) Math.pow(alphabet.length, wordlength);
final int RADIX = alphabet.length;
for (long i = 0; i < MAX_WORDS; i++) {
int[] indices = convertToRadix(RADIX, i, wordlength);
for(int index : indices){
System.out.print(index);
}
System.out.println();
}
}Our output looks promising:
000
001
010
011
100
101
110
111
Now let's alter the alphabet. We choose the characters a and b:
Code:
char[] alphabet = { 'a', 'b' };If you have understood the code you will know that the output remains the same (the binaries above). The characters of the alphabet are never used. What we print out are the indices of the characters. We can easily change that by getting the characters of the alphabet and saving them into a char-array named word:
Code:
char[] word = new char[wordlength];
for (int k = 0; k < wordlength; k++) {
word[k] = alphabet[indices[k]];
}
System.out.println(word);This is the whole program:
Code:
public class WordListGen {
public static void main(String[] args) {
generate();
}
private static void generate() {
int wordlength = 3;
char[] alphabet = { 'a', 'b' };
final long MAX_WORDS = (long) Math.pow(alphabet.length, wordlength);
final int RADIX = alphabet.length;
for (long i = 0; i < MAX_WORDS; i++) {
int[] indices = convertToRadix(RADIX, i, wordlength);
char[] word = new char[wordlength];
for (int k = 0; k < wordlength; k++) {
word[k] = alphabet[indices[k]];
}
System.out.println(word);
}
}
private static int[] convertToRadix(int radix, long number, int wordlength) {
int[] indices = new int[wordlength];
for (int i = wordlength - 1; i >= 0; i--) {
if (number > 0) {
int rest = (int) (number % radix);
number /= radix;
indices[i] = rest;
} else {
indices[i] = 0;
}
}
return indices;
}
}And this our output:
aaa
aab
aba
abb
baa
bab
bba
bbb
That is it. You may test this further with other wordlengths and alphabets. Whatever you do: Happy coding.
Deque
RE: [Tut] Create a wordlist generator (i.e. for bruteforcing) - Jacob - 11-02-2012
You know..this is actually quit a nice approach to it. I know several people who would've just casted some char variables around in a huge for loop. This way is definitely a lot more efficient, especially if you find a way to thread it, although unneeded.
RE: [Tut] Create a wordlist generator (i.e. for bruteforcing) - Deque - 11-02-2012
Thank you. I already have threaded it in one of my applications, but rewrote it to a kind of an iterator to do so (there is a method nextWord() that is used by the threads to get the next generated word).
I figured for an example for programming beginners the unthreaded approach is enough to deal with.
RE: [Tut] Create a wordlist generator (i.e. for bruteforcing) - Psycho_Coder - 04-06-2013
very well explained. I liked it especially that you explained the mathematical part as well. Thank you.
RE: [Tut] Create a wordlist generator (i.e. for bruteforcing) - zomgwtfbbq - 04-06-2013
Great tutorial, I've created something similar where you can use user's input such as keywords to create the wordlist. This allows you to create a wordlist targeted at a single user. If you want to I can post the source. It's the thc_pg module in the hacksuite cms.
RE: [Tut] Create a wordlist generator (i.e. for bruteforcing) - Deque - 04-06-2013
@Psycho_Coder: Thanks.
(04-06-2013, 04:05 PM)zomgwtfbbq Wrote: Great tutorial, I've created something similar where you can use user's input such as keywords to create the wordlist. This allows you to create a wordlist targeted at a single user. If you want to I can post the source. It's the thc_pg module in the hacksuite cms.
Of course you can post the source (create a new thread for it, I will look at it). That sounds very interesting.
RE: [Tut] Create a wordlist generator (i.e. for bruteforcing) - Psycho_Coder - 04-06-2013
(04-06-2013, 04:05 PM)zomgwtfbbq Wrote: Great tutorial, I've created something similar where you can use user's input such as keywords to create the wordlist. This allows you to create a wordlist targeted at a single user. If you want to I can post the source. It's the thc_pg module in the hacksuite cms.
Do you mean something like crunch in Backtrack,
RE: [Tut] Create a wordlist generator (i.e. for bruteforcing) - zomgwtfbbq - 04-07-2013
(04-06-2013, 05:41 PM)Deque Wrote: @Psycho_Coder: Thanks.Does it matter if it's a module (part of a cms, so you will need the cms on top of it)?
(04-06-2013, 04:05 PM)zomgwtfbbq Wrote: Great tutorial, I've created something similar where you can use user's input such as keywords to create the wordlist. This allows you to create a wordlist targeted at a single user. If you want to I can post the source. It's the thc_pg module in the hacksuite cms.
Of course you can post the source (create a new thread for it, I will look at it). That sounds very interesting.
(04-06-2013, 06:05 PM)Psycho_Coder Wrote:I haven't used that program. thc_pg takes for example the date of birth, favorite food, name of partner etc. combines those words(also with other words eg 123,321,qwerty), substrings them, reverses words and much more. It creates wordlists that are several MBs large(depending on the amount of keywords you specify).(04-06-2013, 04:05 PM)zomgwtfbbq Wrote: Great tutorial, I've created something similar where you can use user's input such as keywords to create the wordlist. This allows you to create a wordlist targeted at a single user. If you want to I can post the source. It's the thc_pg module in the hacksuite cms.
Do you mean something like crunch in Backtrack,
RE: [Tut] Create a wordlist generator (i.e. for bruteforcing) - Psycho_Coder - 04-07-2013
Quote:(04-06-2013, 06:05 PM)Psycho_Coder Wrote:I haven't used that program. thc_pg takes for example the date of birth, favorite food, name of partner etc. combines those words(also with other words eg 123,321,qwerty), substrings them, reverses words and much more. It creates wordlists that are several MBs large(depending on the amount of keywords you specify).(04-06-2013, 04:05 PM)zomgwtfbbq Wrote: Great tutorial, I've created something similar where you can use user's input such as keywords to create the wordlist. This allows you to create a wordlist targeted at a single user. If you want to I can post the source. It's the thc_pg module in the hacksuite cms.
Do you mean something like crunch in Backtrack,
Well the number of charecters are many then the number of wordlists will be very large as well.
RE: [Tut] Create a wordlist generator (i.e. for bruteforcing) - Deque - 04-07-2013
(04-07-2013, 01:37 PM)zomgwtfbbq Wrote: Does it matter if it's a module (part of a cms, so you will need the cms on top of it)?
No, it doesn't matter. The interesting part is the algorithm (imho) and that's something you can grasp without having the cms.