The stories and information posted here are artistic works of fiction and falsehood. Only a fool would take anything posted here as fact.
Thirteen Years of Service
Posts: 340
Threads: 6
Information Gathering 07-27-2013, 11:11 PM
#1
Hello [username] !
Today i will be posting a tutorial about information gathering.
Note : i found a similiar post by moroaddict but i believe that it missed some steps and the explanation is too short.
What is Information Gathering ?
Information Gathering is the process of gathering as more data about a system as possible. This is a fundamental (too many times underrated) and very first step in penetration testing.
Just keep in mind the following rule :
more information = higher chance to land a successful attack
Various Info
- All the methods i will present are suitable for every OS however I recommand using Backtrack as it comes with many features and scripts already installed
- Please notice that information gathering involves gathering info about a system, service enumeration and vulnerability identification are usually considered different steps, however i'll talk about Information Gathering like it included also Service Enumeration (most of the times they are considered part of the same process).
- Pentesting steps are the following :
information gathering -> service enumeration -> vulnerability identification -> penetration -> mantaining access -> removing traces
Information Gathering & Service Enumeration
I'll list and explain below each method, the best would be to gather data with each of them.
EDIT (thanks to @
chmod for this) : all the information gathering process, depending on the method can be active or passive. Active reconnissance involves sending a certain request directly to the system. Passive reconnissance, on the other side, involves gathering information from exteranal source (you don't send any request to the system).
- Active methods : url Listing, DNS, SMTP, SNMP, Netbios, Port Scanning
- Passive methods : WHOIS, Google, Email Harvesting, Netcraft
The methods below are purely for Information Gathering :
Whois
Spoiler:
Whois can be used either giving either domain or ip as argument :
- Reverse lookup involves giving the IP as parameter, usually you get different infos:
Code:
> whois <ip_address>
- Whois is the name of TCP protocol, used to query a central Whois database mantained by InterNIC.
- Each domain must have a whois contain all the contact info for the domain they host.
- A Whois query can return many useful information, to list some : domain, status, creation data, last update, expiratio date, registrant info, admin info, technical contact and nameservers. The result vary from query to query but this infos are likely to be always present.
- An interesting feature of the whois command is the technical and admin contacts which can be used to make some search around the web or for a SE attack.
- It is not hard to find IT admin that use their email address to register account in forums where they ask help about some configurations of security services (just think possible implications)
Google
Spoiler:
Google is the most powerful tool for information gathering. Learning how to use it, to use cached pages and advanced operators is a must (not only for information gathering). I'm not talking in depth about Google usage because that would require a lot of time (maybe in a dedicated post i will) but for those of you who are interested in learning more i signal 2 books:
- Google Hacking by Johnny Long
- Google Hacking for Penetration Testers, Vol. 2 (actually possible to find them for free using Google's fileype operator)
Google can be used for :
- Anonimity with Caches
- Directory Listing
- Directory Trasversal
- Configuration Files
- Log Digging
- Database Digging
- Data Mining in general
- Locate Vulnerable Websites
- Find Misconfigured Files
- and much more
Most used Google Advanced Operators :
- intitle
- allintext
- site
- fileype
- link
- cache
- inurl
- related
- and more
Google also offers ways to use its API but be careful while using them for non-totally-legit activity like spidering as Google might not like it.
Email Harvesting
Spoiler:
Email Harvesting is the process of getting as more emails as possible within the sytem organization. To achieve this task there are two specific tools that use Google to perform this search. These are available in Backtrack. (thanks @chomod)
Code:
> cd /pentest/enumeration/google
> ./goog-mail.py <domain>
Code:
> cd /pentest/enumeration/google
> ./theharvester.py -d <domain> -l 100 -b <data_source>
- It is interesting to notice that theharvester.py script lets you limit the data gathered with the -b option and choose the data source which can be either google, msn, linkedin etc
- Getting more email as possible can be useful for spreading a Trojan Horse for example (SE attack)
- Each email is worth checking as you might gather more info about the system
- Each email can be used to launch a Social Engineering (SE) attack
url Listing
Spoiler:
- A certain domain can have many subdomains worth checking as they might run on different servers (possible different vulnerabilities)
- For this purpose, a Python script named list-urls.py can be found in Backtrack.
Code:
> cd /pentest/enumeration/list-urls
> list-urls.py <domain>
- This will return a list of all the urls within a given domain.
- For large results it might be useful to create a simple bash script to extract the ip for each domain.
- I believe it should be trivial however here's the code :
Code:
#!/bin/bash
for hostname in $(cat url_list.txt); do
host $hostname | cut -d" " -f4
done
- Where url_list is a txt file where you saved list-urls.py output
Netcraft
Spoiler:
- Netcraft is a website that monitor various companies
- Provides different infos like : server uptimes, OS, Web Server version
- Can also be used to determine sub-domains
- http://searchdns.netcraft.com
- To provide an example :
![[Image: netcraft.png]](http://pentestlab.files.wordpress.com/2012/08/netcraft.png)
Methods listed below are part of Service Enumeration :
DNS
Spoiler:
- DNS enumeration can be used to gather informations about public and private organization's servers like IP address
- Tools like nslookup and dig (giving a certain domain as parameter) are commonly used
- nslookup command will get you to a prompt, you can also set type=mx to get mail server or type=ns to get DNS
Code:
> nslookup
> type=mx or type=ns
> domain.com
- 3 main DNS enumeration techniques : forward lookup, reverse lookup and zone transfer
- Forward lookup :
The basic idea is to bruteforce possible names within a given domain and resolve them using the host command :
- This technique involves bruteforcing so it is a good idea to automate it with a script
- Try for example ftp.domain.com, ns.domain.com etc.
- Reverse lookup :
Same usage of the forward one but you actually give ip as paramenter (eg. the ones you gathered with url Listing for example)
- Zone transfer :
It is actually the method that usually provides more info compared to the others
It exploits a weakness of misconfigured DNS servers (often happens)
To try a zone transfer you have to issue the following command with the dns servers you already gathered in previous steps
Code:
> host -l ns dns_server_already_found
- Try this with all the dns servers you found, if the server is well configured you will be returned an error message, if on the other hand the transfer succedes you will find lots of useful informations
- Backtrack actually provides dnsenum.pl script that automatically performs all the steps of DNS enumeration and i strongly recommend this one as it is faster and reliable
Code:
> cd /pentest/enumeration/dnsenum
> ./dnsenum.pl
Usage: perl dnsenum.pl <DOMAINNAME> <dns.txt>
dnsenum #
- dns.txt is a file used to perform forward lookup
SMTP
Spoiler:
- Misconfigured email servers can be used to gather info about users within the system
- SMTP supports two commands : VRFY that checks wether a user belongs to the system, EXPN that asks the server for membership of a certain mailing list
Code:
> nc -v mail_server_ip 25
//used to connect to the mail server, i recommend using netcat but you can use what you like most
VRFY abcdef
550 5.1.1 abcdef... User unknown
VRFY lady_godiva
250 2.1.5 lady_godiva <lady_godiva@email.com>
- When you provide a correct username the system answer returning some infos and this can also be used for user enumeration (a script can be made)
SNMP
Spoiler:
- SNMP is a management protocol based on UDP, used to manage network devices remotely
- It is has a weak authentication system and actually provides really a lot of informations
- Authentication system is based upon community strings which can be public or private
- All these strings are not encrypted, hence eavesdropping is possible
- Different tools to enumerate SNMP : snmpwalk, snmpcheck.pl and snmpenum.pl (all of these can be found in Backtrack)
- If you have the occasion to try it yourself you will see that the amount of infos returned is overwhelming, so it must be filtered with proper parameters
- Just to see some examples :
Code:
> snmpwalk -c public -v1 <ip_address> 1 | grep hrSWRunName | cut -d" " -f4
//this will return the process list running on the system (Windows)
> snmpwalk -c public -v1 <ip_address> 1 | grep tcpConnState | cut -d"." -f6 | sort -nu
//this will return open Tcp ports
- Take time to examine snmpwalk command usage
- snmpcheck.pl and snmpenum.pl are easier to use
Code:
> cd pentest/enumeration/snmpenum
> ./snmpcheck-1.3.pl -t <ip_address>
Netbios
Spoiler:
- Netbios is a protocol that can be found in Windows systems
- In this last period care has been taken to secure this protocol however it is still possible to gather informations on some systems
- A so called "NULL Session" is an unauthenticated Netbios session between two computer. Some times this access is actually disabled, other times it is not and it is actually possible to gather informations such as Usernames, User and Host, machine data
- Backtrack provides Smb4k tool that helps identifying all machines running Netbios protocol within a system
- Also provides samrdump.py for username enumeration:
Code:
> cd pentest/enumeration/smb-enum
> ./samrdump.py <ip_address>
Port Scanning
Spoiler:
- Portscanning involves sending appropriate requests to the system in order to determine, trough the response received, open UDP and TCP ports on the target system
- Before explaining the different portscanning techniques, notice that this activity is likely to be detected as potentially malicious traffic on the target system, as it generated noticeable traffic on all the system ports within a very low time window. More over it is considered an illegal activity in many countries. Of course the solution is to use some quite-sophisticated port scanning technique that i'm going to talk about and run the portscanning activity trough proxychains or similar
- There exist many programs to perform portscanning, i recommend using nmap or Zenmap (nap's GUI)
- A little background on how port scanning work (understanding is always important):
TCP : TCP scanning involves an attempt to establish a 3 way handshake (i assume you know what it is) with the system. TCP RFC states that when a SYN is sent to an open port than the system should reply with an ACK that actually allows us to determine wether the port is open or close (an RST is sent if the port is closed)
UDP : being UDP a stateless protocol it is not actually possible to try the establishment of a 3 way handshake. UDP port scanning involves sending to the target port/s ICMP packets. Notice that this kind of scanning can actually be unreliable due to many system dropping ICMP (so there could be false positives and false negatives)
- For nmap detailed guide i found that Ex094 wrote a really good one (credits for this go to him) : http://www.hackcommunity.com/Thread-Tuto...nnaissance
- Note : before performing all these activities on and external server, beware of honeypots, for further explanation about honeypots surf the web or browse the forum, i've seen a good post about it
Metasploit
Spoiler:
- Among all its function (http://www.offensive-security.com/metasploit-unleashed/ for further reading (highly suggested)), the Metasploit framework can also serve Information Gathering purpose
- Backtrack is recommended to use Metasploit
- The reason i didn't write about Metasploit before the edit is that is mixes both Information Gathering, Service Enumeration and Vulnerability Scanning, hence someone who finds a vulnerability might immediatly skip to the exploitation phase : DO NOT. Separation each phase is very important and trust me if I tell you that altering the order is not a good idea
- All this is done from the msfconsole (just type msfconsole to run) which is a MSF interface that provides "all-in-one" console for all the tasks that MSF provides. Looks like it:
- In order to access the Infomation Gathering functions:
Code:
msf > use auxiliary/scanner
- Actually there are many scanners available in this module, take time to revise them :
Code:
msf > use auxiliary/scanner
Display all 237 possibilities? (y or n) //a lot as you can see
- Using MSF it is actually possible to gather informations concerning : open ports, mssql, service identificaiton, sniffing passwords
- I list here some useful command for msfconsole usage:
Code:
msf > search portscan
Matching Modules
================
Name Disclosure Date Rank Description
---- --------------- ---- -----------
auxiliary/scanner/natpmp/natpmp_portscan normal NAT-PMP External Port Scanner
auxiliary/scanner/portscan/ack normal TCP ACK Firewall Scanner
auxiliary/scanner/portscan/ftpbounce normal FTP Bounce Port Scanner
auxiliary/scanner/portscan/syn normal TCP SYN Port Scanner
auxiliary/scanner/portscan/tcp normal TCP Port Scanner
auxiliary/scanner/portscan/xmas normal TCP "XMas" Port Scanner
Code:
msf > use auxiliary/scanner/mssql/mssql_ping
msf auxiliary(mssql_ping) > show options
Module options (auxiliary/scanner/mssql/mssql_ping):
Name Current Setting Required Description
---- --------------- -------- -----------
PASSWORD no The password for the specified username
RHOSTS yes The target address range or CIDR identifier
THREADS 1 yes The number of concurrent threads
USERNAME sa no The username to authenticate as
USE_WINDOWS_AUTHENT false yes Use windows authentification
- Again for further reading (which is strongly reccommended if you plan to use the Metasploit framework) take a look at http://www.offensive-security.com/metasploit-unleashed
- Thanks @chmod for reminding me
- IMPORTANT : Take time to learn and revise how to use command in MSF, understanding how it works its really important to get good results and avoid messing things
Note : Try to keep methods for information gathering more separate as possible from Service Enumeration one.
Note : Most of the times, a lot of info is to be generated, my advice is to keep it well separated and organized as it will help you during the following steps of penetration testing (also to avoid repetitions of same results that you might have gathered with different methods).
![[+]](https://sinister.ly/images/modern/collapse_collapsed.png)
•
Thirteen Years of Service
Posts: 340
Threads: 6
Information Gathering 07-27-2013, 11:11 PM
#2
Hello [username] !
Today i will be posting a tutorial about information gathering.
Note : i found a similiar post by moroaddict but i believe that it missed some steps and the explanation is too short.
What is Information Gathering ?
Information Gathering is the process of gathering as more data about a system as possible. This is a fundamental (too many times underrated) and very first step in penetration testing.
Just keep in mind the following rule :
more information = higher chance to land a successful attack
Various Info
- All the methods i will present are suitable for every OS however I recommand using Backtrack as it comes with many features and scripts already installed
- Please notice that information gathering involves gathering info about a system, service enumeration and vulnerability identification are usually considered different steps, however i'll talk about Information Gathering like it included also Service Enumeration (most of the times they are considered part of the same process).
- Pentesting steps are the following :
information gathering -> service enumeration -> vulnerability identification -> penetration -> mantaining access -> removing traces
Information Gathering & Service Enumeration
I'll list and explain below each method, the best would be to gather data with each of them.
EDIT (thanks to @
chmod for this) : all the information gathering process, depending on the method can be active or passive. Active reconnissance involves sending a certain request directly to the system. Passive reconnissance, on the other side, involves gathering information from exteranal source (you don't send any request to the system).
- Active methods : url Listing, DNS, SMTP, SNMP, Netbios, Port Scanning
- Passive methods : WHOIS, Google, Email Harvesting, Netcraft
The methods below are purely for Information Gathering :
Whois
Spoiler:
Whois can be used either giving either domain or ip as argument :
- Reverse lookup involves giving the IP as parameter, usually you get different infos:
Code:
> whois <ip_address>
- Whois is the name of TCP protocol, used to query a central Whois database mantained by InterNIC.
- Each domain must have a whois contain all the contact info for the domain they host.
- A Whois query can return many useful information, to list some : domain, status, creation data, last update, expiratio date, registrant info, admin info, technical contact and nameservers. The result vary from query to query but this infos are likely to be always present.
- An interesting feature of the whois command is the technical and admin contacts which can be used to make some search around the web or for a SE attack.
- It is not hard to find IT admin that use their email address to register account in forums where they ask help about some configurations of security services (just think possible implications)
Google
Spoiler:
Google is the most powerful tool for information gathering. Learning how to use it, to use cached pages and advanced operators is a must (not only for information gathering). I'm not talking in depth about Google usage because that would require a lot of time (maybe in a dedicated post i will) but for those of you who are interested in learning more i signal 2 books:
- Google Hacking by Johnny Long
- Google Hacking for Penetration Testers, Vol. 2 (actually possible to find them for free using Google's fileype operator)
Google can be used for :
- Anonimity with Caches
- Directory Listing
- Directory Trasversal
- Configuration Files
- Log Digging
- Database Digging
- Data Mining in general
- Locate Vulnerable Websites
- Find Misconfigured Files
- and much more
Most used Google Advanced Operators :
- intitle
- allintext
- site
- fileype
- link
- cache
- inurl
- related
- and more
Google also offers ways to use its API but be careful while using them for non-totally-legit activity like spidering as Google might not like it.
Email Harvesting
Spoiler:
Email Harvesting is the process of getting as more emails as possible within the sytem organization. To achieve this task there are two specific tools that use Google to perform this search. These are available in Backtrack. (thanks @chomod)
Code:
> cd /pentest/enumeration/google
> ./goog-mail.py <domain>
Code:
> cd /pentest/enumeration/google
> ./theharvester.py -d <domain> -l 100 -b <data_source>
- It is interesting to notice that theharvester.py script lets you limit the data gathered with the -b option and choose the data source which can be either google, msn, linkedin etc
- Getting more email as possible can be useful for spreading a Trojan Horse for example (SE attack)
- Each email is worth checking as you might gather more info about the system
- Each email can be used to launch a Social Engineering (SE) attack
url Listing
Spoiler:
- A certain domain can have many subdomains worth checking as they might run on different servers (possible different vulnerabilities)
- For this purpose, a Python script named list-urls.py can be found in Backtrack.
Code:
> cd /pentest/enumeration/list-urls
> list-urls.py <domain>
- This will return a list of all the urls within a given domain.
- For large results it might be useful to create a simple bash script to extract the ip for each domain.
- I believe it should be trivial however here's the code :
Code:
#!/bin/bash
for hostname in $(cat url_list.txt); do
host $hostname | cut -d" " -f4
done
- Where url_list is a txt file where you saved list-urls.py output
Netcraft
Spoiler:
- Netcraft is a website that monitor various companies
- Provides different infos like : server uptimes, OS, Web Server version
- Can also be used to determine sub-domains
- http://searchdns.netcraft.com
- To provide an example :
Methods listed below are part of Service Enumeration :
DNS
Spoiler:
- DNS enumeration can be used to gather informations about public and private organization's servers like IP address
- Tools like nslookup and dig (giving a certain domain as parameter) are commonly used
- nslookup command will get you to a prompt, you can also set type=mx to get mail server or type=ns to get DNS
Code:
> nslookup
> type=mx or type=ns
> domain.com
- 3 main DNS enumeration techniques : forward lookup, reverse lookup and zone transfer
- Forward lookup :
The basic idea is to bruteforce possible names within a given domain and resolve them using the host command :
- This technique involves bruteforcing so it is a good idea to automate it with a script
- Try for example ftp.domain.com, ns.domain.com etc.
- Reverse lookup :
Same usage of the forward one but you actually give ip as paramenter (eg. the ones you gathered with url Listing for example)
- Zone transfer :
It is actually the method that usually provides more info compared to the others
It exploits a weakness of misconfigured DNS servers (often happens)
To try a zone transfer you have to issue the following command with the dns servers you already gathered in previous steps
Code:
> host -l ns dns_server_already_found
- Try this with all the dns servers you found, if the server is well configured you will be returned an error message, if on the other hand the transfer succedes you will find lots of useful informations
- Backtrack actually provides dnsenum.pl script that automatically performs all the steps of DNS enumeration and i strongly recommend this one as it is faster and reliable
Code:
> cd /pentest/enumeration/dnsenum
> ./dnsenum.pl
Usage: perl dnsenum.pl <DOMAINNAME> <dns.txt>
dnsenum #
- dns.txt is a file used to perform forward lookup
SMTP
Spoiler:
- Misconfigured email servers can be used to gather info about users within the system
- SMTP supports two commands : VRFY that checks wether a user belongs to the system, EXPN that asks the server for membership of a certain mailing list
Code:
> nc -v mail_server_ip 25
//used to connect to the mail server, i recommend using netcat but you can use what you like most
VRFY abcdef
550 5.1.1 abcdef... User unknown
VRFY lady_godiva
250 2.1.5 lady_godiva <lady_godiva@email.com>
- When you provide a correct username the system answer returning some infos and this can also be used for user enumeration (a script can be made)
SNMP
Spoiler:
- SNMP is a management protocol based on UDP, used to manage network devices remotely
- It is has a weak authentication system and actually provides really a lot of informations
- Authentication system is based upon community strings which can be public or private
- All these strings are not encrypted, hence eavesdropping is possible
- Different tools to enumerate SNMP : snmpwalk, snmpcheck.pl and snmpenum.pl (all of these can be found in Backtrack)
- If you have the occasion to try it yourself you will see that the amount of infos returned is overwhelming, so it must be filtered with proper parameters
- Just to see some examples :
Code:
> snmpwalk -c public -v1 <ip_address> 1 | grep hrSWRunName | cut -d" " -f4
//this will return the process list running on the system (Windows)
> snmpwalk -c public -v1 <ip_address> 1 | grep tcpConnState | cut -d"." -f6 | sort -nu
//this will return open Tcp ports
- Take time to examine snmpwalk command usage
- snmpcheck.pl and snmpenum.pl are easier to use
Code:
> cd pentest/enumeration/snmpenum
> ./snmpcheck-1.3.pl -t <ip_address>
Netbios
Spoiler:
- Netbios is a protocol that can be found in Windows systems
- In this last period care has been taken to secure this protocol however it is still possible to gather informations on some systems
- A so called "NULL Session" is an unauthenticated Netbios session between two computer. Some times this access is actually disabled, other times it is not and it is actually possible to gather informations such as Usernames, User and Host, machine data
- Backtrack provides Smb4k tool that helps identifying all machines running Netbios protocol within a system
- Also provides samrdump.py for username enumeration:
Code:
> cd pentest/enumeration/smb-enum
> ./samrdump.py <ip_address>
Port Scanning
Spoiler:
- Portscanning involves sending appropriate requests to the system in order to determine, trough the response received, open UDP and TCP ports on the target system
- Before explaining the different portscanning techniques, notice that this activity is likely to be detected as potentially malicious traffic on the target system, as it generated noticeable traffic on all the system ports within a very low time window. More over it is considered an illegal activity in many countries. Of course the solution is to use some quite-sophisticated port scanning technique that i'm going to talk about and run the portscanning activity trough proxychains or similar
- There exist many programs to perform portscanning, i recommend using nmap or Zenmap (nap's GUI)
- A little background on how port scanning work (understanding is always important):
TCP : TCP scanning involves an attempt to establish a 3 way handshake (i assume you know what it is) with the system. TCP RFC states that when a SYN is sent to an open port than the system should reply with an ACK that actually allows us to determine wether the port is open or close (an RST is sent if the port is closed)
UDP : being UDP a stateless protocol it is not actually possible to try the establishment of a 3 way handshake. UDP port scanning involves sending to the target port/s ICMP packets. Notice that this kind of scanning can actually be unreliable due to many system dropping ICMP (so there could be false positives and false negatives)
- For nmap detailed guide i found that Ex094 wrote a really good one (credits for this go to him) : http://www.hackcommunity.com/Thread-Tuto...nnaissance
- Note : before performing all these activities on and external server, beware of honeypots, for further explanation about honeypots surf the web or browse the forum, i've seen a good post about it
Metasploit
Spoiler:
- Among all its function (http://www.offensive-security.com/metasploit-unleashed/ for further reading (highly suggested)), the Metasploit framework can also serve Information Gathering purpose
- Backtrack is recommended to use Metasploit
- The reason i didn't write about Metasploit before the edit is that is mixes both Information Gathering, Service Enumeration and Vulnerability Scanning, hence someone who finds a vulnerability might immediatly skip to the exploitation phase : DO NOT. Separation each phase is very important and trust me if I tell you that altering the order is not a good idea
- All this is done from the msfconsole (just type msfconsole to run) which is a MSF interface that provides "all-in-one" console for all the tasks that MSF provides. Looks like it:
- In order to access the Infomation Gathering functions:
Code:
msf > use auxiliary/scanner
- Actually there are many scanners available in this module, take time to revise them :
Code:
msf > use auxiliary/scanner
Display all 237 possibilities? (y or n) //a lot as you can see
- Using MSF it is actually possible to gather informations concerning : open ports, mssql, service identificaiton, sniffing passwords
- I list here some useful command for msfconsole usage:
Code:
msf > search portscan
Matching Modules
================
Name Disclosure Date Rank Description
---- --------------- ---- -----------
auxiliary/scanner/natpmp/natpmp_portscan normal NAT-PMP External Port Scanner
auxiliary/scanner/portscan/ack normal TCP ACK Firewall Scanner
auxiliary/scanner/portscan/ftpbounce normal FTP Bounce Port Scanner
auxiliary/scanner/portscan/syn normal TCP SYN Port Scanner
auxiliary/scanner/portscan/tcp normal TCP Port Scanner
auxiliary/scanner/portscan/xmas normal TCP "XMas" Port Scanner
Code:
msf > use auxiliary/scanner/mssql/mssql_ping
msf auxiliary(mssql_ping) > show options
Module options (auxiliary/scanner/mssql/mssql_ping):
Name Current Setting Required Description
---- --------------- -------- -----------
PASSWORD no The password for the specified username
RHOSTS yes The target address range or CIDR identifier
THREADS 1 yes The number of concurrent threads
USERNAME sa no The username to authenticate as
USE_WINDOWS_AUTHENT false yes Use windows authentification
- Again for further reading (which is strongly reccommended if you plan to use the Metasploit framework) take a look at http://www.offensive-security.com/metasploit-unleashed
- Thanks @chmod for reminding me
- IMPORTANT : Take time to learn and revise how to use command in MSF, understanding how it works its really important to get good results and avoid messing things
Note : Try to keep methods for information gathering more separate as possible from Service Enumeration one.
Note : Most of the times, a lot of info is to be generated, my advice is to keep it well separated and organized as it will help you during the following steps of penetration testing (also to avoid repetitions of same results that you might have gathered with different methods).
•
Thirteen Years of Service
Posts: 340
Threads: 6
Information Gathering 07-27-2013, 11:11 PM
#3
Hello [username] !
Today i will be posting a tutorial about information gathering.
Note : i found a similiar post by moroaddict but i believe that it missed some steps and the explanation is too short.
What is Information Gathering ?
Information Gathering is the process of gathering as more data about a system as possible. This is a fundamental (too many times underrated) and very first step in penetration testing.
Just keep in mind the following rule :
more information = higher chance to land a successful attack
Various Info
- All the methods i will present are suitable for every OS however I recommand using Backtrack as it comes with many features and scripts already installed
- Please notice that information gathering involves gathering info about a system, service enumeration and vulnerability identification are usually considered different steps, however i'll talk about Information Gathering like it included also Service Enumeration (most of the times they are considered part of the same process).
- Pentesting steps are the following :
information gathering -> service enumeration -> vulnerability identification -> penetration -> mantaining access -> removing traces
Information Gathering & Service Enumeration
I'll list and explain below each method, the best would be to gather data with each of them.
EDIT (thanks to @
chmod for this) : all the information gathering process, depending on the method can be active or passive. Active reconnissance involves sending a certain request directly to the system. Passive reconnissance, on the other side, involves gathering information from exteranal source (you don't send any request to the system).
- Active methods : url Listing, DNS, SMTP, SNMP, Netbios, Port Scanning
- Passive methods : WHOIS, Google, Email Harvesting, Netcraft
The methods below are purely for Information Gathering :
Whois
Spoiler:
Whois can be used either giving either domain or ip as argument :
- Reverse lookup involves giving the IP as parameter, usually you get different infos:
Code:
> whois <ip_address>
- Whois is the name of TCP protocol, used to query a central Whois database mantained by InterNIC.
- Each domain must have a whois contain all the contact info for the domain they host.
- A Whois query can return many useful information, to list some : domain, status, creation data, last update, expiratio date, registrant info, admin info, technical contact and nameservers. The result vary from query to query but this infos are likely to be always present.
- An interesting feature of the whois command is the technical and admin contacts which can be used to make some search around the web or for a SE attack.
- It is not hard to find IT admin that use their email address to register account in forums where they ask help about some configurations of security services (just think possible implications)
Google
Spoiler:
Google is the most powerful tool for information gathering. Learning how to use it, to use cached pages and advanced operators is a must (not only for information gathering). I'm not talking in depth about Google usage because that would require a lot of time (maybe in a dedicated post i will) but for those of you who are interested in learning more i signal 2 books:
- Google Hacking by Johnny Long
- Google Hacking for Penetration Testers, Vol. 2 (actually possible to find them for free using Google's fileype operator)
Google can be used for :
- Anonimity with Caches
- Directory Listing
- Directory Trasversal
- Configuration Files
- Log Digging
- Database Digging
- Data Mining in general
- Locate Vulnerable Websites
- Find Misconfigured Files
- and much more
Most used Google Advanced Operators :
- intitle
- allintext
- site
- fileype
- link
- cache
- inurl
- related
- and more
Google also offers ways to use its API but be careful while using them for non-totally-legit activity like spidering as Google might not like it.
Email Harvesting
Spoiler:
Email Harvesting is the process of getting as more emails as possible within the sytem organization. To achieve this task there are two specific tools that use Google to perform this search. These are available in Backtrack. (thanks @chomod)
Code:
> cd /pentest/enumeration/google
> ./goog-mail.py <domain>
Code:
> cd /pentest/enumeration/google
> ./theharvester.py -d <domain> -l 100 -b <data_source>
- It is interesting to notice that theharvester.py script lets you limit the data gathered with the -b option and choose the data source which can be either google, msn, linkedin etc
- Getting more email as possible can be useful for spreading a Trojan Horse for example (SE attack)
- Each email is worth checking as you might gather more info about the system
- Each email can be used to launch a Social Engineering (SE) attack
url Listing
Spoiler:
- A certain domain can have many subdomains worth checking as they might run on different servers (possible different vulnerabilities)
- For this purpose, a Python script named list-urls.py can be found in Backtrack.
Code:
> cd /pentest/enumeration/list-urls
> list-urls.py <domain>
- This will return a list of all the urls within a given domain.
- For large results it might be useful to create a simple bash script to extract the ip for each domain.
- I believe it should be trivial however here's the code :
Code:
#!/bin/bash
for hostname in $(cat url_list.txt); do
host $hostname | cut -d" " -f4
done
- Where url_list is a txt file where you saved list-urls.py output
Netcraft
Spoiler:
- Netcraft is a website that monitor various companies
- Provides different infos like : server uptimes, OS, Web Server version
- Can also be used to determine sub-domains
- http://searchdns.netcraft.com
- To provide an example :
Methods listed below are part of Service Enumeration :
DNS
Spoiler:
- DNS enumeration can be used to gather informations about public and private organization's servers like IP address
- Tools like nslookup and dig (giving a certain domain as parameter) are commonly used
- nslookup command will get you to a prompt, you can also set type=mx to get mail server or type=ns to get DNS
Code:
> nslookup
> type=mx or type=ns
> domain.com
- 3 main DNS enumeration techniques : forward lookup, reverse lookup and zone transfer
- Forward lookup :
The basic idea is to bruteforce possible names within a given domain and resolve them using the host command :
- This technique involves bruteforcing so it is a good idea to automate it with a script
- Try for example ftp.domain.com, ns.domain.com etc.
- Reverse lookup :
Same usage of the forward one but you actually give ip as paramenter (eg. the ones you gathered with url Listing for example)
- Zone transfer :
It is actually the method that usually provides more info compared to the others
It exploits a weakness of misconfigured DNS servers (often happens)
To try a zone transfer you have to issue the following command with the dns servers you already gathered in previous steps
Code:
> host -l ns dns_server_already_found
- Try this with all the dns servers you found, if the server is well configured you will be returned an error message, if on the other hand the transfer succedes you will find lots of useful informations
- Backtrack actually provides dnsenum.pl script that automatically performs all the steps of DNS enumeration and i strongly recommend this one as it is faster and reliable
Code:
> cd /pentest/enumeration/dnsenum
> ./dnsenum.pl
Usage: perl dnsenum.pl <DOMAINNAME> <dns.txt>
dnsenum #
- dns.txt is a file used to perform forward lookup
SMTP
Spoiler:
- Misconfigured email servers can be used to gather info about users within the system
- SMTP supports two commands : VRFY that checks wether a user belongs to the system, EXPN that asks the server for membership of a certain mailing list
Code:
> nc -v mail_server_ip 25
//used to connect to the mail server, i recommend using netcat but you can use what you like most
VRFY abcdef
550 5.1.1 abcdef... User unknown
VRFY lady_godiva
250 2.1.5 lady_godiva <lady_godiva@email.com>
- When you provide a correct username the system answer returning some infos and this can also be used for user enumeration (a script can be made)
SNMP
Spoiler:
- SNMP is a management protocol based on UDP, used to manage network devices remotely
- It is has a weak authentication system and actually provides really a lot of informations
- Authentication system is based upon community strings which can be public or private
- All these strings are not encrypted, hence eavesdropping is possible
- Different tools to enumerate SNMP : snmpwalk, snmpcheck.pl and snmpenum.pl (all of these can be found in Backtrack)
- If you have the occasion to try it yourself you will see that the amount of infos returned is overwhelming, so it must be filtered with proper parameters
- Just to see some examples :
Code:
> snmpwalk -c public -v1 <ip_address> 1 | grep hrSWRunName | cut -d" " -f4
//this will return the process list running on the system (Windows)
> snmpwalk -c public -v1 <ip_address> 1 | grep tcpConnState | cut -d"." -f6 | sort -nu
//this will return open Tcp ports
- Take time to examine snmpwalk command usage
- snmpcheck.pl and snmpenum.pl are easier to use
Code:
> cd pentest/enumeration/snmpenum
> ./snmpcheck-1.3.pl -t <ip_address>
Netbios
Spoiler:
- Netbios is a protocol that can be found in Windows systems
- In this last period care has been taken to secure this protocol however it is still possible to gather informations on some systems
- A so called "NULL Session" is an unauthenticated Netbios session between two computer. Some times this access is actually disabled, other times it is not and it is actually possible to gather informations such as Usernames, User and Host, machine data
- Backtrack provides Smb4k tool that helps identifying all machines running Netbios protocol within a system
- Also provides samrdump.py for username enumeration:
Code:
> cd pentest/enumeration/smb-enum
> ./samrdump.py <ip_address>
Port Scanning
Spoiler:
- Portscanning involves sending appropriate requests to the system in order to determine, trough the response received, open UDP and TCP ports on the target system
- Before explaining the different portscanning techniques, notice that this activity is likely to be detected as potentially malicious traffic on the target system, as it generated noticeable traffic on all the system ports within a very low time window. More over it is considered an illegal activity in many countries. Of course the solution is to use some quite-sophisticated port scanning technique that i'm going to talk about and run the portscanning activity trough proxychains or similar
- There exist many programs to perform portscanning, i recommend using nmap or Zenmap (nap's GUI)
- A little background on how port scanning work (understanding is always important):
TCP : TCP scanning involves an attempt to establish a 3 way handshake (i assume you know what it is) with the system. TCP RFC states that when a SYN is sent to an open port than the system should reply with an ACK that actually allows us to determine wether the port is open or close (an RST is sent if the port is closed)
UDP : being UDP a stateless protocol it is not actually possible to try the establishment of a 3 way handshake. UDP port scanning involves sending to the target port/s ICMP packets. Notice that this kind of scanning can actually be unreliable due to many system dropping ICMP (so there could be false positives and false negatives)
- For nmap detailed guide i found that Ex094 wrote a really good one (credits for this go to him) : http://www.hackcommunity.com/Thread-Tuto...nnaissance
- Note : before performing all these activities on and external server, beware of honeypots, for further explanation about honeypots surf the web or browse the forum, i've seen a good post about it
Metasploit
Spoiler:
- Among all its function (http://www.offensive-security.com/metasploit-unleashed/ for further reading (highly suggested)), the Metasploit framework can also serve Information Gathering purpose
- Backtrack is recommended to use Metasploit
- The reason i didn't write about Metasploit before the edit is that is mixes both Information Gathering, Service Enumeration and Vulnerability Scanning, hence someone who finds a vulnerability might immediatly skip to the exploitation phase : DO NOT. Separation each phase is very important and trust me if I tell you that altering the order is not a good idea
- All this is done from the msfconsole (just type msfconsole to run) which is a MSF interface that provides "all-in-one" console for all the tasks that MSF provides. Looks like it:
- In order to access the Infomation Gathering functions:
Code:
msf > use auxiliary/scanner
- Actually there are many scanners available in this module, take time to revise them :
Code:
msf > use auxiliary/scanner
Display all 237 possibilities? (y or n) //a lot as you can see
- Using MSF it is actually possible to gather informations concerning : open ports, mssql, service identificaiton, sniffing passwords
- I list here some useful command for msfconsole usage:
Code:
msf > search portscan
Matching Modules
================
Name Disclosure Date Rank Description
---- --------------- ---- -----------
auxiliary/scanner/natpmp/natpmp_portscan normal NAT-PMP External Port Scanner
auxiliary/scanner/portscan/ack normal TCP ACK Firewall Scanner
auxiliary/scanner/portscan/ftpbounce normal FTP Bounce Port Scanner
auxiliary/scanner/portscan/syn normal TCP SYN Port Scanner
auxiliary/scanner/portscan/tcp normal TCP Port Scanner
auxiliary/scanner/portscan/xmas normal TCP "XMas" Port Scanner
Code:
msf > use auxiliary/scanner/mssql/mssql_ping
msf auxiliary(mssql_ping) > show options
Module options (auxiliary/scanner/mssql/mssql_ping):
Name Current Setting Required Description
---- --------------- -------- -----------
PASSWORD no The password for the specified username
RHOSTS yes The target address range or CIDR identifier
THREADS 1 yes The number of concurrent threads
USERNAME sa no The username to authenticate as
USE_WINDOWS_AUTHENT false yes Use windows authentification
- Again for further reading (which is strongly reccommended if you plan to use the Metasploit framework) take a look at http://www.offensive-security.com/metasploit-unleashed
- Thanks @chmod for reminding me
- IMPORTANT : Take time to learn and revise how to use command in MSF, understanding how it works its really important to get good results and avoid messing things
Note : Try to keep methods for information gathering more separate as possible from Service Enumeration one.
Note : Most of the times, a lot of info is to be generated, my advice is to keep it well separated and organized as it will help you during the following steps of penetration testing (also to avoid repetitions of same results that you might have gathered with different methods).
•
Thirteen Years of Service
Posts: 3,253
Threads: 64
RE: Information Gathering 07-28-2013, 12:02 AM
#4
This is a VERY nice tutorial and a must read for beginners I do however have some suggestions (don't take this the wrong way, just trying to add a little constructive criticism:
Spoiler:
passive vs active
A little explanation of what constitutes as passive and active reconnaissance would really help out as both are covered here
Spoiler:
the harvester or metasploit
You covered using goo-mail.py to gather emails which is good but adding a few more methods can never hurt such as
theharvester.py - covers e-mails domain names DNS servers and other information
or
using metasploits use auxiliary/gather/search_email_collector - another great tool but could potentially yield different results, and as you said at the start the more information the better
Spoiler:
DNS spelling error:
Quote:Try this with all the dns servers you found, if the server is well configured you will be returned an erroro message, if on the other hand the transfer succedes you will find lots of useful informations
should be:
Quote:Try this with all the DNS servers you found, if the server is well configured you will be returned an error message, if on the other hand the transfer succeeds you will find lots of useful information
This ones a little picky I know and I apologize, but such a great thread demands attention to detail
Keep up the good work though, this is exactly what we are looking for.
•
Thirteen Years of Service
Posts: 340
Threads: 6
RE: Information Gathering 07-28-2013, 08:36 AM
#5
Oh yea about the last one i am correcting it immediately. Thanks for the other 2 suggestions i will be happy to add them as soon as possible (today, as i have time to do this). Thank for your feedback
•
Thirteen Years of Service
Posts: 2,880
Threads: 216
RE: Information Gathering 07-28-2013, 08:41 AM
#6
This is a very good tutorial and its a must for beginners. Good work. Its been long since a saw a good new member in the community and we have recently got many though and its a fact. Keep up the good work.
•
Thirteen Years of Service
Posts: 340
Threads: 6
RE: Information Gathering 07-28-2013, 10:11 AM
#7
Glad to hear it
i'm happy i found a good community with skilled people, i'll always do my best
EDIT: @
chmod i've added theharvester.py, Information Gathering with MSF and corrected the typo, if some other method comes to my mind i'll update again the post
•
Thirteen Years of Service
Posts: 3,253
Threads: 64
RE: Information Gathering 07-28-2013, 07:04 PM
#8
Awesome looks great, it really shows that you have put a lot of time, effort and thought into this keep it up!
•
Thirteen Years of Service
Posts: 340
Threads: 6
RE: Information Gathering 07-28-2013, 07:40 PM
#9
Ye actually it took me quite a bit, if it will be useful even to a single people than it won't be wasted time!
•
Thirteen Years of Service
Posts: 1,232
Threads: 66
RE: Information Gathering 07-29-2013, 02:22 PM
#10
Cool.
Good to see such a nice and complete thread.
A very good beginning for penetration testing.
![[Image: 2YpkRjy.png]](http://i.imgur.com/2YpkRjy.png)
PM me if you need help.
My pastebin
HERE. My URL Shortener
HERE.
•
Users browsing this thread:
![[Image: OilyCostlyEwe.gif]](http://fat.gfycat.com/OilyCostlyEwe.gif)
