Using the Hadoop Attack Library to Check Your Hadoop Cluster

[#1]

You can run the attack library for OSX or Linux from an edge node or from outside the cluster.

I ran against mine from my OSX laptop against my cluster that I had network access to.

You should try to scan from inside your network, from an edge node and from a remote site on the Internet.

You will need Python 2.7 or Python 3.x installed first.

git clone git@github.com:CERT-W/hadoop-attack-library.git 
pip install requests lxml

You may need root or sudo access to install on your machine.

One of the scanners hits the WebHDFS link that you may have seen a warning about.

python hdfsbrowser.py timscluster 

Beginning to test services accessibility using default ports ...
Testing service WebHDFS
[+] Service WebHDFS is available
Testing service HttpFS
[-] Exception during requesting the service
[+] Sucessfully retrieved 1 services
drwxrwxrwx  hdfs:hdfs  2017-01-15T05:50:27+0000    /
drwxrwxrwx  yarn:hadoop  2017-01-11T19:25:26+0000  app-logs  /app-logs
drwxrwxrwx  hdfs:hdfs  2016-12-21T23:12:40+0000  apps  /apps
drwxrwxrwx  yarn:hadoop  2016-09-15T21:02:30+0000  ats  /ats
drwxrwxrwx  root:hdfs  2016-12-21T23:08:34+0000  avroresults  /avroresults
drwxrwxrwx  hdfs:hdfs  2016-12-13T03:42:55+0000  banking  /banking

To see how available your Hadoop configurations are available, you can use Hadoop Snooper. This is under: Tools\ Techniques\ and\ Procedures \ Getting\ the\ target\ environment\ configuration

python hadoopsnooper.py timscluster -o test
Specified destination path does not exist, do you want to create it ? [y/N]y
[+] Creating configuration directory
[+] core-site.xml successfully created
[+] mapred-site.xml successfully created
[+] yarn-site.xml successfully created

This downloads all those configuration files to a directory named test.

These were not the full configuration files, but they pointed to correct internal servers and give an attacker more information.

Another scan worth running is sqlmap. This tool will let you check various SQL tools in the system. SQLMap requires Python 2.6 or 2.7.

➜  projects git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
Cloning into 'sqlmap-dev'...
remote: Counting objects: 55560, done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 55560 (delta 22), reused 0 (delta 0), pack-reused 55519
Receiving objects: 100% (55560/55560), 47.25 MiB | 2.28 MiB/s, done.
Resolving deltas: 100% (42960/42960), done.
Checking connectivity... done.
➜  projects python sqlmap.py --update
➜  projects cd sqlmap-dev
➜  sqlmap-dev git:(master) python sqlmap.py --update
        ___
       __H__
 ___ ___[.]_____ ___ ___  {1.1.1.14#dev}
|_ -| . [)]     | .'| . |
|___|_  [']_|_|_|__,|  _|
      |_|V          |_|   http://sqlmap.org

[!] legal disclaimer: Usage of sqlmap for attacking targets without prior mutual consent is illegal. It is the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program

[*] starting at 16:49:13
[16:49:13] [INFO] updating sqlmap to the latest development version from the GitHub repository
[16:49:13] [INFO] update in progress .
[16:49:14] [INFO] already at the latest revision 'f542e82'
[*] shutting down at 16:49:14

References:

• http://sqlmap.org/
• http://www.slideshare.net/bunkertor/hadoop-security-54483815
• http://tools.kali.org/
• https://github.com/savio-code/hexorbase
• https://community.hortonworks.com/articles/73035/running-dns-and-domain-scanning-tools-from-apache.h...