Snakebite is a pure python based HDFS client which is much faster than the classic hdfs client operations(hadoop fs / hdfs dfs).Snakebite is developed by Spotify. More details can be found at
I am using HDP 2.4 and Ambari 220.127.116.11 on CentOS/
1). Install Python Pip.
yum install python-pip
pip install snakebite
Now snakebite is installed.
You can run below sample commands.
snakebite ls /tmp
snakebite ls /
Use snakebite help for more options.
Does it support Kerberos by now? Would be very nice to be able to use it instead of os.system("hadoop ... ") commands as I currently do.
@Benjamin Leonhardi. Yes. you need to do
pip install "snakebite[kerberos]"
I haven't installed on kerberized cluster. Will try and let you know.
cool thanks, that should speed things up.