hadoop-lzo jar file is missing for spark service during the cluster setup
rpm -qa| grep hadoop hadoop_XXX-mapreduce-XXX.x86_64 hadoop_XXX-client-XXX.el6.x86_64 hadoop_XXX-libhdfs-XXX.el6.x86_64 hadoop_XXX-yarn-XXX.el6.x86_64 hadoop_XXX-hdfs-XXX.el6.x86_64 ambari-metrics-hadoop-sink-XXX.x86_64
How to enable it during deployment?
Which version of HDP & Spark are you using ? You might try the following as an alternative after installing lzo.
spark-shell --jars /usr/hdp/current/share/lzo/0.6.0/lib/hadoop-lzo-0.6.0.jar
Also can you please check:
yum install lzo lzo-devel hadooplzo hadooplzo-native
I see it in HDP repo:
# yum info hadooplzo Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Available Packages Name : hadooplzo Arch : noarch Version : 0.6.0.2.5.3.0 Release : 37.el6 Size : 2.5 k Repo : HDP-188.8.131.52-37 Summary : hadooplzo Distro virtual package License : APL2 Description : hadooplzo-0.6.0.2.5.3.0 virtual package