Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hadoop-lzo jar file is missing for spark service

hadoop-lzo jar file is missing for spark service

New Contributor

hadoop-lzo jar file is missing for spark service during the cluster setup

rpm -qa| grep hadoop
hadoop_XXX-mapreduce-XXX.x86_64
hadoop_XXX-client-XXX.el6.x86_64
hadoop_XXX-libhdfs-XXX.el6.x86_64
hadoop_XXX-yarn-XXX.el6.x86_64
hadoop_XXX-hdfs-XXX.el6.x86_64
ambari-metrics-hadoop-sink-XXX.x86_64 

How to enable it during deployment?

2 REPLIES 2

Re: hadoop-lzo jar file is missing for spark service

Super Mentor

@zhixun he

Which version of HDP & Spark are you using ? You might try the following as an alternative after installing lzo.

spark-shell --jars /usr/hdp/current/share/lzo/0.6.0/lib/hadoop-lzo-0.6.0.jar 

.

Also can you please check:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_installing_manually_book/content/install...

yum install lzo lzo-devel hadooplzo hadooplzo-native

.

I see it in HDP repo:

# yum info hadooplzo

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Available Packages
Name        : hadooplzo
Arch        : noarch
Version     : 0.6.0.2.5.3.0
Release     : 37.el6
Size        : 2.5 k
Repo        : HDP-2.5.3.0-37
Summary     : hadooplzo Distro virtual package
License     : APL2
Description : hadooplzo-0.6.0.2.5.3.0 virtual package

.

Re: hadoop-lzo jar file is missing for spark service

Expert Contributor
Don't have an account?
Coming from Hortonworks? Activate your account here