Error starting SmartSense (HDP 2.6.4)

[#1]

Hi there,

I managed to install the HDP 2.6.4 plaftorm on a cluster, the smartsense service is failing to start though. Following is the exception stacktrace from the /var/log/smartsense-activity/activity-analyzer.out file:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
        at com.hortonworks.smartsense.activity.util.ActivityUtil.getHadoopConfig(ActivityUtil.java:425)
        at com.hortonworks.smartsense.activity.util.ActivityUtil.login(ActivityUtil.java:434)
        at com.hortonworks.smartsense.activity.ActivityAnalyzerFacade.init(ActivityAnalyzerFacade.java:193)
        at com.hortonworks.smartsense.activity.ActivityAnalyzerFacade.main(ActivityAnalyzerFacade.java:95)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 4 more

[#2]

@Alberto Rodriguez

Can you please check if the following properties are set properly inside the below mentioned script?

# grep 'HADOOP_' /etc/smartsense-activity/conf/activity-env.sh 
export HADOOP_HOME=/usr/hdp/current/hadoop-client
export HADOOP_CONF_DIR=/etc/hadoop/conf/

.

Also please chekc if the HDFS client is installed on the mentioned host where 'activity-analyzer' is installed?

Also if there is any incorrect Environment variable set globally which is pointing to incorrect HADOOP_CONF_DIR or HADOOP_HOME ?

[#3]

@Alberto Rodriguez

I just tested SmartSense on HDP 2.6.4 and it worked well. Then i checked the JARs that it is loading which contains the missing class then i found the following:

# lsof -p `cat /var/run/smartsense-activity-analyzer/activity-analyzer.pid` | grep 'hadoop-common'
java    19274 root  mem       REG              253,1   3668459  8377041 /usr/hdp/2.6.4.0-91/hadoop/hadoop-common-2.7.3.2.6.4.0-91.jar
java    19274 root  110r      REG              253,1   3668459  8377041 /usr/hdp/2.6.4.0-91/hadoop/hadoop-common-2.7.3.2.6.4.0-91.jar

.

So can you please check if you have HDFS clients installed on this host? And if the mentioned JAR exist Or has proper read permission ?

[#4]

@Alberto Rodriguez

Good to see that the issue is resolved after installing HDFS services.

As the issue is resolved, hence it will be also great if you can mark this HCC thread as Answered by clicking on the "Accept" Button on the correct answer. That way other HCC users can quickly find the solution when they encounter the same issue.

[#5]

Hi @Jay Kumar SenSharma, thank you for replying.

I haven't installed HDFS in the cluster yet so I guess that's the problem. I'll install HDFS shortly and get back to you.

Anyway I think the smartsense service shouldn't be automatically installed unless you've got HDFS already deployed in the cluster, don't you think?

[#6 · Accepted solution]

Hi @Jay Kumar SenSharma, after installing HDFS in the cluster, the SmartSense service was installed without any problems.

Thank you again!