Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to get Schema Registry working in HDF 3.0

Highlighted

Unable to get Schema Registry working in HDF 3.0

New Contributor

Hi,

I have a problem with schema registry which is a new service in HDF 3.0. I followed the installation and configuration manual https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.0/bk_installing-hdf-and-hdp/content/configure...

After I finished configuration the Registry service started successfully but then immediately was shut down again. In error log file (/var/log/registry/registry.err) I have only following information

log4j:WARN No appenders could be found for logger (org.jboss.logging).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.util.StringUtils.toLowerCase(Ljava/lang/String;)Ljava/lang/String;
        at org.apache.hadoop.hdfs.server.common.HdfsServerConstants$RollingUpgradeStartupOption.getAllOptionString(HdfsServerConstants.java:80)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<clinit>(NameNode.java:263)
        at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
        at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2625)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2607)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
        at com.hortonworks.registries.common.util.HdfsFileStorage.init(HdfsFileStorage.java:62)
        at com.hortonworks.registries.webservice.RegistryApplication.getJarStorage(RegistryApplication.java:188)
        at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:133)
        at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:68)
        at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:58)
        at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
        at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)
        at io.dropwizard.cli.Cli.run(Cli.java:75)
        at io.dropwizard.Application.run(Application.java:79)
        at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:226)

Is this new service not packaged correctly into HDF 3.0? Could anybody give me a hint what should I do to get schema registry up & running?

Regards,

Andres

4 REPLIES 4

Re: Unable to get Schema Registry working in HDF 3.0

Super Mentor

@Andres Koitmäe

The following error indicates that you might have multiple version of "hadoop-common-xxxx.jar" in your environment. Can you please find and grep all the JARs with this name.

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.util.StringUtils.toLowerCase(Ljava/lang/String;)Ljava/lang/String;

.

Presence of multiple versions of this JARs can cause "NoSuchMethodError" syntax as the signature of the mentioned method seems to have changed with version.

Re: Unable to get Schema Registry working in HDF 3.0

New Contributor

@Jay SenSharma

We do have several hadoop-common-xxx.jar files in our environment. On the machine where Schema Registry is installed (it is our internal development/sandbox) there are also HDP 2.6.1 components installed (this machine is namenode in HDP cluster).

Here are all files which match hadoop-common* pattern on that machine

/var/lib/ambari-agent/cred/lib/hadoop-common-2.7.3.jar
/usr/hdf/3.0.0.0-453/registry/libs/hadoop-common-2.5.1.jar
/usr/hdp/2.6.0.3-8/hadoop/hadoop-common.jar
/usr/hdp/2.6.0.3-8/hadoop/client/hadoop-common.jar
/usr/hdp/2.6.0.3-8/hadoop/client/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/hadoop/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/hadoop/hadoop-common-tests.jar
/usr/hdp/2.6.0.3-8/hadoop/hadoop-common-2.7.3.2.6.0.3-8-tests.jar
/usr/hdp/2.6.0.3-8/phoenix/lib/hadoop-common.jar
/usr/hdp/2.6.0.3-8/ranger-hbase-plugin/install/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-usersync/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-hdfs-plugin/install/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-hive-plugin/install/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/livy2/jars/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-admin/cred/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-admin/ews/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-admin/ews/webapp/WEB-INF/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/slider/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-kafka-plugin/install/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-kafka-plugin/lib/ranger-kafka-plugin-impl/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/druid/extensions/druid-hdfs-storage/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/druid/extensions/druid-kerberos/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/druid/hadoop-dependencies/hadoop-client/2.7.3.2.6.0.3-8/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/spark2/jars/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/kafka/libs/ranger-kafka-plugin-impl/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/atlas/server/webapp/atlas/WEB-INF/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-atlas-plugin/install/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-tagsync/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.0.3-8/ranger-yarn-plugin/install/lib/hadoop-common-2.7.3.2.6.0.3-8.jar
/usr/hdp/2.6.1.0-129/hadoop/hadoop-common.jar
/usr/hdp/2.6.1.0-129/hadoop/hadoop-common-2.7.3.2.6.1.0-129-tests.jar
/usr/hdp/2.6.1.0-129/hadoop/client/hadoop-common.jar
/usr/hdp/2.6.1.0-129/hadoop/client/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/hadoop/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/hadoop/hadoop-common-tests.jar
/usr/hdp/2.6.1.0-129/phoenix/lib/hadoop-common.jar
/usr/hdp/2.6.1.0-129/ranger-hbase-plugin/install/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-usersync/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-hdfs-plugin/install/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-hive-plugin/install/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/livy2/jars/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-admin/cred/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-admin/ews/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/slider/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-kafka-plugin/install/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-kafka-plugin/lib/ranger-kafka-plugin-impl/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/druid/extensions/druid-hdfs-storage/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/druid/extensions/druid-kerberos/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/druid/hadoop-dependencies/hadoop-client/2.7.3.2.6.1.0-129/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/spark2/jars/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/kafka/libs/ranger-kafka-plugin-impl/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/atlas/server/webapp/atlas/WEB-INF/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-atlas-plugin/install/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-tagsync/lib/hadoop-common-2.7.3.2.6.1.0-129.jar
/usr/hdp/2.6.1.0-129/ranger-yarn-plugin/install/lib/hadoop-common-2.7.3.2.6.1.0-129.jar

Is this kind of situation ok or should we have totally separate machine for SAM modules as it's is described here https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.0/bk_planning-your-deployment/content/ch_depl... ?

Registry service started to work when Jar Storage Type was changed from HDFS to Local. Why it works with Local on not with HDFS is still puzzle for me.

Re: Unable to get Schema Registry working in HDF 3.0

New Contributor

In my installation there was:

/usr/hdf/3.0.1.0-43/registry/libs/hadoop-common-2.5.1.jar

I deleted it and copied hadoop-common-2.7.3.jar inside the directory. It worked

Re: Unable to get Schema Registry working in HDF 3.0

New Contributor

Hello,

You will find the same behaviour in HDF 3.0.1.1-5:

ll /usr/hdf/3.0.1.1-5/registry/libs/ | grep hadoop-common hadoop-common-2.5.1.jar
So, following answer by Antonio,
rm /usr/hdf/3.0.1.1-5/registry/libs/hadoop-common-2.5.1.jar

wget http://central.maven.org/maven2/org/apache/hadoop/hadoop-common/2.7.4/hadoop-common-2.7.4.jar -O /usr/hdf/3.0.1.1-5/registry/libs/hadoop-common-2.7.4.jar

It works,

Don't have an account?
Coming from Hortonworks? Activate your account here