Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 12-15-2015 06:31 AM
Hi,
I think I am missing some dependency in my pom.xml to get my kerberose realm picked up correctly via rest webservice call . The code works fine and i get results when I deploy simple java project and run jar file directly from the server . But I get below error when I try to make rest service call from the Linux server . The code works fine on my laptop which is windows laptop .
I verified that I have below config on my hbase box :
$ hadoop version Hadoop 2.6.0.2.2.4.2-2 Subversion [email protected]:hortonworks/hadoop.git -r 22a563ebe448969d07902aed869ac13c652b2872 Compiled by jenkins on 2015-03-31T19:49Z Compiled with protoc 2.5.0 From source with checksum b3481c2cdbe2d181f2621331926e267 This command was run using /usr/hdp/2.2.4.2-2/hadoop/hadoop-common-2.6.0.2.2.4.2-2.jar $ hadoop-client version -ksh: hadoop-client: not found [No such file or directory] $ hbase version 2015-12-14 15:18:10,348 INFO [main] util.VersionInfo: HBase 0.98.4.2.2.4.2-2-hadoop2 2015-12-14 15:18:10,348 INFO [main] util.VersionInfo: Subversion git://ip-10-0-0-5.ec2.internal/grid/0/jenkins/workspace/HDP-2.2.4.1-centos6/bigtop/build/hbase/rpm/BUILD/hbase-0.98.4.2.2.4.2 -r dd8a499345afc1ac49dc5ef212ba64b23abfe110
Error I am getting :
at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:84) at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63)
Caused by: KrbException: Generic error (description in e-text) (60) - Unable to locate Kerberos realm at sun.security.krb5.Config.getRealmFromDNS(Config.java:1102) at sun.security.krb5.Config.getDefaultRealm(Config.java:987)
My pom.xml :
<projectxmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>msoa.hbaseSampleOlderVersion</groupId>
<artifactId>hbaseBeanType</artifactId>
<packaging>war</packaging>
<version>0.0.1-SNAPSHOT</version>
<name>hbaseBeanType</name>
<build>
<finalName>hbaseBeanType</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.5.1</version>
<inherited>true</inherited>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.glassfish.jersey</groupId>
<artifactId>jersey-bom</artifactId>
<version>${jersey.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<!-- not req for local -->
<repositories>
<repository>
<releases>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
<checksumPolicy>warn</checksumPolicy>
</releases>
<snapshots>
<enabled>false</enabled>
<updatePolicy>never</updatePolicy>
<checksumPolicy>fail</checksumPolicy>
</snapshots>
<id>HDPReleases</id>
<name>HDP Releases</name>
<url>http://repo.hortonworks.com/content/repositories/releases/</url>
<layout>default</layout>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet-core</artifactId>
<!-- use the following artifactId if you don't need servlet 2.x compatibility
<artifactId>jersey-container-servlet</artifactId>-->
</dependency>
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-moxy</artifactId>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-bundle-jaxrs</artifactId>
<version>2.7.7</version>
</dependency>
<!-- HBASE DEPENDENCY -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<!-- not req for local -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0</version>
</dependency>
<!-- not req for local -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>0.98.0-hadoop2</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-common</artifactId>
<version>0.98.0-hadoop2</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-protocol</artifactId>
<version>0.98.0-hadoop2</version>
</dependency>
<dependency>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
<scope>system</scope>
<version>1.8.0_60</version>
<systemPath>C:\Program Files\Java\jdk1.8.0_60\lib\tools.jar</systemPath>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>4.2.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>4.2.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
<version>4.2.3.RELEASE</version>
</dependency>
</dependencies>
<properties>
<!-- <jersey.version>1.1</jersey.version> -->
<jersey.version>2.16</jersey.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<hadoop.version>2.6.0.2.2.0.0-2041</hadoop.version>
</properties>
</project>
Created 12-16-2015 07:00 PM
So the main problem was that I was using HTTP Rest webservice (running on linux server ) to call hbase . Since hbase is kerberozied it didn't consider any request from HTTP as its unsecured . I had to change server.xml (tomcat ) on my linux server to allow HTTPS request and open port 8443 . Restart the tomcat .
Once I followed above steps my Webservice Rest requests were going through .
References :
https://steveloughran.gitbooks.io/kerberos_and_had... and
Created 12-15-2015 11:47 AM
It looks like your application doesn't know how to get a kerberos ticket. You need to configure the krb5.conf on the Linux server that you are running from.
Created 12-15-2015 11:38 PM
I have krb5.conf set in my properties and i have verified it exists on my server at below location:
in my property file :
krb.principal=hdpsrvc@HDP_EIT_DEV.com krb.config=/opt/app/apache-tomcat-8.0.15/conf/krb5.conf keytab.path=/opt/app/apache-tomcat-8.0.15/conf/hdpsrvc.keytab
and i verified that its being picked up. by below code :
kerberosUser = props.getProperty("krb.principal"); kerberosKeytab = props.getProperty("keytab.path"); kerberoseConfig = props.getProperty("krb.config"); zookeeperQuorum = props.getProperty("zookeeperQuorum");
System.out.println("krb : principal :"+ kerberoseConfig);
Hbase Code that I use to set kerberose:
System.setProperty("javax.security.auth.useSubjectCredsOnly", "true"); System.setProperty("java.security.krb5.conf", kerberoseConfig);
org.apache.hadoop.conf.Configuration config = new org.apache.hadoop.conf.Configuration(); config.set("hadoop.security.authentication", "kerberos"); UserGroupInformation.setConfiguration(config); --- my code dies here and gives above error .
Created 12-16-2015 10:35 AM
You can find an example on how to write to an HBase kerberised cluster at the following URL.
Created 12-16-2015 07:00 PM
So the main problem was that I was using HTTP Rest webservice (running on linux server ) to call hbase . Since hbase is kerberozied it didn't consider any request from HTTP as its unsecured . I had to change server.xml (tomcat ) on my linux server to allow HTTPS request and open port 8443 . Restart the tomcat .
Once I followed above steps my Webservice Rest requests were going through .
References :
https://steveloughran.gitbooks.io/kerberos_and_had... and
Created 02-02-2016 05:51 PM
@Varsha B has this been resolved? Please accept best answer or provide your own solution.