Created 12-15-2015 06:31 AM
Hi,
I think I am missing some dependency in my pom.xml to get my kerberose realm picked up correctly via rest webservice call . The code works fine and i get results when I deploy simple java project and run jar file directly from the server . But I get below error when I try to make rest service call from the Linux server . The code works fine on my laptop which is windows laptop .
I verified that I have below config on my hbase box :
$ hadoop version Hadoop 2.6.0.2.2.4.2-2 Subversion git@github.com:hortonworks/hadoop.git -r 22a563ebe448969d07902aed869ac13c652b2872 Compiled by jenkins on 2015-03-31T19:49Z Compiled with protoc 2.5.0 From source with checksum b3481c2cdbe2d181f2621331926e267 This command was run using /usr/hdp/2.2.4.2-2/hadoop/hadoop-common-2.6.0.2.2.4.2-2.jar $ hadoop-client version -ksh: hadoop-client: not found [No such file or directory] $ hbase version 2015-12-14 15:18:10,348 INFO [main] util.VersionInfo: HBase 0.98.4.2.2.4.2-2-hadoop2 2015-12-14 15:18:10,348 INFO [main] util.VersionInfo: Subversion git://ip-10-0-0-5.ec2.internal/grid/0/jenkins/workspace/HDP-2.2.4.1-centos6/bigtop/build/hbase/rpm/BUILD/hbase-0.98.4.2.2.4.2 -r dd8a499345afc1ac49dc5ef212ba64b23abfe110
Error I am getting :
at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:84) at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63)
Caused by: KrbException: Generic error (description in e-text) (60) - Unable to locate Kerberos realm at sun.security.krb5.Config.getRealmFromDNS(Config.java:1102) at sun.security.krb5.Config.getDefaultRealm(Config.java:987)
My pom.xml :
<projectxmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>msoa.hbaseSampleOlderVersion</groupId>
<artifactId>hbaseBeanType</artifactId>
<packaging>war</packaging>
<version>0.0.1-SNAPSHOT</version>
<name>hbaseBeanType</name>
<build>
<finalName>hbaseBeanType</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.5.1</version>
<inherited>true</inherited>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.glassfish.jersey</groupId>
<artifactId>jersey-bom</artifactId>
<version>${jersey.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<!-- not req for local -->
<repositories>
<repository>
<releases>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
<checksumPolicy>warn</checksumPolicy>
</releases>
<snapshots>
<enabled>false</enabled>
<updatePolicy>never</updatePolicy>
<checksumPolicy>fail</checksumPolicy>
</snapshots>
<id>HDPReleases</id>
<name>HDP Releases</name>
<url>http://repo.hortonworks.com/content/repositories/releases/</url>
<layout>default</layout>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet-core</artifactId>
<!-- use the following artifactId if you don't need servlet 2.x compatibility
<artifactId>jersey-container-servlet</artifactId>-->
</dependency>
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-moxy</artifactId>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-bundle-jaxrs</artifactId>
<version>2.7.7</version>
</dependency>
<!-- HBASE DEPENDENCY -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<!-- not req for local -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0</version>
</dependency>
<!-- not req for local -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>0.98.0-hadoop2</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-common</artifactId>
<version>0.98.0-hadoop2</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-protocol</artifactId>
<version>0.98.0-hadoop2</version>
</dependency>
<dependency>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
<scope>system</scope>
<version>1.8.0_60</version>
<systemPath>C:\Program Files\Java\jdk1.8.0_60\lib\tools.jar</systemPath>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>4.2.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>4.2.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
<version>4.2.3.RELEASE</version>
</dependency>
</dependencies>
<properties>
<!-- <jersey.version>1.1</jersey.version> -->
<jersey.version>2.16</jersey.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<hadoop.version>2.6.0.2.2.0.0-2041</hadoop.version>
</properties>
</project>
Created 12-16-2015 07:00 PM
So the main problem was that I was using HTTP Rest webservice (running on linux server ) to call hbase . Since hbase is kerberozied it didn't consider any request from HTTP as its unsecured . I had to change server.xml (tomcat ) on my linux server to allow HTTPS request and open port 8443 . Restart the tomcat .
Once I followed above steps my Webservice Rest requests were going through .
References :
https://steveloughran.gitbooks.io/kerberos_and_had... and
Created 12-15-2015 11:47 AM
It looks like your application doesn't know how to get a kerberos ticket. You need to configure the krb5.conf on the Linux server that you are running from.
Created 12-15-2015 11:38 PM
I have krb5.conf set in my properties and i have verified it exists on my server at below location:
in my property file :
krb.principal=hdpsrvc@HDP_EIT_DEV.com krb.config=/opt/app/apache-tomcat-8.0.15/conf/krb5.conf keytab.path=/opt/app/apache-tomcat-8.0.15/conf/hdpsrvc.keytab
and i verified that its being picked up. by below code :
kerberosUser = props.getProperty("krb.principal"); kerberosKeytab = props.getProperty("keytab.path"); kerberoseConfig = props.getProperty("krb.config"); zookeeperQuorum = props.getProperty("zookeeperQuorum");
System.out.println("krb : principal :"+ kerberoseConfig);
Hbase Code that I use to set kerberose:
System.setProperty("javax.security.auth.useSubjectCredsOnly", "true"); System.setProperty("java.security.krb5.conf", kerberoseConfig);
org.apache.hadoop.conf.Configuration config = new org.apache.hadoop.conf.Configuration(); config.set("hadoop.security.authentication", "kerberos"); UserGroupInformation.setConfiguration(config); --- my code dies here and gives above error .
Created 12-16-2015 10:35 AM
You can find an example on how to write to an HBase kerberised cluster at the following URL.
Created 12-16-2015 07:00 PM
So the main problem was that I was using HTTP Rest webservice (running on linux server ) to call hbase . Since hbase is kerberozied it didn't consider any request from HTTP as its unsecured . I had to change server.xml (tomcat ) on my linux server to allow HTTPS request and open port 8443 . Restart the tomcat .
Once I followed above steps my Webservice Rest requests were going through .
References :
https://steveloughran.gitbooks.io/kerberos_and_had... and
Created 02-02-2016 05:51 PM
@Varsha B has this been resolved? Please accept best answer or provide your own solution.