question Re: How to Make Kylin work with HDP 2.3 in Support Questions

How to Make Kylin work with HDP 2.3

amcbarnett — Mon, 12 Oct 2015 23:59:53 GMT

Here is the issue with Kylin (http://kylin.incubator.apache.org/). For now it only supports HDP 2.2.x In HDP 2.3, Hbase has upgraded to 1.0 and APIS have changed. See https://issues.apache.org/jira/browse/KYLIN-892 and https://issues.apache.org/jira/browse/KYLIN-920

Kyle did not keep up to date with the changes and have not migrated to HBase v1.0 APIs. https://issues.apache.org/jira/browse/KYLIN-782

Re: How to Make Kylin work with HDP 2.3

amcbarnett — Tue, 13 Oct 2015 00:00:11 GMT

The github branch that actually works with HDP 2.3 is 1.x-HBase1.xhttps://github.com/apache/incubator-kylin/tree/1.x-HBase1.x

Use the following instructions to build it http://kylin.incubator.apache.org/development/howto_package.html

How to Build Binary Package

###Generate Binary Package

> git clone https://github.com/apache/incubator-kylin kylin

> cd kylin

> ./script/package.sh

In order to generate binary package, maven and npm are pre-requisites.

Re: How to Make Kylin work with HDP 2.3

nsabharwal — Wed, 21 Oct 2015 00:58:42 GMT

@amcbarnett@hortonworks.com

[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check (default) on project kylin-storage: Too many files with unapproved license: 1 See RAT report in: /root/kylin/storage/target/rat.txt -> [Help 1] [ERROR]

I followed the above steps and hitting this issue

Re: How to Make Kylin work with HDP 2.3

abajwa — Wed, 21 Oct 2015 01:19:55 GMT

Just delete the files mentioned in /root/kylin/storage/target/rat.txt

Usually they are log files or files user created/copied

Re: How to Make Kylin work with HDP 2.3

nsabharwal — Wed, 21 Oct 2015 17:22:23 GMT

@abjawa@hortonworks.com

That helped. I had to delete this files.

rm -f /root/kylin/storage/src/main/java/org/apache/kylin/storage/hbase/coprocessor/endpoint/protobuf/II.proto


Unapproved licenses:


rm -f  /root/kylin/query/src/test/resources/query/sql_verifyCount/query04.sql.expected
rm -f   /root/kylin/query/src/test/resources/query/sql_verifyCount/query07.sql.expected
rm -f   /root/kylin/query/src/test/resources/query/sql_verifyCount/query08.sql.expected
 rm -f  /root/kylin/query/src/test/resources/query/sql_verifyCount/query06.sql.expected
  rm -f  /root/kylin/query/src/test/resources/query/sql_verifyCount/query05.sql.expected

Re: How to Make Kylin work with HDP 2.3

nsabharwal — Wed, 21 Oct 2015 17:44:43 GMT

@amcbarnett@hortonworks.com

Did you test that in your environment?

Re: How to Make Kylin work with HDP 2.3

nsabharwal — Wed, 21 Oct 2015 18:17:51 GMT

I have reached out to Kylin Dev team to comment on this.

I am working on setting up Kylin with HDP 2.3.2. Package.sh failed because of unapproved licensees messages in rat.txt
I deleted following and package.sh moved forward.
rm -f /root/kylin/storage/src/main/java/org/apache/kylin/storage/hbase/coprocessor/endpoint/protobuf/II.proto
Unapproved licenses:
rm -f  /root/kylin/query/src/test/resources/query/sql_verifyCount/query04.sql.expected
rm -f   /root/kylin/query/src/test/resources/query/sql_verifyCount/query07.sql.expected
rm -f   /root/kylin/query/src/test/resources/query/sql_verifyCount/query08.sql.expected
 rm -f  /root/kylin/query/src/test/resources/query/sql_verifyCount/query06.sql.expected
  rm -f  /root/kylin/query/src/test/resources/query/sql_verifyCount/query05.sql.expected
Issues
<strong>Permission
</strong>[root@nsfed01 bin]# ls -l
total 40
-rw-r--r--. 1 root hadoop 1604 Oct 21 03:46 check-env.sh
-rw-r--r--. 1 root hadoop 1300 Oct 21 03:46 find-hbase-dependency.sh
-rw-r--r--. 1 root hadoop 1785 Oct 21 03:46 find-hive-dependency.sh
-rw-r--r--. 1 root hadoop 1023 Oct 21 03:46 get-properties.sh
-rw-r--r--. 1 root hadoop 1638 Oct 21 03:46 health-check.sh
-rw-r--r--. 1 root hadoop 3881 Oct 21 03:46 kylin.sh
-rwxr-xr-x. 1 root hadoop 2321 Oct 21 03:46 metastore.sh
-rw-r--r--. 1 root hadoop 1395 Oct 21 03:46 performance-monitor.sh
-rw-r--r--. 1 root hadoop 1456 Oct 21 03:46 sample.sh
-rwxr-xr-x. 1 root hadoop 2487 Oct 21 03:46 setenv.sh
[root@nsfed01 bin]# 
[root@nsfed01 kylin]# bin/kylin.sh start
KYLIN_HOME is set to /root/kylin/
WARNING: Use "yarn jar" to launch YARN applications.
Logging initialized using configuration in file:/etc/hive/2.3.2.0-2950/0/hive-log4j.properties
Please visit <a href="http://%3Cyour_sandbox_ip%3E:7070/kylin">http://<your_sandbox_ip>:7070/kylin</a> to play with the cubes! (Useranme: ADMIN, Password: KYLIN)
You can check the log at bin/../tomcat/logs/kylin.log
[root@nsfed01 kylin]# bin/kylin.sh: line 62: bin/../tomcat/logs/kylin.log: No such file or directory
mkdir -p tomcat/logs
[root@nsfed01 tomcat]# 
[root@nsfed01 logs]# ls
kylin.log
[root@nsfed01 logs]# cat kylin.log 
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128M; support was removed in 8.0
Not a valid JAR: /root/kylin/tomcat/bin/bootstrap.jar
[root@nsfed01 logs]# 
[root@nsfed01 logs]# ls -l /root/kylin/tomcat/bin/bootstrap.jar
ls: cannot access /root/kylin/tomcat/bin/bootstrap.jar: No such file or directory
[root@nsfed01 logs]#

Re: How to Make Kylin work with HDP 2.3

amcbarnett — Fri, 13 Nov 2015 04:16:51 GMT

@Neeraj I thought I got it to work. It does not. Did the Kylin Dev Team provide an answer?

Re: How to Make Kylin work with HDP 2.3

nsabharwal — Mon, 16 Nov 2015 05:40:35 GMT

@Ancil McBarnett

No response from them. I tried again in the sandbox

You can check the log at bin/../tomcat/logs/kylin.log

[root@sandbox kylin]# tail -f tomcat/logs/kylin.log

Not a valid JAR: /root/kylin/tomcat/bin/bootstrap.jar

[root@sandbox kylin]# ls -l /root/kylin/tomcat/bin/bootstrap.jar

ls: cannot access /root/kylin/tomcat/bin/bootstrap.jar: No such file or directory

[root@sandbox kylin]#

Re: How to Make Kylin work with HDP 2.3

bhagan — Tue, 17 Nov 2015 02:44:06 GMT

A little late to this thread, but late last month, I used the binary build that Kylin says should work with the version of HBASE in HDP 2.3 (apache-kylin-1.2-HBase1.1-incubating-SNAPSHOT). It installed ok, but when creating a cube, I got the following error:

Error: java.lang.NullPointerException at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsMapper.setup(FactDistinctColumnsMapper.java:73)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at

org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Re: How to Make Kylin work with HDP 2.3

rgelhausen — Tue, 08 Dec 2015 14:30:31 GMT

The installation and quick-start steps now work with HDP 2.3.2 out of the box. Make sure you download the binary for HBase 1.1.

Re: How to Make Kylin work with HDP 2.3

abajwa — Tue, 08 Dec 2015 14:41:05 GMT

Awesome! Will be trying this out

Re: How to Make Kylin work with HDP 2.3

nsabharwal — Tue, 08 Dec 2015 20:35:57 GMT

@Randy Gelhausen Did you set this up in multinode cluster setup? I will try and update the thread.

Re: How to Make Kylin work with HDP 2.3

dixingxing — Fri, 18 Dec 2015 16:31:01 GMT

I got the same error as you. Did you resolve this ?

Re: How to Make Kylin work with HDP 2.3

nsabharwal — Fri, 18 Dec 2015 21:37:58 GMT

@xingxing di Could you post the steps that you are taking?

Re: How to Make Kylin work with HDP 2.3

ccasano — Tue, 29 Dec 2015 04:41:46 GMT

This worked for me. A few other simple things I needed to do:

create /kylin folder in HDFS

add 7070 in port forwarding of sandbox VM

make sure HBase is started

Re: How to Make Kylin work with HDP 2.3

jmjava — Sun, 03 Jan 2016 00:43:24 GMT

I am getting this:

KYLIN_HOME is set to /kylin/apache-kylin-1.2-HBase1.1-incubating-SNAPSHOT-bin

mkdir: Permission denied: user=root, access=WRITE, inode="/kylin":hdfs:hdfs:drwxr-xr-x

failed to create /kylin, Please make sure the user has right to access /kylin

Re: How to Make Kylin work with HDP 2.3

jmjava — Sun, 03 Jan 2016 03:14:16 GMT

sudo -u hdfs hadoop fs -mkdir /kylin

and disabled dfs.permissions in Ambari console

restarted the sandbox

but now getting this on startup

Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.apache.kylin.rest.service.AclService]:

Caused by: org.apache.hadoop.hbase.MasterNotRunningException: java.io.IOException: Can't get master address from ZooKeeper; znode data == null

Went back into Ambari and started HBASE.

Also added 7070 port mapping in VirtualBox

I get the login screen now 🙂

Re: How to Make Kylin work with HDP 2.3

andriy_khavro — Mon, 19 Aug 2019 12:58:39 GMT

I have an issue running find-hive-dependency.sh script:

hortonworks-sandbox-with-hdp-2.png

Even though message says that Kylin is running, hive dependency is not resolved and Kylin is not actually running, here is the screenshot:

Re: How to Make Kylin work with HDP 2.3

garunkumar85 — Wed, 24 Feb 2016 01:10:55 GMT

@xingxing di @bhagan @Neeraj Sabharwal

can you guys please post step by steps to start Kylin on HDP 2.3.2 and pre-requisites for the same?

Thanks in advance