Hi Team , Please, let me know the charges of   Cloudera Enterprise Support Subscription as I need to install apache phoenix for cdh 6.2. According to official documentation, you must have cloudera enterprise subscription.  ... View more

Hi, I have installed and configured Apache Kylin 2.4 on Cloudera Platform for creating the Cube. I have been able to create a cube in MapReduce mode but getting the below-mentioned caveat while executes on spark mode. I have had followed all the steps and tried many remedies for debugging the problem.   Please let me know how to resolve this bug. Thanks in Advance.     1091 [main] ERROR org.apache.spark.SparkContext  - Error adding jar (java.lang.IllegalArgumentException: requirement failed: JAR kylin-job-2.4.0.jar already registered.), was the --addJars option used? [Stage 0:>                                                          (0 + 0) / 2] [Stage 0:>                                                          (0 + 2) / 2]                                                                                  60505 [dispatcher-event-loop-6] ERROR org.apache.spark.scheduler.cluster.YarnScheduler  - Lost executor 1 on ** ******* : Container marked as failed: container_e62_1566915974858_6628_01_000003 on host: ******* . Exit status: 50. Diagnostics: Exception from container-launch. Container id: container_e62_1566915974858_6628_01_000003 Exit code: 50 Stack trace: ExitCodeException exitCode=50:  at org.apache.hadoop.util.Shell.runCommand(Shell.java:604) at org.apache.hadoop.util.Shell.run(Shell.java:507) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:789) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:213) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 50 82664 [dispatcher-event-loop-5] ERROR org.apache.spark.scheduler. cluster.YarnScheduler  - Lost executor 2 on *******: Container marked as failed: container_e62_1566915974858_ 6628_01_000004 on host: *******. Exit status: 50. Diagnostics: Exception from container-launch. Container id: container_e62_1566915974858_ 6628_01_000004 Exit code: 50 Stack trace: ExitCodeException exitCode=50:  at org.apache.hadoop.util.Shell. runCommand(Shell.java:604) at org.apache.hadoop.util.Shell. run(Shell.java:507) at org.apache.hadoop.util.Shell$ ShellCommandExecutor.execute( Shell.java:789) at org.apache.hadoop.yarn.server. nodemanager. DefaultContainerExecutor. launchContainer( DefaultContainerExecutor.java: 213) at org.apache.hadoop.yarn.server. nodemanager.containermanager. launcher.ContainerLaunch.call( ContainerLaunch.java:302) at org.apache.hadoop.yarn.server. nodemanager.containermanager. launcher.ContainerLaunch.call( ContainerLaunch.java:82) at java.util.concurrent. FutureTask.run(FutureTask. java:262) at java.util.concurrent. ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1145) at java.util.concurrent. ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread. java:745) Container exited with a non-zero exit code 50 The command is:  export HADOOP_CONF_DIR=/etc/hadoop/ conf && /usr/lib/spark/bin/spark- submit --class org.apache.kylin.common.util. SparkEntry  --conf spark.executor.instances=1  --conf spark.yarn.archive=hdfs:// namenode:8020/kylin/spark/ spark-libs.jar  --conf spark.yarn.queue=default  --conf spark.yarn.am. extraJavaOptions=-Dhdp. version=current  --conf spark.history.fs.logDirectory= hdfs:///kylin/spark-history  --conf spark.driver.extraJavaOptions= -Dhdp.version=current  --conf spark.io.compression.codec= org.apache.spark.io. SnappyCompressionCodec  --conf spark.master=yarn  --conf spark.executor. extraJavaOptions=-Dhdp. version=current  --conf spark.hadoop.yarn.timeline- service.enabled=false  --conf spark.executor.memory=4G  --conf spark.eventLog.enabled=true  --conf spark.eventLog.dir=hdfs:/// kylin/spark-history  --conf spark.executor.cores=2  --conf spark.submit.deployMode= cluster --jars /opt/apache-kylin-2.4.0-bin- cdh57/lib/kylin-job-2.4.0.jar /opt/apache-kylin-2.4.0-bin- cdh57/lib/kylin-job-2.4.0.jar -className org.apache.kylin.engine.spark. SparkCubingByLayer -hiveTable default.kylin_intermediate_ kylin_sales_cube_c1526d16_ 9719_4dec_be41_346f43654e02 -input hdfs://nameservice1/kylin/ kylin_metadata/kylin-2159d40b- f14e-4500-af95-1fbfd5a4073f/ kylin_intermediate_kylin_ sales_cube_c1526d16_9719_4dec_ be41_346f43654e02 -segmentId c1526d16-9719-4dec-be41- 346f43654e02 -metaUrl kylin_metadata@hdfs,path=hdfs: //nameservice1/kylin/kylin_ metadata/kylin-2159d40b-f14e- 4500-af95-1fbfd5a4073f/kylin_ sales_cube/metadata -output hdfs://nameservice1/kylin/ kylin_metadata/kylin-2159d40b- f14e-4500-af95-1fbfd5a4073f/ kylin_sales_cube/cuboid/ -cubename kylin_sales_cube ... View more

I have been using SAP Enterprise from last 3 years,now we are thinking to migrate the SAP Data to Hadoop platform for implementing Deep Learning/Machine learning. Please let me know do we need to pay any licensing fee when to use SAP-Hadoop connector for importing of SAP (HANA) via sqoop? ... View more

Hi, I have created a sqoop job using below mentioned command but when i execute the sqoop job,then it asks for password.Command works fine when I give the password. sqoop job --create incnew -- import -connect jdbc:mysql://*********:3306/hto --username ***** --password ******* --table daily_sales --target-dir /user/q3tech/All --incremental append --check-column added_on --last-value '2018-08-15 00:00:00' Please let me know how to execute the sqoop command i.e "sqoop job --exec incnew" without the need of password. ... View more

I am running the beeline or hive command on terminal ,it is asking me for username and password.but i want to open beeline or hive cli directly without asking username and password.Please let me know is there any way to permanently set the username and password in beeline. beeline.png ... View more

I am trying to do automation through oozie over hive. For that I wrote a below given hive query INSERT OVERWRITE LOCAL DIRECTORY '/tmp/hivefinal' SELECT * FROM gourav1; Here, gourav1 table is already created inside hive. When I am submitting oozie job. Oozie application goes to running mode and doesn't execute. Although I checked the yarn application list, were job was hanged on 95%.Please check the attached workflow.xml, script.q, workflow.xml.security and job.properties. I am also attaching the logs screenshot ,hive-site.xml, oozie-site.xml and yarn-site.xml with this post. Please let me know how to resolve this problem. Thanks in Advance. final.pdf ... View more