I was referring to the following which is not available yet in spark 1.6 :  1)create a DF 2)create a table to write direct sql queries on: df.createGlobalTempView("people") 3)query on this table : spark.sql("SELECT * FROM global_temp.people")   But I think what is required for the section "data analysis: use spark sql to interact with the metastore programmatically in your application" is to create a SQL/HiveContext and then query on tables that are already stored in the HIVE metastore. ANy idea if this is correct?     ... View more

Dear community manager, Which one exactly is this? Is this all the documentation available on this page: https://www.cloudera.com/documentation/enterprise/latest.html - these are the pages that I am mainly using to study all the topics that are part of the exam? Or only the administration section (this is a pdf of nearly 600 pages)?     Thanks for response!! ... View more

I have the same issue. I am following both the documentation in  https://www.bmc.com/blogs/how-to-write-a-hive-user-defined-function-udf-in-java/ and the link mentioned in previous post: https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cm_mc_hive_udf.html . These are the steps I have taken:   1) The goal is to create a temporary user defined function FNV.java. I have put in dir /src/main/java/com/company/hive/udf/FNV.java the following code: package com.company.hive.udf; import org.apache.hadoop.hive.ql.exec.UDF; import org.apache.hadoop.hive.ql.exec.Description; import org.apache.hadoop.io.Text; import java.math.BigInteger; public final class FNV extends UDF{ <...all tha java code...> } 2)I have added the 2 required JARS for the imports to the CLASSPATH, compiled, build a jar out of this: /src/main/java/com/company/hive/udf/FNV.jar. This is present on the host where the hive metastore and hiveserver2 is running. I check with jar tvf FNV.jar and see that my class src/main/java/com/company/hive/udf/FNV.class is present 3)I put the FNV.jar file on hdfs and did a chown hive:hive and a chmod with full 777 rights 4)I changed the configuration for 'Hive Auxiliary JARs Directory' in Hive to the path of the jar: /src/main/java/com/company/hive/udf/ 5)I redeployed the client config and restarted hive. Here I notice that the 2nd hiveserver (on a different node-not where the JAR is located) has trouble restarted. The host with the hive metastore, hiveserver2 and the jar is up and running. 6) I granted access to the hdfs location and the file on the local host to a role called 'hive_jar'. This is done by logging into beeline !connect jdbc:hive2://node009.cluster.local:10000/default GRANT ALL ON URI 'file:///src/main/java/com/company/hive/udf/FNV.jar' TO ROLE HIVE_JAR; GRANT ALL ON URI 'hdfs:///user/name/FNV.jar' TO ROLE HIVE_JAR; I do notice that SHOW CURRENT ROLES in beeline for the hive user does give the HIVE_JAR role as wanted. 7)I start hive and add the jar using the local hosts's path: add jar /src/main/java/com/company/hive/udf/FNV.jar; I check with list jars that the jar is present 😎 In the same session I try to create the temporary function: create temporary function FNV as 'com.company.hive.udf.FNV'; I keep on getting error : FAILED: Class com.company.hive.udf.FNV not found FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.FunctionTask     Any clue what I am missing?? THanks for feedback! ... View more