Hi,I want to update Hive UDFs without requiring a restart of hive. According to: https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cm_mc_hive_udf.html#concept_zb2_rxr_lw setting hive.reloadable.aux.jars.path is required. I have set it to /user/hive/libs/udf (which resides on HDFS). However following their documentation I see: file:///usr/lib/hive/lib/foo.jar which is confusing me. Does this property only work for files residing on the local file system? Do I understand correctly. that I should execute beelines reload manually? Also in case this property works for HDFS does it automatically pick up the classes in the jar (load them) and no longer requires to specify the CREATE FUNCTION foo AS 'my/path/to/jar-1.jar'? Desired behaviour: 1. copy jar to HDFS /user/hive/lib/udf/foo-1.jar 2. add function to hive: DROP FUNCTION IF EXISTS foo; CREATE FUNCTION foo AS 'my.class.path.in.jar.FooUDF' USING JAR '/user/hive/lib/udf/foo-1.jar'; 3. add a new jar to HDFS /user/hive/lib/udf/foo-2.jar 4. update function in hive: DROP FUNCTION IF EXISTS foo; CREATE FUNCTION foo AS 'my.class.path.in.jar.FooUDF' USING JAR '/user/hive/lib/udf/foo-2.jar'; This currently does not work and requires a restart of hive. It results in round robin seeing the updated UDF (or still the old one). How can I get hive to not require a restart when updating UDF? Also I do not want to put the UDF locally into a directory. It should reside on HDFS. Best, Georg ... View more

When executing the following query on LLAP: WITH first as (SELECT 1 as a, 1 as b) SELECT * FROM first as f JOIN first as f2 ON f.a=f2.a Hive fails with: IllegalArgumentException: max parallelism must be positive, currently is 0 ... View more

I face problems with a hive udf and java dependency hell: 2018-02-19 10:11:49,328 [ERROR] [main] |app.DAGAppMaster|: Error starting DAGAppMaster java.lang.VerifyError: Bad return type Exception Details: Location: org/apache/hadoop/hdfs/DFSClient.getQuotaUsage(Ljava/lang/String;)Lorg/apache/hadoop/fs/QuotaUsage; @94: areturn Reason: Type 'org/apache/hadoop/fs/ContentSummary' (current frame, stack[0]) is not assignable to 'org/apache/hadoop/fs/QuotaUsage' (from method signature) Obviously, somehow dependencies are clashing with wrong versions. When looking at the jar via jar tf myjar.jar | grep hdfs no contents are returned. The same for fd This seems strange to me as the exception clearly states that these should be involved in the problem. The error occurs only on Tez, not on regular hive. ... View more

I want to use sparks JDBC connection to write a data frame to oracle. My data frame has a string column which is very long. At least longer than the default 255 characters the spark will allocate when creating the schema. How can I still write to the oracle table? Does it work when I manually create the schema first with CLOB datatypes? If yes, how can I get spark to only `TRUNCATE` instead of overwrite the table? ... View more

Hi, is there any possibility in the newly released HDP 2.6 to install NIFI in cluster mode via ambari? ... View more