Below are the version details CDP 7.1.8 CM : 7.8.1 HIVE : 3.1.3 We are trying to insert data into partitioned table using ORC file. It contains approx 50,000 rows Using below command to load data LOAD DATA INPATH '/user/test/r_14_5.orc' INTO TABLE va_offer_16; we see that all the map and reducer task are completed withing 500s. ---------------------------------------------------------------------------------------------- VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED ---------------------------------------------------------------------------------------------- Map 1 .......... container SUCCEEDED 7 7 0 0 0 0 Reducer 2 ...... container SUCCEEDED 33 33 0 0 0 0 ---------------------------------------------------------------------------------------------- VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 2601.60 s ---------------------------------------------------------------------------------------------- However further there is no progress for next 2000 sec before failure with below error. ERRROR : ERROR : FAILED: Execution Error, return code 40000 from org.apache.hadoop.hive.ql.exec.MoveTask. org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out INFO : Completed executing command(queryId=hive_20230216163408_de7cb993-2086-4011-8788-50f46ed6e7f3); Time taken: 2605.485 seconds INFO : OK Error: Error while compiling statement: FAILED: Execution Error, return code 40000 from org.apache.hadoop.hive.ql.exec.MoveTask. org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out (state=08S01,code=40000) ... View more

Hello Team,   Facing bellow issues with hive while loading data from HDFS. Following are the error:   Error 1 ERROR : Vertex failed, vertexName=Reducer 2, vertexId=vertex_1672408587164_0154_2_01, diagnostics=[Task failed, taskId=task_1672408587164_0154_2_01_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1672408587164_0154_2_01_000001_0:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing vector batch (tag=0) (vectorizedVertexNum 1) Error 2 Caused by: org.apache.hadoop.hive.ql.metadata.HiveFatalException: [Error 20004]: Fatal error occurred when node tried to create too many dynamic partitions. The maximum number of dynamic partitions is controlled by hive.exec.max.dynamic.partitions and hive.exec.max.dynamic.partitions.pernode. Maximum was set to 2000 partitions per node, number of dynamic partitions on this node: 2001   Error 3 ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1672408587164_0175_1_00, diagnostics=[Vertex vertex_1672408587164_0175_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: offer_15__temp_table_for_load_data__ initializer failed, vertex=vertex_1672408587164_0175_1_00 [Map 1], java.lang.RuntimeException: ORC split generation failed with exception: java.io.IOException: Illegal type id 0. The valid range is 0 to -1   Please suggest the resolution for above errors.Thanks. ... View more

Hello Team, I have CDP Setup on premise and I am unable to login in ranger admin console. Details CDP: 7.1.7 | CM : 7.6.5 | Non Kerberised | Database : Mysql for all services I have tried all the below ways https://cloudera.ericlin.me/2020/02/how-to-update-ranger-web-ui-admin-users-password/ https://community.cloudera.com/t5/Support-Questions/Ranger-Password-reset/m-p/280992 In my case table: x_portal_user under ranger database was empty set. SO I manually inserted values for fields insert into x_portal_user(id,first_name,login_id,password,status) values(1,'Admin','admin','ceb4f32325eda6142bd65215f4c0f371',1); With the above query it got inserted into table. Then did FLUSH PRIVILEGES , commit steps in mysql database. Also tried further insert one more insert statement like above. On Ranger Side configuration: I have disable kerberos Admin Authentication Method: Unix Please help/suggest with the solution. Thanks ... View more

Hello,   I'm facing issue while executing query (show databases;) on hive.   Following is my setup: Cluster : CDP-7.1.7 Components Installed : HDFS, Yarn, Yarn Queue Manager, Oozie, Schema Registry, Hive, Ziplin, Spark, Tez and Zookeeper.    I have accessed hive using beeline, but when I tried to execute 'show databases' it is giving following error :   ERROR : FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.metadata.HiveException(Error while invoking PostHook. hooks: java.lang.RuntimeException: java.lang.NullPointerException at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:50) at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.getPipeline(AbstractHiveExecHookContext.java:86) at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.run(AbstractHiveExecHookContext.java:69) at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296) at org.apache.hadoop.hive.ql.HookRunner.runPostExecHooks(HookRunner.java:278) at org.apache.hadoop.hive.ql.Executor.postExecutionActions(Executor.java:477) at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:111) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:749) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:504) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:498) at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226) at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:88) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:327) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:345) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException at java.io.File.<init>(File.java:277) at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:43) ... 24 more ) org.apache.hadoop.hive.ql.metadata.HiveException: Error while invoking PostHook. hooks: java.lang.RuntimeException: java.lang.NullPointerException at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:50) at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.getPipeline(AbstractHiveExecHookContext.java:86) at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.run(AbstractHiveExecHookContext.java:69) at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296) at org.apache.hadoop.hive.ql.HookRunner.runPostExecHooks(HookRunner.java:278) at org.apache.hadoop.hive.ql.Executor.postExecutionActions(Executor.java:477) at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:111) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:749) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:504) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:498) at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226) at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:88) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:327) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:345) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException at java.io.File.<init>(File.java:277) at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:43) ... 24 more at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:302) at org.apache.hadoop.hive.ql.HookRunner.runPostExecHooks(HookRunner.java:278) at org.apache.hadoop.hive.ql.Executor.postExecutionActions(Executor.java:477) at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:111) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:749) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:504) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:498) at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226) at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:88) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:327) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:345) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: java.lang.NullPointerException at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:50) at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.getPipeline(AbstractHiveExecHookContext.java:86) at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.run(AbstractHiveExecHookContext.java:69) at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296) ... 21 more Caused by: java.lang.NullPointerException at java.io.File.<init>(File.java:277) at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:43) ... 24 more INFO : Completed executing command(queryId=hive_20220330193703_a3c2039f-3446-4d1c-8ed6-b4e88ac77771); Time taken: 0.051 seconds INFO : OK Error: Error while compiling statement: FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.metadata.HiveException(Error while invoking PostHook. hooks: java.lang.RuntimeException: java.lang.NullPointerException at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:50) at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.getPipeline(AbstractHiveExecHookContext.java:86) at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.run(AbstractHiveExecHookContext.java:69) at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296) at org.apache.hadoop.hive.ql.HookRunner.runPostExecHooks(HookRunner.java:278) at org.apache.hadoop.hive.ql.Executor.postExecutionActions(Executor.java:477) at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:111) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:749) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:504) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:498) at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226) at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:88) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:327) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:345) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException at java.io.File.<init>(File.java:277) at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:43) ... 24 more ) (state=08S01,code=12)     Can anyone please help me this?   Thanks.  ... View more