Cloudera Community
jayes
user rank
Rising Star

Re: Not Able to run import command. it fails with ...

by Rising Star in Support Questions
‎05-07-2024 03:03 AM
1 Kudo
‎05-07-2024 03:03 AM
1 Kudo
Hello @smruti , Thanks for the help.  ... View more

Not Able to run import command. it fails with inva...

by Rising Star in Support Questions
‎05-02-2024 05:03 AM
1 Kudo
‎05-02-2024 05:03 AM
1 Kudo
Hello Team, I am unable to run the import command for Hive, since it is failing with error invalid path. I copied the table content to another folder and it has permission for hive user, but while running import command it says invalid path. i ran import command on the path where the source table location still it fails with same reason. Beeline version 3.1.3000.7.1.8.28-1 by Apache Hive [root@machine1 Log_Files]# sudo -u hive hadoop fs -ls /warehouse/tablespace/managed/hive/demo.db/test1/* Found 1 items -rw-r--r-- 3 hive hive 621 2024-05-02 16:05 /warehouse/tablespace/managed/hive/demo.db/test1/delta_0000004_0000004_0000/bucket_00000_0 [root@machine1 Log_Files]# sudo -u hive hadoop fs -ls /jay123/restore_jobid123243/jayesh/* Found 1 items drwxr-xr-x - hive hive 0 2024-05-02 16:05 /jay123/restore_jobid123243/jayesh/test1/delta_0000004_0000004_0000 [root@machine1 Log_Files]# sudo -E -u hive beeline -u "jdbc:hive2://machine1.testride.com:2181,machine22.testride.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;;principal=hive/_HOST@hdpcluster.test;" -n hive --showHeader=false --outputformat=tsv2 -e "use demo; show create table test1;" CREATE TABLE `test1`( `nm` int) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' LOCATION 'hdfs://machine1.testride.com:8020/warehouse/tablespace/managed/hive/demo.db/test1' TBLPROPERTIES ( 'bucketing_version'='2', 'last_modified_by'='hive', 'last_modified_time'='1713262492', 'transactional'='true', 'transactional_properties'='default', 'transient_lastDdlTime'='1714646145') 17 rows selected (0.329 seconds) 0: jdbc:hive2://machine1.testride.com:2181,machine2.t> use jayesh;import table test1 from '/jay123/restore_jobid123243/jayesh/test1'; INFO : Compiling command(queryId=hive_20240502173023_4e2433fa-dc38-4357-a5be-90892c65a217): use jayesh INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20240502173023_4e2433fa-dc38-4357-a5be-90892c65a217); Time taken: 0.007 seconds INFO : Executing command(queryId=hive_20240502173023_4e2433fa-dc38-4357-a5be-90892c65a217): use jayesh INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20240502173023_4e2433fa-dc38-4357-a5be-90892c65a217); Time taken: 0.014 seconds INFO : OK No rows affected (0.13 seconds) Error: Error while compiling statement: FAILED: SemanticException [Error 10027]: Invalid path (state=42000,code=10027)   0: jdbc:hive2://machine1.testride.com:2181,machine2.t> use jayesh;import table test1 from '/warehouse/tablespace/managed/hive/demo.db/test1'; INFO : Compiling command(queryId=hive_20240502173047_d1f036dc-8722-4fc2-be00-1bafd032971e): use jayesh INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20240502173047_d1f036dc-8722-4fc2-be00-1bafd032971e); Time taken: 0.006 seconds INFO : Executing command(queryId=hive_20240502173047_d1f036dc-8722-4fc2-be00-1bafd032971e): use jayesh INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20240502173047_d1f036dc-8722-4fc2-be00-1bafd032971e); Time taken: 0.006 seconds INFO : OK No rows affected (0.027 seconds) Error: Error while compiling statement: FAILED: SemanticException [Error 10027]: Invalid path (state=42000,code=10027) Can you please help me here to understand what's the issue and how can i resolve this.   ... View more
Labels:

Re: Beeline command failing with error 4

by Rising Star in Support Questions
‎12-26-2023 10:49 PM
‎12-26-2023 10:49 PM
Hello @smruti , Please let me know if any way where we can have call or meet to review the script ? I will in mean time will try to remove the -n options and check if that works. ... View more

Re: Beeline command failing with error 4

by Rising Star in Support Questions
‎12-25-2023 11:13 PM
‎12-25-2023 11:13 PM
Hi @smruti , Yes the exit code is 4. I tried to redirect the stdout and stderr to the file but there was no file created. Following is output of ps -eaf :  loki 3936766 3935975 0 02:06 ? 00:00:00 bash /opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/bin/../lib/hive/bin/beeline -u 'jdbc:hive2://machine.domain.com:2181/default;password=**;principal=hive/_HOST@DEVEMC.COMMVAULT.COM;serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=***;user=loki;zooKeeperNamespace=hiveserver2' -n loki -n /tmp/cmdcvhive.3935975.140226804802880 --showHeader=false --outputformat=tsv2 --verbose=true -e 'use underscoretest; show tables;' >> /tmp/file 2>&1 loki 3936766 3935975 2 02:06 ? 00:00:00 bash /opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/bin/../lib/hive/bin/beeline -u 'jdbc:hive2://machine.domain.com:2181/default;password=**;principal=hive/_HOST@DEVEMC.COMMVAULT.COM;serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=**;user=loki;zooKeeperNamespace=hiveserver2' -n loki -n /tmp/cmdcvhive.3935975.140226804802880 --showHeader=false --outputformat=tsv2 --verbose=true -e 'use underscoretest; show tables;' >> /tmp/file 2>&1 ... View more

Re: Beeline command failing with error 4

by Rising Star in Support Questions
‎12-22-2023 03:19 AM
‎12-22-2023 03:19 AM
Hi @smruti , The setup has kerberos authentication.  There is no error message the moment the process is launched with loki user, the process exits with error 4. Sorry but the HiveUser and loki user is same its just replacing the username went wrong. The same command when i do su loki and then run the same command it works. or if the same command is ran using sudo it works. Issue is with launching it with other process.  ... View more

Re: Beeline command failing with error 4

by Rising Star in Support Questions
‎12-22-2023 03:16 AM
‎12-22-2023 03:16 AM
Hi @cloude , Which logs should i share cause the moment the beeline command is launched in another process it fails. so i dont have a way to check why is it failing. it just says process exited with error 4. Can you please help here to know which logs file might have those error ? ... View more

Beeline command failing with error 4

by Rising Star in Support Questions
‎12-19-2023 01:59 AM
‎12-19-2023 01:59 AM
Hello Team, I am running a process in which i am trying to run beeline command. While running beeline command with sudo it is working but if i am trying to run beeline in a process which if forked from parent process with the hive user it is failing with error 4  Can you please help me to debug what's going wrong or anything that i need to set. If i do su hiveuser and run the same command t works but while running via process it is failing. Command that i see while running via fork process : bash /opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/bin/../lib/hive/bin/beeline -u 'jdbc:hive2://machine1.domain.com:2181/default;password=***;principal=hive/_HOST@DOMAIN.COM;serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=*****;user=HiveUser;zooKeeperNamespace=hiveserver2' -n loki -n /tmp/cmdcvhive.4015998.140520474391872 --showHeader=false --outputformat=tsv2 -e 'use underscoretest; show tables;' ... View more
Labels:

Re: Not able to set dfs.replication while running ...

by Rising Star in Support Questions
‎11-06-2023 10:27 PM
‎11-06-2023 10:27 PM
Hi @smruti , Yes agree that we can set parameter in hdfs-site.xml, so that it will reflect, but i dont want to change replication factor of whole cluster. Its just while running command i want to change. i tried setting the property in hive-site.xml on all 3 nodes of cluster and restarted the hive service, but still it is failing with same error. <property> <name>hive.security.authorization.sqlstd.confwhitelist.append</name> <value>mapred\..*|hive\..*|mapreduce\..*|spark\..*|dfs\..*</value> </property> Error : Connected to: Apache Hive (version 3.1.3000.7.1.7.0-551) Driver: Hive JDBC (version 3.1.3000.7.1.7.0-551) Transaction isolation: TRANSACTION_REPEATABLE_READ Error: Error while processing statement: Cannot modify dfs.replication at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)     ... View more

Re: Not able to set dfs.replication while running ...

by Rising Star in Support Questions
‎11-06-2023 10:03 PM
‎11-06-2023 10:03 PM
IT is still not working tried from the beeline shell too ... View more

Re: Not able to set dfs.replication while running ...

by Rising Star in Support Questions
‎11-06-2023 10:03 PM
‎11-06-2023 10:03 PM
0: jdbc:hive2://machine1.***.***.c> show databases; INFO : Compiling command(queryId=hive_20231107010124_eaff8e06-cd0a-4484-a1b0-80f9eb2bae64): show databases INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null) INFO : Completed compiling command(queryId=hive_20231107010124_eaff8e06-cd0a-4484-a1b0-80f9eb2bae64); Time taken: 0.038 seconds INFO : Executing command(queryId=hive_20231107010124_eaff8e06-cd0a-4484-a1b0-80f9eb2bae64): show databases INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20231107010124_eaff8e06-cd0a-4484-a1b0-80f9eb2bae64); Time taken: 0.037 seconds INFO : OK +---------------------+ | database_name | +---------------------+ | default | | information_schema | | largedb | | newtest | | sys | | testdb | | underscore | | underscoretest | +---------------------+ 0: jdbc:hive2://machine1.***.***.c> set def-replication=1; Error: Error while processing statement: Cannot modify def-replication at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)   ... View more
Contact Me
Online
Offline
Last Visited
‎12-10-2024 01:21 AM
Community Statistics
Member Since ‎09-26-2023 06:53 AM
Last Visited ‎12-10-2024 01:21 AM
Posts 26
Kudos received 7