Member since
09-26-2023
24
Posts
6
Kudos Received
0
Solutions
10-20-2024
11:30 PM
1 Kudo
Hi @MGreen , i am unable to find how to pass which type of format we want the export to get dumped. is it using beeline if yes can you please help me with the command or the argument to pass to export in compress format. since today when i run export command the data size of table and data size of exported data approximately same.
... View more
10-17-2024
11:20 PM
1 Kudo
Thanks for the help. Also have a doubt can we export the data to hdfs in compressed format i.e., into zip file or something else
... View more
10-14-2024
02:37 AM
1 Kudo
Hello Team, can we import/export hive table to s3 location or nfs share ? as per what i searched it is only supported for hdfs location. can you please help me to know if we can do it and if yes how can we achieve it ?
... View more
Labels:
- Labels:
-
Apache Hive
06-07-2024
02:53 AM
1 Kudo
Hi @ggangadharan , Can we export the table data into multiple files via some command like numfile or anything ?
... View more
06-03-2024
01:45 AM
Hello @ggangadharan , Can you please help me with the size per file ? or some setting where the size is mentioned ?
... View more
06-03-2024
01:44 AM
Command to dump the command : export table test to '/jayesh/testdump1';
... View more
05-29-2024
07:15 AM
i am trying to check the files that are created under data folder when we export table to hdfs location. I have single column table with 5000 rows but after export it created only one data file named as 000000_0 I have another table with partition, 2 partitions with 95 rows each approx. and for this table export is see 2 folders each for partition and then under it only one file. Can you please help me to know how are the data files for the exported table are created and how is it distributed? I mean is it only one data file for each table without partitioned? or it will create multiple files based on specific row count or size count or on what criteria it is split?
... View more
Labels:
- Labels:
-
Apache Hive
05-07-2024
03:03 AM
1 Kudo
Hello @smruti , Thanks for the help.
... View more
05-02-2024
05:03 AM
1 Kudo
Hello Team, I am unable to run the import command for Hive, since it is failing with error invalid path. I copied the table content to another folder and it has permission for hive user, but while running import command it says invalid path. i ran import command on the path where the source table location still it fails with same reason. Beeline version 3.1.3000.7.1.8.28-1 by Apache Hive [root@machine1 Log_Files]# sudo -u hive hadoop fs -ls /warehouse/tablespace/managed/hive/demo.db/test1/* Found 1 items -rw-r--r-- 3 hive hive 621 2024-05-02 16:05 /warehouse/tablespace/managed/hive/demo.db/test1/delta_0000004_0000004_0000/bucket_00000_0 [root@machine1 Log_Files]# sudo -u hive hadoop fs -ls /jay123/restore_jobid123243/jayesh/* Found 1 items drwxr-xr-x - hive hive 0 2024-05-02 16:05 /jay123/restore_jobid123243/jayesh/test1/delta_0000004_0000004_0000 [root@machine1 Log_Files]# sudo -E -u hive beeline -u "jdbc:hive2://machine1.testride.com:2181,machine22.testride.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;;principal=hive/_HOST@hdpcluster.test;" -n hive --showHeader=false --outputformat=tsv2 -e "use demo; show create table test1;" CREATE TABLE `test1`( `nm` int) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' LOCATION 'hdfs://machine1.testride.com:8020/warehouse/tablespace/managed/hive/demo.db/test1' TBLPROPERTIES ( 'bucketing_version'='2', 'last_modified_by'='hive', 'last_modified_time'='1713262492', 'transactional'='true', 'transactional_properties'='default', 'transient_lastDdlTime'='1714646145') 17 rows selected (0.329 seconds) 0: jdbc:hive2://machine1.testride.com:2181,machine2.t> use jayesh;import table test1 from '/jay123/restore_jobid123243/jayesh/test1'; INFO : Compiling command(queryId=hive_20240502173023_4e2433fa-dc38-4357-a5be-90892c65a217): use jayesh INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20240502173023_4e2433fa-dc38-4357-a5be-90892c65a217); Time taken: 0.007 seconds INFO : Executing command(queryId=hive_20240502173023_4e2433fa-dc38-4357-a5be-90892c65a217): use jayesh INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20240502173023_4e2433fa-dc38-4357-a5be-90892c65a217); Time taken: 0.014 seconds INFO : OK No rows affected (0.13 seconds) Error: Error while compiling statement: FAILED: SemanticException [Error 10027]: Invalid path (state=42000,code=10027) 0: jdbc:hive2://machine1.testride.com:2181,machine2.t> use jayesh;import table test1 from '/warehouse/tablespace/managed/hive/demo.db/test1'; INFO : Compiling command(queryId=hive_20240502173047_d1f036dc-8722-4fc2-be00-1bafd032971e): use jayesh INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20240502173047_d1f036dc-8722-4fc2-be00-1bafd032971e); Time taken: 0.006 seconds INFO : Executing command(queryId=hive_20240502173047_d1f036dc-8722-4fc2-be00-1bafd032971e): use jayesh INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20240502173047_d1f036dc-8722-4fc2-be00-1bafd032971e); Time taken: 0.006 seconds INFO : OK No rows affected (0.027 seconds) Error: Error while compiling statement: FAILED: SemanticException [Error 10027]: Invalid path (state=42000,code=10027) Can you please help me here to understand what's the issue and how can i resolve this.
... View more
Labels:
- Labels:
-
Apache Hive