Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Problem with create new table in Hue

avatar
Explorer

I am using metastore manager to copy hdfs file and create a new table in hive.  But, this table file (orders) has 3 different parquet files.  I am using the "create a new table from a file" utility and  I am getting an error message as shown below. I want to know if this is the process to create a hive table from this kind of a file.  I want to know if this is more of permission issue to upload files.  I would appreciate if someone responds to this post.

 

Mark

 

"

Failed to open file '/user/hive/warehouse/orders_new': [Errno 21] Is a directory: '/user/hive/warehouse/orders_new'

None

 
1 ACCEPTED SOLUTION

avatar
Explorer

Thanks very much for your reply.  I will try to work on it.  It could be that I dont have the necessary background to do this kind of task.

 

Mark

View solution in original post

9 REPLIES 9

avatar
Explorer

Please note there are 3 parquet files that make up the orders_new table. 

 

Mark

 

 

avatar
Explorer

I am also including the three files in the hdfs file I am trying to upload.  Please provide me instructions if I should modify permissions  on these files. 

 

Mark

 

[ec2-user@cloudera1 ~]$ hadoop fs -ls /user/hive/warehouse/orders_new
Found 4 items
drwxrwxrwt   - admin hive          0 2015-10-30 09:58 /user/hive/warehouse/orders_new/.metadata
-rw-r--r--   2 admin hive     188686 2015-10-30 09:58 /user/hive/warehouse/orders_new/0ea8c3e9-493c-4eb7-9e8f-2e9a26b1620a    .parquet
-rw-r--r--   2 admin hive     184808 2015-10-30 09:58 /user/hive/warehouse/orders_new/37c29bc0-7528-4fed-b794-b7532dea7e92    .parquet
-rw-r--r--   2 admin hive     184752 2015-10-30 09:58 /user/hive/warehouse/orders_new/785ec0d6-9d45-479b-9778-8a4121783c0d    .parquet

avatar
Super Guru
The create table metastore wizard does not currently support parquet files
in input.

You should be able to manually create an external table based on those
files:
https://www.google.com/search?q=hive+create+table+from+parquet+files&oq=hive+create+table+from+parqu...


avatar
Explorer

Hi Romain,

 

The link you sent is a Google search page.  Can you be more specific where I can get detailed instructions to upload three  different parquet files?  I can create the table but I will need help with regards to loading data.  I would appreciate your help.

 

Mark

avatar
Explorer

Hi Romain,

 

I tried creating the table and I could not progress very far as I doesn't know the SerDe properties.  That is where I got stuck.  Can you help me with this issue?

 

Mark

avatar
Super Collaborator
Hi,

Here is a link that show you the HIVE Syntax for creating a table stored as parquet :
https://cwiki.apache.org/confluence/display/Hive/Parquet#Parquet-Hive0.13andlater

avatar
Explorer

Hi Mathieu,

 

Thanks for sending that information.  This is what I am looking for.  I created a copy of orders hdfs file in the same directory.  I wanted to create a table using Hue Metastore Manager.  The table file consists of three different parquet files.  Is there a utility or set script to load these parquet files into the new table?  I am not a programmer and I am new to Big Data.  I would appreciate if you could send me a sample script that I could use.  Thanks.

 

Mark

avatar
Super Collaborator

As far as I'm aware of, I don't know of any utility which will automatically create the "right" table for your parquet data files.

You will need to create your table manualy (and according to the structure defined by your parquet files).

 

In order to do that, I think you should follow theses steps :

- 1 : Puts the 3 files inside the same folder on HDFS (only theses 3 files).

- 2 : Create an external tables :

---- pointing to that directory

---- specifying that the format is "parquet"

---- specifying the field expected by your existing parquet file (you need to know the structure of your parquet files for this)

 

By the way : does your 3 parquet files have the same structure ? (same number of field, same field name)

If no : this will not work.

 

regards

 

 

avatar
Explorer

Thanks very much for your reply.  I will try to work on it.  It could be that I dont have the necessary background to do this kind of task.

 

Mark