Problem with create new table in Hue

[#1]

I am using metastore manager to copy hdfs file and create a new table in hive.  But, this table file (orders) has 3 different parquet files.  I am using the "create a new table from a file" utility and  I am getting an error message as shown below. I want to know if this is the process to create a hive table from this kind of a file.  I want to know if this is more of permission issue to upload files.  I would appreciate if someone responds to this post.

Mark

"

Failed to open file '/user/hive/warehouse/orders_new': [Errno 21] Is a directory: '/user/hive/warehouse/orders_new'

None

[#2]

I am using metastore manager to copy hdfs file and create a new table in hive.  But, this table file (orders) has 3 different parquet files.  I am using the "create a new table from a file" utility and  I am getting an error message as shown below. I want to know if this is the process to create a hive table from this kind of a file.  I want to know if this is more of permission issue to upload files.  I would appreciate if someone responds to this post.

Mark

"

Failed to open file '/user/hive/warehouse/orders_new': [Errno 21] Is a directory: '/user/hive/warehouse/orders_new'

None

[#3]

Please note there are 3 parquet files that make up the orders_new table. 

Mark

[#4]

I am also including the three files in the hdfs file I am trying to upload.  Please provide me instructions if I should modify permissions  on these files. 

Mark

[ec2-user@cloudera1 ~]$ hadoop fs -ls /user/hive/warehouse/orders_new
Found 4 items
drwxrwxrwt   - admin hive          0 2015-10-30 09:58 /user/hive/warehouse/orders_new/.metadata
-rw-r--r--   2 admin hive     188686 2015-10-30 09:58 /user/hive/warehouse/orders_new/0ea8c3e9-493c-4eb7-9e8f-2e9a26b1620a    .parquet
-rw-r--r--   2 admin hive     184808 2015-10-30 09:58 /user/hive/warehouse/orders_new/37c29bc0-7528-4fed-b794-b7532dea7e92    .parquet
-rw-r--r--   2 admin hive     184752 2015-10-30 09:58 /user/hive/warehouse/orders_new/785ec0d6-9d45-479b-9778-8a4121783c0d    .parquet

[#5]

The create table metastore wizard does not currently support parquet files
in input.

You should be able to manually create an external table based on those
files:
https://www.google.com/search?q=hive+create+table+from+parquet+files&oq=hive+create+table+from+parqu...

[#6]

Hi Romain,

The link you sent is a Google search page.  Can you be more specific where I can get detailed instructions to upload three  different parquet files?  I can create the table but I will need help with regards to loading data.  I would appreciate your help.

Mark

[#7]

Hi Romain,

I tried creating the table and I could not progress very far as I doesn't know the SerDe properties.  That is where I got stuck.  Can you help me with this issue?

Mark

[#8]

Hi,

Here is a link that show you the HIVE Syntax for creating a table stored as parquet :
https://cwiki.apache.org/confluence/display/Hive/Parquet#Parquet-Hive0.13andlater

[#9]

Hi Mathieu,

Thanks for sending that information.  This is what I am looking for.  I created a copy of orders hdfs file in the same directory.  I wanted to create a table using Hue Metastore Manager.  The table file consists of three different parquet files.  Is there a utility or set script to load these parquet files into the new table?  I am not a programmer and I am new to Big Data.  I would appreciate if you could send me a sample script that I could use.  Thanks.

Mark

[#10]

As far as I'm aware of, I don't know of any utility which will automatically create the "right" table for your parquet data files.

You will need to create your table manualy (and according to the structure defined by your parquet files).

In order to do that, I think you should follow theses steps :

- 1 : Puts the 3 files inside the same folder on HDFS (only theses 3 files).

- 2 : Create an external tables :

---- pointing to that directory

---- specifying that the format is "parquet"

---- specifying the field expected by your existing parquet file (you need to know the structure of your parquet files for this)

By the way : does your 3 parquet files have the same structure ? (same number of field, same field name)

If no : this will not work.

regards

[#11 · Accepted solution]

Thanks very much for your reply.  I will try to work on it.  It could be that I dont have the necessary background to do this kind of task.

Mark