About balavignesh_nag

balavignesh_nag · ‎09-12-2017

Hi @RANKESH NATH Hive table with ACID property enabled table will work good for your use case. Also with the help of Merge update you will be able to perform these inserts/updates easily. In typing data warehousing implementing SCD type 2 tables are performed easily with the help of Merge update function in hive. To answer whether spark in terms of insert/updates you can complex computation with much ease. But when it comes to updating the existing record its better to go with Hive than spark. Hope it Helps!!

balavignesh_nag · ‎08-25-2017

Hi @Vivekanandan Gunasekaran Log states that there is already file contained in the place where you are trying to create a hive table. You may need to delete that file and try to re-create it or create the new hive table pointing to different location. Hope It Help!!

balavignesh_nag · ‎08-18-2017

@Saurab Dahal Yes its achievable. But there are few tweeks which has to be done. Partitioned table should be created with additional field("month") along with sale_date. Create the hive table with month as partitioned column. When inserting into the table, extract only the month from sales_date and pass it to insert statement. Insert into table table_name partitioned(month) select col1,col2,MONTH(sales_date),sale_date from source_table; Above command should work. Make sure the below property is enabled. set hive.exec.dynamic.partition=true; set hive.exec.dynamic.partition.mode=nonstrict Hope It helps!

balavignesh_nag · ‎08-17-2017

@palgaba You mean without major issue?

balavignesh_nag · ‎08-17-2017

@pavan p If it answers you question please choose it as a best answer!

balavignesh_nag · ‎08-17-2017

@Mallik Sunkara Are you able to query the table before altering the column name? I believe that you are facing this issue only after you have renamed the column name? Is my understanding is right?

balavignesh_nag · ‎08-16-2017

Thanks @Shawn Weeks

balavignesh_nag · ‎08-16-2017

@Greg Keys Thanks. Actually I have implemented already like the way which you have mentioned. But following this is easier for files with less no of columns. But I wanted to build a job which should be easier to handle even if we have large no of columns. I such case I came across multi-delimiter & regex. Wanted to know how it can be implemented.

balavignesh_nag · ‎08-14-2017

@Luis Ruiz Are you trying to insert from one table to another and facing this issue? If that is the case then when you are inserting from one table to another then you may need to convert the columns into MAP as you have used MAP in the second table.

balavignesh_nag · ‎08-14-2017

I have a file which has data like follows: provider_specialty,provider_specialty_desc 1,General,Practice|third column 2,General Surgery|third column 3,Allergy/Immunology|third column 4,Otolaryngology|third column 5,Anesthesiology|third column 6,Cardiology|third column 7,Dermatology|third column 8,Family Practice|third column 9,Interventional Pain Management|third column It has 3 columns in which first column and second column is separated by ','. Second column and third column is separated by '|'. Also few values in the second columns has delimiter as value. Is it possible to attain this in hive using multidelimiter. Could anyone help in providing the logic to create the table for the data provided above.

Online	Offline
Last Visited	‎10-03-2019 09:01 AM

Member Since	‎05-02-2017 01:47 PM
Last Visited	‎10-03-2019 09:01 AM
Posts	360
Kudos received	64

Cloudera Community

Re: what is the best way to get ftp file to hdfs c...

Re: when yarn communicates with the namenodes when...

Re: [TEZ] are partition, sort and shuffle built-in...

Re: CASE statement Error in Beeline HIVE

Re: hive query to display Week of the timestamp an...

Re: Is Spark a good solution to insert/update data...

Re: FileAlreadyExistsException when inserting in t...

Re: Create hive partition by month

Re: Create empty table if no data in source table

Re: consequences of changing block size?

Re: renaming the parquet column

Re: Multi-delimiter in Hive using regex

Re: Multi-delimiter in Hive using regex

Re: Hive Insert into table issue

Multi-delimiter in Hive using regex