Skip to main content
itdm_bmi
itdm_bmi
Silver ⭐️⭐️⭐️
July 27, 2023
Solved

Cannot validate serde: org.apache.hadoop.hive.serde2.MultiDelimitSerDe

Hi,

 

I got an error when creating external table with multiple delimiter such as "||".

It says "Cannot validate serde: org.apache.hadoop.hive.serde2.MultiDelimitSerDe". I have tried add jar hive-contribute.3.1.3.jar, but still not working and also using regex.

My Cloudera Version is CDH 6.1.1.

How to solve this problem so i can achieve to set multiple delimiter in hive create external table?

Please help.

 

Sincerely,

 

Gideon Maruli

Bank Mayapada


cannot validate serde.jpeg
udeshmukh
udeshmukhAnswer
Gold ⭐️
July 28, 2023

Hi @itdm_bmi 

After adding the jar , 

 

Instead of 

ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.MultiDelimitSerde'

 

Please try 

ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe'

View in thread

8 replies

udeshmukh
Gold ⭐️
udeshmukh
Gold ⭐️
July 27, 2023

Hi @itdm_bmi 

 

Can you share the output of 

 

ls -l /opt/cloudera/parcels/CDH/jars| grep -i contrib

 

Also, may i know what exact add command you executed ?

Translate
itdm_bmi
itdm_bmiAuthor
Silver ⭐️⭐️⭐️
July 27, 2023

Hi @udeshmukh,

 

Here's the output

itdm_bmi_1-1690446591120.png

I had tried these 2 commands

1. add jar /usr/lib/hive/lib/hive-contrib-3.1.3.jar;
2. add jar /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/jars/hive-contrib-3.1.3.jar;

 

Sincerely,

 

 

Gideon Maruli

IT Data Management - Bank Mayapada

Translate
udeshmukh
Gold ⭐️
udeshmukh
Gold ⭐️
July 27, 2023

Hi @itdm_bmi 

 

Please try to add the 2nd jar "hive-contrib-2.1.1-cdh6.1.1.jar" 

Translate
itdm_bmi
itdm_bmiAuthor
Silver ⭐️⭐️⭐️
July 27, 2023

I did, but still get same error.


error1.PNG
Translate
itdm_bmi
itdm_bmiAuthor
Silver ⭐️⭐️⭐️
July 28, 2023

Hi @udeshmukh,

 

Can you help me with another solution?

 

Sincerely,

 

Gideon Maruli

Bank Mayapada

Translate
udeshmukh
Gold ⭐️
udeshmukhAnswer
Gold ⭐️
July 28, 2023

Hi @itdm_bmi 

After adding the jar , 

 

Instead of 

ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.MultiDelimitSerde'

 

Please try 

ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe'

Translate
VidyaSargur
Community Manager
VidyaSargur
Community Manager
July 31, 2023

@itdm_bmi, Have the replies from @udeshmukh helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.

 

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
itdm_bmi
itdm_bmiAuthor
Silver ⭐️⭐️⭐️
August 1, 2023

Hi,

 

Sorry to make you waiting. Yes, that solution resolved the issue.

But i have 1 more issue.

After the data successfully inserted to textfile table. Then i need to insert the data to parquet table from textfile table. But it failed insert to parquet table. Error message: "[FATAL] 10:25:08 vaproject.vmlog_0_4.VMLOG- tHiveRow_1 Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask".

What is wrong or missing? Please need your help. Thank you.

 

Sincerely,

 

Gideon Maruli

IT Data Management - Bank Mayapada


Error MapRedTask.JPG
Translate
Terms & ConditionsAccessibility statement