Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Transform Hive Data to Druid and 6 Lakes Records are missing

Transform Hive Data to Druid and 6 Lakes Records are missing

Explorer

Hi Team,


We are not able to transform Hive Data to Druid.


Total records are 8L in Hive, only 2L records are transform to Druid and missing some 6 L records.

Even after adding Cast to every column of hive.


If i do count(*) on both tables some 6L records are missing in Druid tables.


Count of Druid table - 200392
Count of Hive table - 794705



Below is my Hive external table.



CREATE EXTERNAL TABLE afssales006

STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'

AS

select

select

current_timestamp() as `__time`,

CAST(`transactiondate` as STRING) transactiondate,

CAST(`airlinecode` AS STRING) airlinecode,

CAST(`airlinename` AS STRING) airlinename,

CAST(`billtocode` AS STRING) billtocode,

CAST(`shiptocode` AS STRING) shiptocode,

CAST(`shiptoname` AS STRING) shiptoname,

CAST(`locationid` AS STRING) locationid,

CAST(`customertype` AS STRING) customertype,

CAST(`loc_name` AS STRING) loc_name,

CAST(`loc_code` AS STRING) loc_code,

CAST(`aircarfttype` AS STRING) aircarfttype

from tab_sales where billtocode is not NULL;


Please help me how to solve this issue.

Don't have an account?
Coming from Hortonworks? Activate your account here