Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HBase to Hive showing null values for decimal data type


HBase to Hive showing null values for decimal data type

Prblm statement:


I have a table which collects raw data from server and stores in mysql. i want to move to hbase. i moved using sqoop from mysql to hbase.


my work is to write impala query for calculating percentile function(95th and 97th ).


I sqoop mysql to hbase . create external table in hive . one column which is of decimal (32,1) is having null values.


create external table nrd_app_spt.cap(hbid string,MATRIX_col string,MATRIX_VALUE_col double,COUNTRY_col string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,RAW:MATRIX,RAW:MATRIX_VAL,RAW:COUNTRY")TBLPROPERTIES("" = "nrd_app_spt:capacity_new");


I have a raw data in MySQL table with 5 columns. hostname string, timestamp string,country string, collec string, matrixvalue decimal(32,1). I sqoop and moved to HBase.


Doubt 1: which column is to set row key in hbase.?all column values are not unique. same hostname same date and time same country same collec i will get a matrix value. so there is no uniqueness. so i kept hostname and timestamp as row key.


Is that a good approach?

any other suggestion?

If i want to write a query then 2 columns will be as row key how to do it ?


Doubt 2:


Fetching null values for decimal (32,1)


Doubt 3:


I need to calculate percentile (95 and 97 ) for this column of data type decimal(32,1). is that possible in impala or hive? percentile function for all rows .

any udf function need to write or any built in function available?