Former Member
Posts: 0

HBase to Hive showing null values for decimal data type

Prblm statement:


I have a table which collects raw data from server and stores in mysql. i want to move to hbase. i moved using sqoop from mysql to hbase.


my work is to write impala query for calculating percentile function(95th and 97th ).


I sqoop mysql to hbase . create external table in hive . one column which is of decimal (32,1) is having null values.


create external table nrd_app_spt.cap(hbid string,MATRIX_col string,MATRIX_VALUE_col double,COUNTRY_col string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,RAW:MATRIX,RAW:MATRIX_VAL,RAW:COUNTRY")TBLPROPERTIES("" = "nrd_app_spt:capacity_new");


I have a raw data in MySQL table with 5 columns. hostname string, timestamp string,country string, collec string, matrixvalue decimal(32,1). I sqoop and moved to HBase.


Doubt 1: which column is to set row key in hbase.?all column values are not unique. same hostname same date and time same country same collec i will get a matrix value. so there is no uniqueness. so i kept hostname and timestamp as row key.


Is that a good approach?

any other suggestion?

If i want to write a query then 2 columns will be as row key how to do it ?


Doubt 2:


Fetching null values for decimal (32,1)


Doubt 3:


I need to calculate percentile (95 and 97 ) for this column of data type decimal(32,1). is that possible in impala or hive? percentile function for all rows .

any udf function need to write or any built in function available?