@sagar pavan, instead of using: --target-dir '/user/tsldp/patelco/' try: --warehouse-dir '/user/tsldp/patelco/' Each table will then be in a subdirectory under '/user/tsldp/patelco/' Cheers, Steven. ... View more

@Mahesh Mallikarjunappa The Hive serde can be used to load data into HBase via Hive. A simple example follows: -- Create a hive-managed HBase table CREATE TABLE MyHBaseTable(MyKey string, Col1 string, Col2 string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,colfam:col1,colfam:col2") TBLPROPERTIES("hbase.table.name" = "MyNamespace:MyTable") ; -- Insert data into it INSERT INTO TABLE MyHBaseTable SELECT SourceKey, SourceCol1, SourceCol2 FROM SourceHiveTable ; And from Pig, you can read form that same source and write to that same target using the HBase serde as follows: pig -useHCatalog -f script.pig Where script.pig is as below RawData = LOAD 'SourceHiveTable' USING org.apache.hive.hcatalog.pig.HCatLoader(); KeepColumns = FOREACH RawData GENERATE SourceKey, SourceCol1, SourceCol2; STORE KeepColumns INTO 'hbase://MyNamespace:MyTable' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('colfam:col1,colfam:col2'); Note: You don't specify the key in the STORE statement - the first column is always the key. Hope this helps! ... View more

@Emily Sharpe I believe your issue relates to the way Pig is processing the NULL Avro data. Rather than ignoring those NULL values, Pig passes the key and an empty value to HBase, which dutifully stores it. To avoid storing these values, filter them out. The following Pig code shows how to do this for a single key/value Avro source: ImageAvro = LOAD '/path/to/RawAvroData' USING org.apache.pig.piggybank.storage.avro.AvroStorage ('no_schema_check', 'schema_file', '/path/to/AvroSchemaFile.avsc'); filteredImage = FOREACH (FILTER ImageAvro BY SIZE(ImageColumn) > 0) GENERATE KeyColumn, ImageColumn; STORE filteredImage INTO 'hbase://namespace:table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('colFamily:col'); Similarly, you can identify the empty cells with the same FILTER operation. Here's how to save a list of keys that have an empty colFamily:col cell: ImageHBase = LOAD 'hbase://namespace:table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('colFamily:col', '-loadKey true') as (KeyColumn:chararray, ImageColumn:bytearray); NullImage = FOREACH (FILTER ImageHBase BY SIZE(ImageColumn) == 0) GENERATE KeyColumn; STORE NullImage into '/path/to/flat/file' USING PigStorage; ... View more