Support Questions

umang_instantwe · ‎03-21-2018

Please tell me how to store json multiple line data in hbase from NiFi.

Shu_ashu · ‎03-22-2018

You can use PutHbasecell processor for this use case and keep the Row Identifier as UUID then you can get json format message inserted for the uuid.

Example:-

my input json document is

{"id":"1334134","name":"Apparel Fabric","path":"Arts, Crafts & Sewing/Fabric/Apparel Fabric"}

PutHbasecell configs:-

as you can see in the above screenshot i'm having Row Identifier as ${UUID()} because this uuid is unique for each flowfile in NiFi so that we are not overwriting any existing data in hbase table.

Output:-

hbase(main):008:0> scan 'test'
ROW                                             COLUMN+CELL
 c7ca74ad-4933-4340-a9c7-e55370a4501b           column=category:category:details, timestamp=1521711352302, value={"id" : "1334134","name" : "Apparel Fabric","path" : "Arts, Crafts & Sewin
                                                g/Fabric/Apparel Fabric"}
1 row(s) in 0.1130 seconds

Case2:-

If your input json document is

{"id":"1334134","name":"Apparel Fabric","path":"Arts, Crafts & Sewing/Fabric/Apparel Fabric"},
{"id":"412","name":"Apparel Fabric","path":"Arts, Crafts & Sewing/Fabric/Apparel Fabric"}

Then in hbase the document looks like

View solution in original post

Shu_ashu · ‎03-22-2018

@umang s

That's expected behavior from Hbase table because we are having unique Row Key for each record in Hbase table now. But your data that you are writing to this category1 table is duplicated data and Hbase only overwrites the existing data if it found same Row key already exists in the table. But in your case we are using UUID i.e unique id for the each flowfile in NiFi, so we will have unique id for each flowfile (although the content of the flowfile is same).

umang_instantwe · ‎03-22-2018

i hve tried uuid() function but it repeat records like :-

priyanshu_soni · ‎05-21-2021

Hi @umang_instantwe ,

Could you please suggest how the same record can be put in HBase via HBase shell?

{"id" : "1334134","name" : "Apparel Fabric","path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}

Ex. Below query does not work.

put 'table_name','row_key','cf:n1','{"id" : "1334134","name" : "Apparel Fabric","path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}', timestamp

Thanks,

Priyanshu

smdas · ‎05-21-2021

Hello @priyanshu_soni

You can skip the "Timestamp" part as the same is inserted by HBase implicitly. I tried the same Query as you, excluding the Timestamp & the same was Successful:

hbase(main):018:0> put 'Table_X1','125','Cf1:CheckItem','{"ID" : "1334134","Name" : "Apparel Fabric","Path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}'
Took 0.0077 seconds                                                                                                                                                                                                                                                                                                           
hbase(main):019:0> scan 'Table_X1'
ROW                                                                              COLUMN+CELL                                                                                                                                                                                                                                  
 125                                                                             column=Cf1:CheckItem, timestamp=1621593487680, value={"ID" : "1334134","Name" : "Apparel Fabric","Path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}                                                                                     
1 row(s)
Took 0.0057 seconds

As you may see above, the "timestamp" field corresponds to the Epoch Timestamp of the Inserted Row Time of Operation. If you wish to explicitly specify the Timestamp, You can include a EpochTime as shared below:

hbase(main):020:0> put 'Table_X1','126','Cf1:CheckItem','{"ID" : "1334134","Name" : "Apparel Fabric","Path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}',1621593487680
Took 0.0202 seconds                                                                                                                                                                                                                                                                                                           
hbase(main):021:0> scan 'Table_X1'
ROW                                                                              COLUMN+CELL                                                                                                                                                                                                                                  
 125                                                                             column=Cf1:CheckItem, timestamp=1621593487680, value={"ID" : "1334134","Name" : "Apparel Fabric","Path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}                                                                                     
 126                                                                             column=Cf1:CheckItem, timestamp=1621593487680, value={"ID" : "1334134","Name" : "Apparel Fabric","Path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}                                                                                     
2 row(s)
Took 0.0071 seconds

Let us know if you have any issues with the Put Operation.

- Smarak

Cloudera Community

Support Questions

How to put Json data as a Json format in HBase