- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to put Json data as a Json format in HBase
- Labels:
-
Apache HBase
-
Apache NiFi
Created ‎03-21-2018 11:08 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Please tell me how to store json multiple line data in hbase from NiFi.
Created on ‎03-22-2018 09:42 AM - edited ‎08-17-2019 11:29 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can use PutHbasecell processor for this use case and keep the Row Identifier as UUID then you can get json format message inserted for the uuid.
Example:-
my input json document is
{"id":"1334134","name":"Apparel Fabric","path":"Arts, Crafts & Sewing/Fabric/Apparel Fabric"}
PutHbasecell configs:-
as you can see in the above screenshot i'm having Row Identifier as ${UUID()} because this uuid is unique for each flowfile in NiFi so that we are not overwriting any existing data in hbase table.
Output:-
hbase(main):008:0> scan 'test' ROW COLUMN+CELL c7ca74ad-4933-4340-a9c7-e55370a4501b column=category:category:details, timestamp=1521711352302, value={"id" : "1334134","name" : "Apparel Fabric","path" : "Arts, Crafts & Sewin g/Fabric/Apparel Fabric"} 1 row(s) in 0.1130 seconds
Case2:-
If your input json document is
{"id":"1334134","name":"Apparel Fabric","path":"Arts, Crafts & Sewing/Fabric/Apparel Fabric"}, {"id":"412","name":"Apparel Fabric","path":"Arts, Crafts & Sewing/Fabric/Apparel Fabric"}
Then in hbase the document looks like
Created ‎03-22-2018 12:20 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That's expected behavior from Hbase table because we are having unique Row Key for each record in Hbase table now. But your data that you are writing to this category1 table is duplicated data and Hbase only overwrites the existing data if it found same Row key already exists in the table. But in your case we are using UUID i.e unique id for the each flowfile in NiFi, so we will have unique id for each flowfile (although the content of the flowfile is same).
Created on ‎03-22-2018 12:09 PM - edited ‎08-17-2019 11:28 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
i hve tried uuid() function but it repeat records like :-
Created ‎05-21-2021 02:49 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @umang_instantwe ,
Could you please suggest how the same record can be put in HBase via HBase shell?
{"id" : "1334134","name" : "Apparel Fabric","path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}
Ex. Below query does not work.
put 'table_name','row_key','cf:n1','{"id" : "1334134","name" : "Apparel Fabric","path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}', timestamp
Thanks,
Priyanshu
Created ‎05-21-2021 03:43 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @priyanshu_soni
You can skip the "Timestamp" part as the same is inserted by HBase implicitly. I tried the same Query as you, excluding the Timestamp & the same was Successful:
hbase(main):018:0> put 'Table_X1','125','Cf1:CheckItem','{"ID" : "1334134","Name" : "Apparel Fabric","Path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}'
Took 0.0077 seconds
hbase(main):019:0> scan 'Table_X1'
ROW COLUMN+CELL
125 column=Cf1:CheckItem, timestamp=1621593487680, value={"ID" : "1334134","Name" : "Apparel Fabric","Path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}
1 row(s)
Took 0.0057 seconds
As you may see above, the "timestamp" field corresponds to the Epoch Timestamp of the Inserted Row Time of Operation. If you wish to explicitly specify the Timestamp, You can include a EpochTime as shared below:
hbase(main):020:0> put 'Table_X1','126','Cf1:CheckItem','{"ID" : "1334134","Name" : "Apparel Fabric","Path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}',1621593487680
Took 0.0202 seconds
hbase(main):021:0> scan 'Table_X1'
ROW COLUMN+CELL
125 column=Cf1:CheckItem, timestamp=1621593487680, value={"ID" : "1334134","Name" : "Apparel Fabric","Path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}
126 column=Cf1:CheckItem, timestamp=1621593487680, value={"ID" : "1334134","Name" : "Apparel Fabric","Path" : "Arts, Crafts & Sewing/Fabric/Apparel Fabric"}
2 row(s)
Took 0.0071 seconds
Let us know if you have any issues with the Put Operation.
- Smarak

- « Previous
-
- 1
- 2
- Next »