@srini one thing you could try is instead of using that one attribute to bucket on, create another attribute that denormalizes all of the attributes into one string, and use that column to bucket on (still leave the other attributes in place). When you have duplicate columns this would lead to those dupes being bucketed under that one column in the subsequent operation. Then the rest of the operations would pick the unique one and then remove the denormalized column. It's a bit of a dance but I think could work. Does that makes sense? One thing I think would be good to get on the radar is upgrading Jolt in NiFi, perhaps once the modify feature upgrades from beta. I think that will help to simplify some of the hoops needed to do this type of work. ... View more

Yes, the length of the row, not the entire key. The row component in the keys that you provided is "row8". Please refer to the HBase documentation if you are unclear on what components make up an HBase Key. ... View more

This post will show you how to very quickly (seconds) build a flow that logs nifi at the processor and error level (or however you wish to customize). Very handy for development and production as well: https://community.hortonworks.com/articles/65027/nifi-easy-custom-logging-of-diverse-sources-in-mer.html ... View more

@Greg Keys Your blog is great to clarify a bit the approach one can use, however i run into problems with diverse format of logs that span through multiple lines and then with this approach one logline can be split in multiple flowfiles which is not desired. Is there a way to use SplitText to split the files in specific patterns? ... View more

Thank you . I am refering this link https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HConstants.html#MAX_ROW_LENGTH what i understood is MAX_ROW_LENGTH is about whole row not only key . Here is my confusion , how can i take this property is only belongs to key ? ... View more

@Josh Elser @srinivas padala The "Read/Write" stats on the processor have nothing to do with writing to your SQL end-point. This particular stat is all about reads from and writes to the NIFi content Repository. This helps identify where in your flow you may have disk high disk I/O in the form of either reads or more expensive writes. From the screenshot above, I see that this processor brought in off inbound connections 35,655 FlowFiles in the past 5 minutes. It read 20.87 MB of content from the content repository in that same timeframe. The processor then output 0 FlowFiles to any outbound connection (This indicates all files where either routed to a an auto-terminated relationship). Assuming only the "success" relationship was auto-terminated, all data was sent successfully. If the "failure" relationship (which should not be auto-terminated here) is routed to another processor, the 0 "out" indicates that in the past 5 minutes 0 files failed. The Tasks shows a cumulative total CPU usage reported over the past 5 minutes. A high "Time" value indicates a cpu intensive processor. Thanks, Matt ... View more

Hello, I think you need to use SplitText and SplitContent. SplitText can split lines, then pass each line to SplitContent, which can be configured delimiter by hexadecimal format as "Byte Sequence". Semicolon ";" is "3B". Hope this will work for you. ... View more

Can you provide the full code?I cant know how to get the 'indexList' and so on,Thank you. ... View more