Member since
09-28-2016
59
Posts
15
Kudos Received
0
Solutions
05-15-2020
07:15 PM
hello! If I insert a string containing 'or "or, PutSQL to Phoenix will be return the grammatical errors, this should be how to solve?
... View more
04-24-2020
07:45 AM
I have some sort of the same scenario. But I have to route out on department value, like department="IT" route to some other flow and for department="DB" route to another flow. For that, I have written out expression as $.[?(@.department=="DB")]. But this doesn't work. Please suggest for the above scenario. And also if we have to apply some conditions like route data for a>10 like that.
... View more
10-23-2017
12:05 PM
How have you set up your shell script to handle closing on job timeout?
I'm not aware of ExecuteStreamCommand having a hidden timeout property, though you could add something like a 'max wait' timeout if you wanted to do a PR.
... View more
07-28-2017
11:03 AM
@srini I couldn't find any special exception handling behavior implemented for pyspark. But you can do a native python try/except in utility.function1, returning None if there is an exception and then filter the resulting rdd on that. (or handle as you want to)
... View more
04-04-2017
03:34 PM
Sounds like the output of a MapReduce job that you ran. HBase does not store anything outside of the value of "hbase.rootdir" in hbase-site.xml. On HDP, this defaults to "/apps/hbase/data". As long as you have no changed this configuration property, you can rest assured that HBase is not actively referring to files stored in "/tmp".
... View more
06-27-2019
12:19 AM
The above question and the entire reply thread below were originally posted in the Community Help track. On Thu Jun 27 00:18 UTC 2019, a member of the HCC moderation staff moved it to the Hadoop Core track. The Community Help Track is intended for questions about using the HCC site itself, not technical questions about RegionServers or Hbase.
... View more
01-26-2017
05:36 PM
1 Kudo
@srini
The default "FlowFIle Policy" is "use clone" within the advanced UI. For an "and" operation, you will want change that to "use original". Just keep in mind (does not apply here since both rules are unique) that if more then one rule tries to perform the exact same "Actions", only the last rule in the list that will be applied to the original FlowFile on output. Both "Rules" are run against every FlowFile that passes through this UpdateAttribute processor, so it will work in cases where only one or the other is set and cases where both are not set. Rules are operated on in the order they are listed. You can drag rules up and down in the list to order them how you like. If this answer addresses your question, please click "accept". Thank you, Matt
... View more
12-15-2016
08:49 PM
2 Kudos
Hi @srini, One way to do it is to use the MonitorActivity processor that can emit a flow file if there is no activity since X minutes. This generated flow file can then be routed to a processor sending emails. Hope this helps.
... View more
12-14-2016
05:16 AM
What benefit you are expecting by archiving hbase data like "Hadoop Archive"? or Is your purpose to just archive HBase data in any form?
... View more
11-07-2016
02:09 AM
1 Kudo
Hi
@srini,
NiFi does support the addition of custom transformations which can be referenced with a "drop in" jar. You could create your own transformation class, which extends the jolt library, that provides the functionality you need and be available to NiFi on the file system. Given that this is a new function in a later version of Jolt I would be careful using that the as a custom jar; that may cause a conflict but I haven't tested this myself to be sure. Below are some use cases on how to apply custom functions in NiFi
Case 1 – Custom Transform Selected
In this case if the Custom option of Transform is selected then 1) a Custom Transform Class Name should be entered and 2) one or more module paths should be provided. The Custom Transform Class Name should be a fully qualified classname (e.g. eu.zacheusz.jolt.date.Dater). The Module Path can take a comma delimited list of directory locations or one or more jar files. Once these fields are populated the Advanced view will support validation and saving of the specification. A user can switch between transformation types in the UI but not custom class names & module paths.
Case 2 – Chain Transformation Selected with Custom Transformations embedded
In this case if you wants to use one or more transforms (that include custom transformations) in your specification then the Chainr spec option can be used with one or more module path provides. In this case the Custom Transform Class Name property is not required and would be ignored (since one or more custom transformations could be invoked in the spec). As in Case 1 the Advanced view will support validation and saving of specification if the required fields are populated for this case. I hope this helps! Please let me know if you have any questions.
... View more