Member since
06-26-2017
191
Posts
10
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1981 | 09-22-2017 07:13 PM |
10-31-2017
07:39 PM
@Matt Burgess Can we do join query here from differen tables. I looked at the properties for QueryDatabase table, there is no property for entering query? Any help when query is join query from 2 different tables
... View more
10-31-2017
01:51 PM
@Matt Burgess Thanks for the prompt reply. I will try and confirm it Thanks again
... View more
10-31-2017
01:37 PM
I tried using this temp_${now():format("yyyy-MM-dd-HH-mm-ss"):join(${random():mod(10):plus(1)})}, this time it is a valid EL expressioon but not generating the random number still
... View more
10-31-2017
01:25 PM
Hi All, I am creating a new file name based on the time stamp and concatenate with random number but not able figure out the el expression for it temp_${now():format("yyyy-MM-dd-HH-mm-ss"):join(random():mod(10):plus(1))}, however this expression is not valid EL expression Thanks Dheeru
... View more
Labels:
- Labels:
-
Apache NiFi
10-30-2017
09:00 PM
@Matt Clarke Thanks for the response, yes I set the max timer driven event count to 40. I might have to increase it. I will set the number of flow files to 15000 to 20000 and I shold also set the max age, so that no flow files are lingering on, will comeback with results. Thanks Dheeru
... View more
10-30-2017
08:22 PM
Hi @Matt Clarke Thanks a lot, appreciate your help I did the same thing , need your suggestion here https://community.hortonworks.com/questions/144575/mergecontent-processor-and-default-block-size-of-h.html In addition , I need flow files based on size (120 to 128 MB default size) however it is forcing me to add the "min number entries " I also read somewhere that is a "and" condition, in this link https://community.hortonworks.com/content/kbentry/1876/mergecontent-processor-inner-workings.html Thanks Dheeru
... View more
10-30-2017
07:15 PM
Hi All, Thanks a lot to this awesome community We have a high event per second data source feeding to mergecontent using listenTCP (15000 event per second), right now I have 3 mergecontent processors in serial, first one 32 MB second one to merge till 64 mb and last one to 128 MB (default block size) , however the message are still getting queued up on the success of listentcp. the settings in the success queue is 100000 message and 2 GB size limit. should I add 2 more mergecontent in serial to merge 8 mb and then 16 mb? also there are 2 concurrent processes for each mergeocntent processor I have done all the settings to make our cluster a high perfromance cluster read a blog on this community https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.html any help or suggesttions Thanks Dheeru
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache NiFi
10-30-2017
06:47 PM
@Shu Thanks for your response. that helps. However I am wondering if my mergecontent processor is waiting. Probably it is waiting for 128 MB to flush, then why not clear queue? because now it is with mergecontent processor Thanks Dheeru
... View more
10-30-2017
05:15 PM
success.pngmereg1.pngHi All Thanks a lot this awesome community I am facing this problem a lot of time The success between listentcp and mergeconten gets filled up and starts showing in red attached is the image When I click and do list queue nothing appears I have configure mergecontent to merge till 128 mb and using text as delimiter. Any help what is happening Thanks Dheeru
... View more
Labels:
- Labels:
-
Apache NiFi