About dhieru

dhieru · ‎11-07-2017

@Bryan Bende Thanks a lot for the response appreciate it, yes you re absolutely correct about, I have 4 listenudp merging on to 2 merge processors in serial and the puthdfs. I am batching the message for throughput in each of the listenUDP processors and the default matching delimiter is \n so sometimes "This is message 1\n\nThis is message 2\n" this happens. What I am looking for is this pattern in hdfs This is message 1 This is message 2 however right now it is writing to hdfs as This is message 1 \n This is message 2 here the extraline takes the additional memory. Any way we can avoid it? Thanks Dheery

dhieru · ‎11-07-2017

@Abdelkrim Hadjidj Thanks a lot solidifies my understanding, appreciate it

dhieru · ‎11-07-2017

Hi All, Thanks a lot to this aweosome community. We have a listenTCP and listenUDP processors listening for events. We have set "Max Batch Size" to 20000 increase throughput. However sometime in between messages a new line chacrater is added in a flowfile. Is it because of Batching Message delimiter? I guess no beause it is happening in between the messages in a flowfile. any suggestions? Thanks Dheeru

dhieru · ‎11-07-2017

@Matt Burgess need your advise Thanks Dheeru

dhieru · ‎11-07-2017

Hi All, Thanks to this awesome community. I am trying to understand how does a querydatabasetable works behind the scenes. I have not scheduled the porcessor to run. So does it keeps querying to database? keeps polling? or runs on some default schedule if we do not provide one. From the documentation it says processor is intended to run on primary node only, any specific reason? Any suggestions to clarify my doubt here. Thanks Dheeru

dhieru · ‎11-06-2017

Hi All, Thanks a lot to this awesome community. I am trying to replicate a directory structure from where I am listing and fetching the files using list and fetchsftp I did a PoC and I can see it stores the filename but not sure if it can store the directory structure if I give the top level direcotry to list. Any suggestion or help Thanks

dhieru · ‎11-01-2017

@Shu Appreciate your help This is as clear an explanation as it can get Thanks again

dhieru · ‎11-01-2017

@Shu Thanks a lot for confirming, appreciate it.

dhieru · ‎10-31-2017

@Wynner Thanks a lot for theconfirmation, yes I set nifi.cluster.node.protocol.max.threads=50 and nifi.cluster.node.protocol.threads=49, I have 5 nodes in a cluster 16 cores

dhieru · ‎10-31-2017

Hi All, Thanks a lot this awesome community. I read tha JOIN queries are not possible in NiFi, https://community.hortonworks.com/questions/64667/joining-tables-within-nifi.html so I thought of a workaround by creating a view in the database and then queriyng the view so I will be using tablename as the view name in the properties of querydatabasetable, I do not have a sample database, is this something which can be done? Thanks

Online	Offline
Last Visited	‎10-10-2019 12:00 AM

Member Since	‎06-26-2017 06:54 PM
Last Visited	‎10-10-2019 12:00 AM
Posts	191
Kudos received	10

Cloudera Community

Re: ListHDFS - Hadoop User Login, Non-Kerberos

Re: listenudp listentcp newline gets added in betw...

Re: QueryDatabaseTable internal workings

listenudp listentcp newline gets added in betwenn ...

Re: QueryDatabaseTable internal workings

QueryDatabaseTable internal workings

does listsftp anf fetchsftp stores the directory s...

Re: Joining tables by creating a view and querying...

Re: Joining tables by creating a view and querying...

Re: Suggested value for properties "nifi.cluster.n...

Joining tables by creating a view and querying the...