Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Anishkumarv
Anishkumarv
Contributor
My Accepted Solutions
Show All

Re: Zookeeper update

by Contributor Anishkumarv in Support Questions
‎04-06-2019 03:09 AM
‎04-06-2019 03:09 AM
Can you share further more details to help further.   Cloudera bundle version zookeeper version. Complete logs to cross check the error messages.     With Regards Anishkumar Valsalam ... View more

Re: How to calculate Nifi threads.

by Contributor Anishkumarv in Support Questions
‎04-06-2018 05:27 PM
‎04-06-2018 05:27 PM
@Matt Clarke Can you explain.. how we can ensure the remaining threads was using by other components Ex :controller services / Reporting tasks. If you explain that really helpful to close this thread. ... View more

Re: How to calculate Nifi threads.

by Contributor Anishkumarv in Support Questions
‎04-05-2018 05:09 PM
‎04-05-2018 05:09 PM
@Matt Clarke How we can ensure that ? the remaining threads was using by other Ex :controller services / Reporting tasks. ... View more

Re: How to calculate Nifi threads.

by Contributor Anishkumarv in Support Questions
‎04-05-2018 04:57 PM
‎04-05-2018 04:57 PM
@Matt Clarke Recently we didn't do any update and we have tried with restart also but still we are seeing differences in the thread values in status bar. What actually we configured. What actually running. How this drastic differences ? pls help to close this. ... View more

Re: How to calculate Nifi threads.

by Contributor Anishkumarv in Support Questions
‎04-05-2018 03:49 AM
‎04-05-2018 03:49 AM
We are running only two processor group and the overall usage you can see in the 2 PG ( 8 + 4). Hdf - 3.0.1.0-43 Maximum Timer Driven Thread Count = 100 Max Event Driven Thread Count = 5 we have two nodes but it not configured as cluster. ... View more

Re: How to calculate Nifi threads.

by Contributor Anishkumarv in Support Questions
‎04-05-2018 03:40 AM
‎04-05-2018 03:40 AM
We are running only two processor group and the overall usage you can see in the 2 PG ( 8 + 4). Hdf - 3.0.1.0-43 Maximum Timer Driven Thread Count = 100 Max Event Driven Thread Count = 5 we have two nodes but it not configured as cluster. ... View more

Re: How to calculate Nifi threads.

by Contributor Anishkumarv in Support Questions
‎04-04-2018 07:24 PM
‎04-04-2018 07:24 PM
@Bryan Bende experts inputs required. ... View more

Re: How to calculate Nifi threads.

by Contributor Anishkumarv in Support Questions
‎04-04-2018 04:55 PM
‎04-04-2018 04:55 PM
@Matt Clarke Hi Bro need your inputs. ... View more

How to calculate Nifi threads.

by Contributor Anishkumarv in Support Questions
‎04-04-2018 04:51 PM
‎04-04-2018 04:51 PM
Hi Team, How to calculate Nifi threads. ? because in the screenshot it showing totally 68 threads its using but the actual used is only 12 so how to find the remaining threads details ?? Kinldy help on this. ... View more
Labels:

Re: VM for Nifi Prod

by Contributor Anishkumarv in Support Questions
‎03-27-2018 06:04 PM
‎03-27-2018 06:04 PM
@Matt Clarke Hi Matt, We are having an internal argument on whether its a good idea to keep VMS for production NIFI need your inputs on this What are the pros and cons of each hardware configuration? ... View more

VM for Nifi Prod

by Contributor Anishkumarv in Support Questions
‎03-27-2018 05:53 PM
‎03-27-2018 05:53 PM
Hi Team, Why Virtual Machine is not advisable for Nifi Production environment ? ... View more
Labels:

VM for Nifi prod

by Contributor Anishkumarv in Support Questions
‎03-27-2018 05:51 PM
‎03-27-2018 05:51 PM
Hi Team, Why Virtual Machine is not advisable for Nifi Production environment ? ... View more
Labels:

Re: Nifi - capacity planning - moitoring

by Contributor Anishkumarv in Support Questions
‎02-22-2018 04:30 PM
‎02-22-2018 04:30 PM
@Matt Clarke Need your inputs bro. ... View more

Nifi - capacity planning - moitoring

by Contributor Anishkumarv in Support Questions
‎02-22-2018 04:29 PM
‎02-22-2018 04:29 PM
Hi Team, We are planning to move HDF nodes from some non hdf nifi instances befor moving we just want to calculate the current utilzation of utilization of Nifi (non HDF). for better planning. How to get that stats for example for last 1day nifi how much mem/cores used? ... View more

Re: Hadoop Clients - HDF Cluster

by Contributor Anishkumarv in Support Questions
‎11-25-2017 12:01 AM
‎11-25-2017 12:01 AM
@Bryan Bende @Matt Clarke Kindly help on the above issue. ... View more

Hadoop Clients - HDF Cluster

by Contributor Anishkumarv in Support Questions
‎11-24-2017 04:58 PM
‎11-24-2017 04:58 PM
Hi Team, Kindly share us the simple steps to install “Hadoop clients” on the HDF Cluster. Meanwhile, there are many applications which requires Hadoop clients that are deployed in HDF cluster and hence we are synchronizing those clients manually. The main drawback is that there may be missing of jars as part of manual synchronization. Hence, request you to share the steps for synchronization With Regards Anishkumar Valsalam ... View more

Re: Resource Allocation - Nifi Processor Group

by Contributor Anishkumarv in Support Questions
‎11-09-2017 03:46 PM
‎11-09-2017 03:46 PM
Hi team, Please advice on the above issue it will helpful for others too. ... View more

Re: Resource Allocation - Nifi Processor Group

by Contributor Anishkumarv in Support Questions
‎11-08-2017 04:59 PM
‎11-08-2017 04:59 PM
@Matt Clarke hi matt can you guide me to resolve this issue. ... View more

Resource Allocation - Nifi Processor Group

by Contributor Anishkumarv in Support Questions
‎11-08-2017 04:57 PM
‎11-08-2017 04:57 PM
Hi Team, Is there any option to allocate resourcces for nifi processor group? see in the below image we have allocated "Maximum Timer Driven Thread Count " 30cores all the 30 cores took the below processor itself becuase of that other PG not processing. is there any option available allocate resource for PG or set priority ? in nifi. Please guide me to close this issue. ... View more
Labels:

Re: RPG - Cluster not processing

by Contributor Anishkumarv in Support Questions
‎11-01-2017 06:38 PM
‎11-01-2017 06:38 PM
Hi Abdelkrim, Thanks its working now The same we can do in standalone as well right? in rpg also it taking from one node? how we can process from multiple nodes and balance the load. Why we need to put input port in outside of processor group? root path. ... View more

Re: RPG - Cluster not processing

by Contributor Anishkumarv in Support Questions
‎11-01-2017 06:09 PM
‎11-01-2017 06:09 PM
@Abdelkrim Hadjidj Thanks if the new file comes it processing and it fetch to hdfs but i am getting the below error also in Fetchsftp. 2017-11-02 01:56:41,942 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.standard.FetchSFTP FetchSFTP[id=788c7b35-015f-1000-0000-00000f3979fd] Failed to fetch content for StandardFlowFileRecord[uuid=af918b98-5632-4877-9876-221477204e73,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1509553663114-11, container=default, section=11], offset=677691, length=0],offset=0,name=a.txt,size=0] from filename /home/1548691/*.txt on remote host 10.20.174.137:22 due to java.io.IOException: Failed to obtain file content for /home/1548691/*.txt; routing to failure: java.io.IOException: Failed to obtain file content for /home/1548691/*.txt 2017-11-02 01:56:41,943 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.standard.FetchSFTP java.io.IOException: Failed to obtain file content for /home/1548691/*.txt at org.apache.nifi.processors.standard.util.SFTPTransfer.getInputStream(SFTPTransfer.java:300) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.processors.standard.FetchFileTransfer.onTrigger(FetchFileTransfer.java:236) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_60] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_60] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_60] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_60] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_60] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_60] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60] Caused by: com.jcraft.jsch.SftpException: /home/1548691/*.txt is not unique: [/home/1548691/b.txt, /home/1548691/HKLPATHAS03.txt, /home/1548691/a.txt] at com.jcraft.jsch.ChannelSftp.isUnique(ChannelSftp.java:2965) ~[jsch-0.1.54.jar:na] at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1314) ~[jsch-0.1.54.jar:na] at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1290) ~[jsch-0.1.54.jar:na] at org.apache.nifi.processors.standard.util.SFTPTransfer.getInputStream(SFTPTransfer.java:292) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] ... 13 common frames omitted @ how to process all set of files instead of .txt and it picking from only one node? how to pick from all 3 nodes ? ... View more

Re: RPG - Cluster not processing

by Contributor Anishkumarv in Support Questions
‎11-01-2017 05:52 PM
‎11-01-2017 05:52 PM
yes i have data in that. [root@HKLPATHAS03 1548691]# ll total 4 -rw-r--r-- 1 1548691 root 0 Nov 2 00:38 HKLPATHAS03.txt drwxr-xr-x 9 1548691 domain users 4096 Nov 2 00:27 nifi-toolkit-1.1.2 see it not taking that file to hdfs. [root@HKLPATHAS03 1548691]# sudo -u hdfs hadoop fs -ls /user/1548691/HKLPATHAS03.txt ls: `/user/1548691/HKLPATHAS03.txt': No such file or directory [root@HKLPATHAS03 1548691]# ... View more

RPG - Cluster not processing

by Contributor Anishkumarv in Support Questions
‎11-01-2017 05:45 PM
‎11-01-2017 05:45 PM
Hi Team, I am trying to configure the RPG in my cluster environemt (3nodes) how to distribute the data between the 3 nodes. This the below processor i am trying with RPG but it not processing at all attached the details FYR. Kindly help me to close this thread. ... View more
Labels:

Re: Nifi - RPG - does not have any input port erro...

by Contributor Anishkumarv in Support Questions
‎11-01-2017 03:20 PM
‎11-01-2017 03:20 PM
@Abdelkrim Hadjidj Thanks i am able to create policy now , I was checked into default flow policy so its was not enabled. In Input port policy it is enabled And In your flow i can see this is between cluster to cluster. https://community.hortonworks.com/articles/88473/site-to-site-communication-between-secured-https-a.html How we can test the same between 3 instances ( 3 node cluster) ? I am looking for the sample flows to test RPG in the cluster. Thanks Matt and Abdelkrim. ... View more

Re: Nifi - RPG - does not have any input port erro...

by Contributor Anishkumarv in Support Questions
‎10-31-2017 03:30 PM
‎10-31-2017 03:30 PM
Hi @Matt Clarke @Abdelkrim Hadjidj Thanks for the detailed note but stll the below 2 options are not enabled eventhough we careated polocies (Retrieve Site-To-Site Details) for nodes. -> receive data via site-to-site --> send data via site-to-site And please guide now in standalone cluster we are facing some load(mem / cpu) , In cluster do we face this kind of issue? or it will distribute the data and process so that the above mem/cpur load issue we can avoid. Is my understanding correct? ... View more

Re: Nifi after upgrade not starting - getting errr...

by Contributor Anishkumarv in Support Questions
‎10-21-2017 06:16 PM
‎10-21-2017 06:16 PM
@Bryan Bende @Pierre Villard Need your advice on this issue. ... View more

Re: Nifi after upgrade not starting - getting errr...

by Contributor Anishkumarv in Support Questions
‎10-21-2017 05:49 PM
‎10-21-2017 05:49 PM
@Matt Clarke need your help on this. ... View more

Nifi after upgrade not starting - getting errror "...

by Contributor Anishkumarv in Support Questions
‎10-21-2017 05:48 PM
‎10-21-2017 05:48 PM
Hi Team, After upgrade to HDF 3.0.1 nifi not starting and getting below error while starting.kinldy helpt to fix this error. /loginIdentityProviders> 2017/10/22 01:42:07 ERROR [main] org.apache.nifi.properties.ConfigEncryptionTool: Encountered an error javax.crypto.BadPaddingException: pad block corrupted at org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher$BufferedGenericBlockCipher.doFinal(Unknown Source) at org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher.engineDoFinal(Unknown Source) at javax.crypto.Cipher.doFinal(Cipher.java:2165) at javax.crypto.Cipher$doFinal$2.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at org.apache.nifi.properties.ConfigEncryptionTool.decryptFlowElement(ConfigEncryptionTool.groovy:541) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190) at org.apache.nifi.properties.ConfigEncryptionTool$_migrateFlowXmlContent_closure4.doCall(ConfigEncryptionTool.groovy:636) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019) at groovy.lang.Closure.call(Closure.java:426) at groovy.lang.Closure.call(Closure.java:442) at org.codehaus.groovy.runtime.StringGroovyMethods.getReplacement(StringGroovyMethods.java:1543) at org.codehaus.groovy.runtime.StringGroovyMethods.replaceAll(StringGroovyMethods.java:2580) at org.codehaus.groovy.runtime.StringGroovyMethods.replaceAll(StringGroovyMethods.java:2506) at org.codehaus.groovy.runtime.dgm$1127.invoke(Unknown Source) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) at org.apache.nifi.properties.ConfigEncryptionTool.migrateFlowXmlContent(ConfigEncryptionTool.groovy:635) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.apache.nifi.properties.ConfigEncryptionTool.main(ConfigEncryptionTool.groovy:1184) pad block corrupted ... View more

Re: HDF Nifi not starting

by Contributor Anishkumarv in Support Questions
‎10-13-2017 01:15 AM
‎10-13-2017 01:15 AM
@Matt Clarke Thanks Bro you are my saviour 🙂 ... View more

Re: HDF Nifi not starting

by Contributor Anishkumarv in Support Questions
‎10-12-2017 04:42 PM
‎10-12-2017 04:42 PM
@Matt Clarke Bro you have any idea on this error? ... View more
Contact Me
Online
Offline
Last Visited
‎10-10-2019 01:59 AM
Public Statistics
Date Registered ‎04-06-2019 02:41 AM
Last Visited ‎10-10-2019 01:59 AM
Total Messages Posted 124
Total Kudos Received 2