About abhinav_phutela

abhinav_phutela · ‎07-24-2018

This means that a TCP RST was received and the connection is now closed. This occurs when a packet is sent from your end of the connection but the other end does not recognize the connection; it will send back a packet with the RST bit set in order to forcibly close the connection. This can happen if the other side crashes and then comes back up or if it calls close() on the socket while there is data from you in transit, and is an indication to you that some of the data that you previously sent may not have been received. It is up to you whether that is an error; if the information you were sending was only for the benefit of the remote client then it may not matter that any final data may have been lost. However you should close the socket and free up any other resources associated with the connection.

abhinav_phutela · ‎06-22-2018

What should be the permission of the hdfs-site.xml and core-site.xml files which I'm supposed to copy in /home/nifi folder? And I'm able to telnet to datanode and namenode both (50075 and 50070)

abhinav_phutela · ‎06-21-2018

We have integrated NiFi service (HDF service) to our existing HDP cluster. NiFi installation was successful and NiFi is running without any error. But there is an error with PutHDFS. It's showing that there is an error in our Hadoop configuration. I'm posting the error and screenshot too. 2018-06-20 12:00:14,246 WARN [StandardProcessScheduler Thread-6] org.apache.hadoop.conf.Configuration /tmp/core-site.xml:an attempt to override final parameter: fs.defaultFS; Ignoring. 2018-06-20 12:00:14,248 ERROR [StandardProcessScheduler Thread-6] o.apache.nifi.processors.hadoop.PutHDFS PutHDFS[id=11a40827-0164-1000-ffff-ffffb07a04d9] HDFS Configuration error - java.net.ConnectException: Connection refused: {} java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495) at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.checkHdfsUriForTimeout(AbstractHadoopProcessor.java:345) at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.resetHDFSResources(AbstractHadoopProcessor.java:260) at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.abstractOnScheduled(AbstractHadoopProcessor.java:205) at sun.reflect.GeneratedMethodAccessor728.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) at org.apache.nifi.controller.StandardProcessorNode$1.call(StandardProcessorNode.java:1334) at org.apache.nifi.controller.StandardProcessorNode$1.call(StandardProcessorNode.java:1330) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2018-06-20 12:00:14,248 ERROR [StandardProcessScheduler Thread-6] org.apache.nifi.engine.FlowEngine A flow controller task execution stopped abnormally java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.reflect.InvocationTargetException: null

abhinav_phutela · ‎05-24-2018

The best solution is to stop all the services and start Zookeeper only. And then go to CLI where your namenode is present. Run the command => hdfs zkfc -formatZK (you'll be asked for an option as Y or N, input Y) and through ambari start all the services again. Your Namenode will be in Active-Standby mode again!

abhinav_phutela · ‎04-17-2018

@Ranjit masetty Though I'm too late to answer your query but I have gone through all the previous comments, and I have tested your query on my cluster. And the answer to your question is NO. You won't be able to access a.txt once you have changed the file name to b.txt because hdfs changes are really quick and the changes will be stored in edit_logs. So, you won't be able to access your file as a.txt.

Online	Offline
Last Visited	‎03-19-2019 10:29 PM

Member Since	‎04-04-2018 11:14 AM
Last Visited	‎03-19-2019 10:29 PM
Posts	11

Cloudera Community

Re: Connection failed to DataNode:50075 sometimes

Re: NiFi: Hadoop Configuration Error

NiFi: Hadoop Configuration Error

Re: Both NameNodes are StandBy

Re: Actual use of FSimage and Edits log?