About mqureshi

SK1 · ‎02-02-2017

@Jay SenSharma: Can you please help me to get dir last accessed time also. Above one is not working for dir.

mqureshi · ‎01-31-2017

@Prasanna G your putty is an ssh client and not an hdfs client. once you ssh to your sandbox, then you are able to run hdfs command because sandbox is where HDFS is installed including on your shell commands. This is simialr to the fact that you cannot run "ls /some directory" from putty before you ssh into the box.

mqureshi · ‎02-03-2017

@Avijeet Dash I agree with you. It is much more reliable if after your streaming job, your data lands in Kafka and then written to HBase/HDFS. This decouples your streaming job from writing. I wouldn't recommend using Flume. Go with the combination of Nifi and Kafka.

HeartSaVioR · ‎02-02-2017

Latencies for each component excludes queue wait time and transfer latency between worker. 'complete latency' means all the nodes in tuple tree are acked, so it reflects slowest path of the tree. Btw, behind the scene, 'complete latency' includes waiting time for spout to handle ack from acker, and if your spout spends long time in nextTuple it will heavily affect 'complete latency'. It is fixed from STORM-1742 and will be included to next release. (Storm 1.0.3 for Apache side, but not sure which HDP / HDF versions contain this fix. Please let me know your HDP/HDF version.) Hope this helps.

mqureshi · ‎01-23-2017

The only thing you can do is limit which IP's can access your cluster. Basically specifying security rules for inbound traffic (or outbound also). http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html#ec2-classic-security-groups

mqureshi · ‎01-09-2017

@Hoang Le No, Ambari UI will set it for future files that you will create. It will not run setrep command for you. That you will have to run from shell as described above.

slachterman · ‎12-20-2016

@jzhang good call, I changed to yarn-cluster mode for the Livy interpreter and was not able to reproduce the error in HDP 2.5.

sajindas · ‎01-23-2018

Thanks. In the above solution, how will the external system (syslog source) identify as to which Nifi node to be sent with the messages? Will it be ZK url or Primary Node itself? In case of latter, how will a PN fail-over can be made aware to external source?

FSTunis · ‎12-14-2016

Hi @mqureshi, Here i want more explain my problem I want use the cloudera CDH 5.8 for Talend Open Studio Big Data 6.3.0. For this , i must to connect to hadoop manualy. i must fill the different fields. (picture 1) for field 1, 3 and 5 i don't have problem. i find here value on core-site.xml and mapred-site.xml. For field 2 and 3 , i don't have her value in yarn-site.xml, i leaved the default values. When i chek services, i obtain 2 error log and i can't see them.(picture 2) what do you think i should do here ? thank you

mqureshi · ‎12-13-2016

@Junaid Rao No, this is not possible even when Windows was a supported platform. Now, HDP support for Windows has been deprecated. We strongly suggest using Linux platforms.

Online	Offline
Last Visited	‎10-31-2017 03:17 AM

Member Since	‎06-07-2016 09:05 AM
Last Visited	‎10-31-2017 03:17 AM
Posts	923
Kudos received	310

Cloudera Community

Re: YARN recommended configuration

Re: How to resolve for NULL values when they are c...

Re: Why is spark has better speed than Hadoop

Re: Is it possible to assign Hadoop queues to Hado...

Re: Kafka NiFi HDF Installation

Re: Is there anyway to get last access time of hdf...

Re: Cannot find files copied to Sandbox from Windo...

Re: streaming ingest to hdfs

Re: Storm complete latency more than individual bo...

Re: ports required to be open

Re: [HDFS] Block replication dfs.replication affec...

Re: Livy HTTP 403 Error

Re: Load Balance NiFi Cluster

Re: Where find resource manager URI value

Re: Add Windows Server 2012 Data Node to a CENTOS7...