About pvillard

pvillard · ‎09-02-2016

Thanks a lot! It works like a charm.

ledel · ‎10-23-2016

@Pierre Villard I got it working with -D mapred.job.name=mySqoopTest

Adija1 · ‎08-21-2016

Thank you @Matt Burgess

arturo_amador · ‎08-10-2016

Thanks a lot for the great help, @Pierre Villard !

yogesh_sharma05 · ‎08-16-2016

@mclark Thanks for the response and appreciated. Do I need to configure something at back-end as well i.e. in nifi.properties or any other file in cluster or node because I am facing attached error.

anandi · ‎08-09-2016

HI Pierre, We would need to look at the code. Can you a do a persist just before stage 63 and before stage 65 check the spark UI storage tab and executor tab for data skew. If there is data skew, you will need to add a salt key to your key. You could also look at creating a dataframe from the RDD rdd.toDF() and apply UDF on it. DF manage memory more efficiently. Best, Amit

pvillard · ‎08-29-2016

@Alvin Ji, This is correct with NiFi 0.x. Unless you implement your own MapCacheServer service and separate it from NiFi, I am not sure there is a solution. With NiFi 1.x (first version to be released in coming days, RC vote in progress), this is solved with a zero-master clustering paradigm.

pvillard · ‎07-25-2016

Hi @Obaid Salikeen, Another option is to use ExecuteProcess or ExecuteStreamCommand to execute a custom script that will SCP to your remote Linux instance. Otherwise there is a JIRA for a SCP processor: https://issues.apache.org/jira/browse/NIFI-539 Hope this helps.

pvillard · ‎06-17-2016

Answering my question, I was just not using the concat function correctly... the syntax is: ${fs:exists(concat(wf:actionData('hdfs-lookup')['outputPath'], '/2'))}

manikandan · ‎06-14-2016

Thanks @ Pierre Villard This is what i was expecting... Thanks once again...

Online	Offline
Last Visited	‎07-30-2024 08:59 AM

Member Since	‎04-11-2016 09:20 AM
Last Visited	‎07-30-2024 08:59 AM
Posts	471
Kudos received	325

Cloudera Community

Re: ValidateRecord doesn't maintain column order?

Re: For NiFi S2S, is it better to us load balancer...

Re: How to Limit Number of Threads for Each Proces...

Re: Once YARN queue is at capacity, running jobs s...

Re: putHiveQL error

Re: Sqoop Hortonworks TDCH - Queryband option

Re: Sqoop job name

Re: Error while executing SelecftHiveQL

Re: NiFi's ConsumeAMQP RabbitMQ to HDFS

Re: How to get only unique data from flow files

Re: Spark driver memory keeps growing

Re: nifi detect duplicate MapCacheServer Error

Re: How to copy files from a remote linux box usin...

Re: String concatenation in Oozie expression langu...

Re: How to get files based on dates in NiFi?