@nejm hadj Adding more information based on your comments https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.flume.ExecuteFlumeSink/additionalDetails.html You should stick with NiFi and use built in processor to ingest the data from various social media sources. Please do read docs. In NiFi, the contents of a FlowFile are accessed via a stream, but in Flume it is stored in a byte array. This means the full content will be loaded into memory when a FlowFile is processed by the ExecuteFlumeSink processor. You should consider the typical size of the FlowFiles you'll process and the batch size, if any, your sink is configured with when setting NiFi's heap size. ... View more

Thank you!!! I am happy to join this network for the level of support being provided. It keeps me motivated!!! ... View more

Hello @Michael Dennis "MD" Uanang can you confirm that this worked on a 3-nodes Zookeeper install? I mean, I need to move my zk cluster from the original 3 hosts to other 3 hosts, will this work repeating 3 times the same procedure? ... View more

@ccasano I don't see any issues in having Isilon to store the workflow repositories. Isilon is scalable storage solution and based on my experience, Isilon can be a good solution based on ... View more

Ah cool didn't see that! ... View more

on Sandbox 2.5, Datafu is indeed 1.3, validated the function albeit with different dataset DEFINE HCatLoader org.apache.hive.hcatalog.pig.HCatLoader(); DEFINE SampleByKey datafu.pig.sampling.SampleByKey('0.2'); ROWS = load 'sample_08' using HCatLoader(); SAMPLE_BY_total_emp = filter ROWS by SampleByKey(total_emp); STORE SAMPLE_BY_total_emp into 'sample_total_emp'; [guest@sandbox ~]$ hdfs dfs -cat sample_total_emp/part-v000-o000-r-00000 | head -n 5 11-3011 Administrative services managers 246930 79500 11-9121 Natural sciences managers 43060 123140 13-1032 Insurance appraisers, auto damage 11280 53980 13-1051 Cost estimators 218400 60320 13-1072 Compensation, benefits, and job analysis specialists 116250 57060 ... View more

@Smart Solutions When you add Spark through Ambari, you will be asked to choose where to deploy master service (Spark History Service) And then to choose where to deploy clients services Finally you will be asked for several properties screen-shot-2016-03-03-at-61725-pm.png ... View more

sure thing ... View more

@Artem Ervits @Mehrdad Niasari I believe we can lose this question. i have opened new one on default namespace here. ... View more