Member since
04-19-2016
13
Posts
4
Kudos Received
0
Solutions
09-06-2016
11:51 PM
@Rendiyono Wahyu Saputro I recommend you look at storm vs spark in a different manner. if your stream response can handle some latency (as little as 1/2 a second) then spark may be the way to go. This is just my opinion as spark streaming is so darn easy. Storm is a POWERFUL engine with virtually zero latency. Storm has been clocked on millions of tuples per node per second. So you have to ask yourself if your use case needs zero latency or can you handle micro batch (spark streaming)
... View more
08-29-2016
04:42 PM
1 Kudo
Hello @Rendiyono Wahyu Saputro Yes, you can import python libraries and use them in Spark, which supports a full Python API via the pyspark shell. For instance, if you wanted to load and use the python scikit-fuzzy library to run fuzzy logic, then you just: 1) Download python library, either using maven update to local repo, or directly via github, and add the library to your Spark classpath 2) Kick off job with pyspark shell (Example: $ pyspark --jars /path/to/scikit-fuzzy.jar ) 3) Import python library in your code (Example: "import skfuzzy as fuzz") 4) Use the library More information about scikit-fuzzy library here: https://pypi.python.org/pypi/scikit-fuzzy Hints about dependencies and install: Scikit-Fuzzy depends on
NumPy >= 1.6 SciPy >= 0.9 NetworkX >= 1.9 and is available on PyPi! The lastest stable release can always be obtained and installed simply by running $ pip install -U scikit-fuzzy
... View more
10-08-2017
10:34 PM
easy to integrate NiFi -> Kafka -> Spark or Storm or Flink or APEX Also NiFi -> S2s -> Spark / Flink / ...
... View more
08-16-2016
06:44 AM
@Davide Isoardi Good Afternoon, sir.
May i ask you something ? Is http://www.ecubecenter.it/Hadoop-Big-Data-Twitter-Map/ using this https://github.com/disoardi/tweetsdemo_with_ambari_views. It has been a while that i lost my mind to work my undergraduate project. Maybe, i will start again from this. Thanks before
... View more