Reply
Highlighted
Explorer
Posts: 62
Registered: ‎01-22-2014

Drools In Spark for Streaming File

Hi, 

 

We were able to successfully Integrate drools with spark, When we try to apply rules from Drools we were able to do for Batch file hich is present in HDFS, But we tried to use drools for Streaming file so that we can make decision instantly, But we couldn't figure out how to do it.Below is the sniipet of the code what we are trying to achieve.

SparkConf conf = new SparkConf().setAppName("sample");
		JavaSparkContext sc = new JavaSparkContext(conf);

		JavaRDD<String> javaRDD = sc.textFile("/user/root/spark/sample.dat");
		List<String> store = new ArrayList<String>();
		store = javaRDD.collect();

2nd case when we use streaming context

SparkConf sparkconf = new SparkConf().setAppName("sparkstreaming");
		JavaStreamingContext ssc = 
	              new JavaStreamingContext(sparkconf, new Duration(1));
		
		JavaDStream<String> lines = ssc.socketTextStream("xx.xx.xx.xx", xxxx);
	 
	    

 In the first case we were able apply our rules on the variable store, but in the second case we were not able to apply rules on the dstream lines. If someone has some idea, how we can it will be a great help. Thanks in Advance.

 

Regards,

Arun

New Contributor
Posts: 1
Registered: ‎02-10-2015

Re: Drools In Spark for Streaming File

We have been using drools with CDH but not with spark stream though. If you can share a short test case with pom everything in a zip, we may be able to help you figure out the root cause. 

Announcements