Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Drools In Spark for Streaming File

Drools In Spark for Streaming File

Explorer

Hi, 

 

We were able to successfully Integrate drools with spark, When we try to apply rules from Drools we were able to do for Batch file hich is present in HDFS, But we tried to use drools for Streaming file so that we can make decision instantly, But we couldn't figure out how to do it.Below is the sniipet of the code what we are trying to achieve.

SparkConf conf = new SparkConf().setAppName("sample");
		JavaSparkContext sc = new JavaSparkContext(conf);

		JavaRDD<String> javaRDD = sc.textFile("/user/root/spark/sample.dat");
		List<String> store = new ArrayList<String>();
		store = javaRDD.collect();

2nd case when we use streaming context

SparkConf sparkconf = new SparkConf().setAppName("sparkstreaming");
		JavaStreamingContext ssc = 
	              new JavaStreamingContext(sparkconf, new Duration(1));
		
		JavaDStream<String> lines = ssc.socketTextStream("xx.xx.xx.xx", xxxx);
	 
	    

 In the first case we were able apply our rules on the variable store, but in the second case we were not able to apply rules on the dstream lines. If someone has some idea, how we can it will be a great help. Thanks in Advance.

 

Regards,

Arun

1 REPLY 1

Re: Drools In Spark for Streaming File

New Contributor

We have been using drools with CDH but not with spark stream though. If you can share a short test case with pom everything in a zip, we may be able to help you figure out the root cause.