@Abdul Rahim Please use following code case class Person(index:Long,item:String,cost:Float,Tax:Float,Total:Float) val peopleDs = sc.textFile("C:/hcubeapi/test-case1123.txt").map(_.split(",").map(_.trim)).map(attributes=> Person(attributes(0).toLong,attributes(1).toString,attributes(2).toFloat,attributes(3).toFloat,attributes(4).toFloat)).toDF() peopleDs.createOrReplaceTempView("people") val res = spark.sql("Select * from people") res.show() ... View more

@Ahmad Debbas You shouldn't lose FlowFiles which are in queues as the attributes themselves are stored in FlowFile repository and content in content repository. What you see in the queue is just a reference which will be restored after the restart. You can find additional information in below link https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html ... View more

@Ahmad Debbas Can you please check if NiFi JVM is using UTF-8 encoding, otherwise here are a couple of approaches 1. export JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8 2. Add encoding parameters to bootstrap.conf "java.arg.8=-Dfile.encoding=UTF8" Please restart NiFi after update and test to see if FetchFile/GetFile processors are working Thanks Kiran. ... View more

@Venkata Sudheer Kumar M Couple of things to note, 1. If hive-site.xml file is manually copied to spark2/conf folder, any Spark configuration changes from Ambari might have removed the hite-site.xml 2. As the deploy mode is cluster, you need to check if hive-site.xml and hbase-site.xml files are available under Spark conf in the driver machine and not on the machine where spark-submit command was executed. ... View more

@Venkata Sudheer Kumar M, I'm not sure if SPARK_YARN_DIST_FILES is a valid spark-env value, but you can pass comma separated files using spark.yarn.dist.files spark property. ... View more