Hi Nilesh, I know it is too late to answer, still lets add a solution for future issues Please try --conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -Djava.security.auth.login.config=kafka_jaas.conf" instead of --conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -Djava.security.auth.login.config=/home/user/sparktest/kafka_jaas.conf" Also ensure to use just the file name 'user.headless.keytab' in jaas file (not absolute file path) . eg: KafkaClient { com.sun.security.auth.module.Krb5LoginModule required doNotPrompt=true useTicketCache=false principal="userprincipalname" useKeyTab=true serviceName="kafka" keyTab="user.headless.keytab" client=true; }; Thanks Vinod ... View more

@Venkata Sudheer Kumar M Can you please share the spark documentation which refers "SPARK_YARN_DIST_FILES" ? In Spark 2.2 code, I couldn't locate usage of this env variable. ... View more

Hi Anilkumar, While submitting spark application , you need to pass your Teradata jdbc Driver jar file that with --jar option Thanks Vinod ... View more

@kenny creed You may try the APIs suggested in this link : https://github.com/databricks/spark-xml/issues/114#issuecomment-215301575 ... View more

@kenny creed Using a sample xml file with "xmlns: /data/path/d" to the root element tag, I'm able to parse it with this code sqlContext.read.format("com.databricks.spark.xml").option("rowTag", "root").load("file:///testxml/data.xml").show() //Sample XML file <root xmlns:h="http://www.w3.org/TR/html4/" xmlns:f="https://www.abc.com/furniture"> <h:table> <h:tr> <h:td>Apps</h:td> <h:td>bean</h:td> </h:tr> </h:table> <f:table> <f:name>tables</f:name> <f:width>80</f:width> <f:length>120</f:length> </f:table> </root> If it is not solving your current issue, please share your sample xml file to understand the xml content better. ... View more

@kenny creed Using sample xml given below, I'm able to parse and get the result sqlContext.read.format("com.databricks.spark.xml").option("rowTag", "root").load("file:///root/testxml/data.xml").show() <root xmlns:h="http://www.w3.org/TR/html4/" xmlns:f="https://www.abc.com/furniture"> <h:table> <h:tr> <h:td>Apples</h:td> <h:td>Bananas</h:td> </h:tr> </h:table> <f:table> <f:name>Coffee Table</f:name> <f:width>80</f:width> <f:length>120</f:length> </f:table> </root ... View more

Please try this and let me know. I've tested in spark 1.6.3. ./bin/spark-shell --packages com.databricks:spark-xml_2.10:0.4.1 scala> sqlContext.read.format("com.databricks.spark.xml").option("rowTag","WSAOnRoad").load("file:///root/problem.xml").show(false) ... View more

@kenny creed, Can you please share a sample xml file with "xmlns: /data/path/d" to try out a solution? Thanks Vinod ... View more

Updated link : https://github.com/hortonworks/data-tutorials/blob/master/tutorials/hdf/hdf-2.1/realtime-event-processing-in-hadoop-with-nifi-kafka-and-storm/assets/IoT_Lab_Series_DataFlow.xml ... View more