spark .session() .read() .option("encoding", "UTF-8") .option("delimiter", "^") .option("mode", "PERMISSIVE") .schema(SCHEMA_STORE.getIPDRschema()) .csv( JavaConverters.collectionAsScalaIterableConverter(_files_to_process) .asScala() .toSeq()) .withColumn("filename", org.apache.spark.sql.functions.input_file_name()) .dropDuplicates();   Written in java please convert it into scala hope this will work 🙂 ... View more