I am reading a parquet file which has French characters and transforming the Rdd to a new format and creating output file using saveAsTextFile. In my output I see Junk characters wherever input has French characters, Please help resolve the issue. How do I take care of the French Characters ?
gffRdd.filter((row) => row._1.equals(source)).map(row => row._2).repartition(1).saveAsTextFile(s"$outputLocalPath/$source")
What are the characters causing issues? I have a feeling this is going to have to go in a JIRA and we can make a manual work around for now.