Created on 06-13-2017 04:05 PM - edited 09-16-2022 04:44 AM
I was successfully able to parse the sas files using the parso lib if the files are present in the local or HDFs. if it is from cloud storage unable to parse the files. getting following exception.
<code>
ERROR SFP: Failed to read page from file 17/06/13 12:03:29 INFO SRR: Bitness: x86
</code>
Created 06-14-2017 09:27 PM
It seems to be an issue with the SasParser file. Could you please check if you have the latest SasFileParser lib.
Please find below my SBT confiig
libraryDependencies ++= Seq( "com.databricks" % "spark-csv_2.11" % "1.5.0", "org.slf4j" % "slf4j-api" % "1.7.5" )
Created 06-13-2017 06:06 PM
what is your cloudstorage- wasb should be an issue
Created 06-13-2017 06:26 PM
this is not wasb - we are using s3 for the storage.
Created 06-13-2017 09:47 PM
Can you try with S3a instead of S3n and post the outcome here ?
Created 06-14-2017 03:57 PM
Nope. now the parsing error is gone. But it is not reading anything from s3. empty Data frame
Created 06-14-2017 09:27 PM
It seems to be an issue with the SasParser file. Could you please check if you have the latest SasFileParser lib.
Please find below my SBT confiig
libraryDependencies ++= Seq( "com.databricks" % "spark-csv_2.11" % "1.5.0", "org.slf4j" % "slf4j-api" % "1.7.5" )
Created 06-15-2017 01:12 AM
Thanks @njayakumar - works now.