Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 09-13-2022 02:25 AM
Dear all,
I am trying to load a PySpark Dataframe into an Impala table, using the jdbc connector. However, the df.write statement fails, because the "Create table" - statement that is generated contains quotation marks for the column names:
Do you have any idea how to get rid of these quotation marks? If not, what would be a different approach to load a dataframe into an Impala table?
I also tried
spark.sql('select identifier_id as identifier from tempView').write.jdbc(...), but here I am getting the error "File /tmp/hive does not exist".
Thanks a lot in advance for any help!
Created 09-14-2022 12:54 AM
Hi @Ploeplse
Could you please share reproducible sample code and impala tab creation script?
Created 09-22-2022 12:51 AM
HI @Ploeplse, If you are still experiencing the issue, can you provide the information @RangaReddy has requested?
Regards,
Vidya Sargur,Created 10-11-2022 02:13 AM
Hi @Ploeplse
Still, if you are facing the issue, could you share the requested information (i.e code and impala table creation script)