Created on 03-01-2019 03:13 AM - edited 09-16-2022 07:12 AM
I am using Scala on data science workbench and I have file with sql stataement would like to make it dynamic sql by passing argument to parameters. I then would like to pass it sqlContext.sql(string) .
This is what I have tried but does not work.
val FromDate = "2019-02-25"
val sqlfile = fromFile("sql3.py").getLines.mkString
val result = sqlContext.sql(sqlfile)
On the file I have:
Select col1, col2 from table1 where transdate = '${FromDate}'
Any help would be appreciated.
Thanks
Created 03-07-2019 05:55 AM
This worked at the end..
val FromDate = "2019-02-25"
val sqlfile = fromFile("sql3.py").getLines.mkString
val sqlfil2= sqlfile.replace("${From_timeStamp}",From_timeStamp)
val result = sqlContext.sql(sqlfile2)
On the file I have:
Select col1, col2 from table1 where transdate = '${FromDate}'
Created 03-07-2019 04:17 AM
Did you try CDSW environment variables?
Created 03-07-2019 05:55 AM
This worked at the end..
val FromDate = "2019-02-25"
val sqlfile = fromFile("sql3.py").getLines.mkString
val sqlfil2= sqlfile.replace("${From_timeStamp}",From_timeStamp)
val result = sqlContext.sql(sqlfile2)
On the file I have:
Select col1, col2 from table1 where transdate = '${FromDate}'
Created 10-09-2021 11:56 AM
Only one question I have the line which you made bold in that you are trying to replace "from_timestamp" but you are using "fromdate" in your sql file.
Any reason behind it ?