pass parameters to file with sql statement using Scala to be used sqlContext.sql(string)

Question

I am using Scala on data science workbench and I have file with sql stataement would like to make it dynamic sql by passing argument to parameters. I then would like to pass it sqlContext.sql(string) .

This is what I have tried but does not work.

val FromDate = "2019-02-25"

val sqlfile = fromFile("sql3.py").getLines.mkString

val result = sqlContext.sql(sqlfile)

On the file I have:

Select col1, col2 from table1 where transdate = '${FromDate}'

Any help would be appreciated.

Thanks

TF123 · Accepted Answer

This worked at the end..

val FromDate = "2019-02-25"

val sqlfile = fromFile("sql3.py").getLines.mkString

val sqlfil2= sqlfile.replace("${From_timeStamp}",From_timeStamp)

val result = sqlContext.sql(sqlfile2)

On the file I have:

Select col1, col2 from table1 where transdate = '${FromDate}'

Gopinath · Answer

Did you try CDSW environment variables? https://www.cloudera.com/documentation/data-science-workbench/latest/topics/cdsw_environment_variables.html#access_env_variables

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.