Skip to main content
RangaReddy
Gold ⭐️
June 24, 2021

5 replies

roshanbi
roshanbiAuthor
Gold ⭐️
June 23, 2021

I have been working with Oracle databases, in what way is DataFrames and DataSets similar to Oracle? Are they similar to views?

Translate
RangaReddy
Gold ⭐️
Gold ⭐️
June 24, 2021
roshanbi
roshanbiAuthor
Gold ⭐️
June 24, 2021

Hi @RangaReddy 

 thanks a lot for sharing the link. It will help me a lot.

 

Can you please advise why we have to include df (data frame name) before each column?

df.select(df("name"), df("age") + 1).show()

 I noticed in groupBy() there is no df.

 

Grateful if you can clarify this.

 

Thanks,

 

Roshan

Translate
RangaReddy
Gold ⭐️
Gold ⭐️
June 24, 2021

Hi @roshanbi 

 

Apache Spark offers several methods to use when selecting a column.

 

Scala Spark:

// Scala
import org.apache.spark.sql.functions.{expr, col, column}
// 6 ways to select a column
df.select(df.col("ColumnName"))
df.select(col("ColumnName"))
df.select(column("ColumnName"))
df.select(`ColumnName)
df.select($"ColumnName")
df.select(expr("ColumnName"))

 

PySpark:

# Python
from pyspark.sql.functions import expr, col, column
# 4 ways to select a column
df.select(df.ColumnName)
df.select(col("ColumnName"))
df.select(column("ColumnName"))
df.select(expr("ColumnName"))

 

Translate