Hi,   Spark is a tool used for processing the data and Sqoop is used to import/export.   NOTE:- Both the tools have different behaviours and are used accordingly.   Hope able helps.   Regards Nitish ... View more

@Venkat You can use either .select(expr(function)) (or) .selectExpr("function") Example: val n_df = Seq(("A", 1, "US"), ("AB", 1, "US"), ("ABC", 1, "US")).toDF("city", "num", "country") .select(expr) n_df.select(expr("(num * 10) as num"),lit(1).as("op")).show() +---+---+ |num| op| +---+---+ | 10|  1| | 10|  1| | 10|  1| +---+---+ .selectExpr: n_df.selectExpr("num * 10 as num").show(false) +---+ |num| +---+ |10 | |10 | |10 | +---+ ... View more

Hello @Venkat! Hm, the first that comes to my mind is to use cross join: SELECT b.value FROM a,b WHERE a.date >= b.date Just remember, depending on the size table this could be a risk to run since you're multiplying each line from a against b. Hope this helps ... View more

I am asked about the Yarn Top .With the help of that command may I know the size of the cluster? and could please explain the attributes? ... View more

Sorry @ssharma it is working but when I trying like this hive -e 'set hive.cli.print.header=true; create table test row format delimited fields terminated by '|' as select * from test1'>/home/yourfile.csv in this scenario it only showing the header not the whole data csv file ... View more

Check whether the source (HDFS) datatypes and target table(MY SQL) datatypes are same or not and Can you please try to export the file with delimiters provide in the HDFS file (--input-fields-terminated-by << >>) ... View more

@Venkat Metastore connection options: Argument Description --meta-connect <jdbc-uri> Specifies the JDBC connect string used to connect to the metastore By default, a private metastore is instantiated in $HOME/.sqoop. If you have configured a hosted metastore with the sqoop-metastore tool, you can connect to it by specifying the --meta-connect argument. This is a JDBC connect string just like the ones used to connect to databases for import. In conf/sqoop-site.xml, you can configure sqoop.metastore.client.autoconnect.url with this address, so you do not have to supply --meta-connect to use a remote metastore. This parameter can also be modified to move the private metastore to a location on your filesystem other than your home directory. If you configure sqoop.metastore.client.enable.autoconnect with the value false, then you must explicitly supply --meta-connect. Hope this helps ... View more