Member since
02-18-2017
10
Posts
0
Kudos Received
0
Solutions
05-12-2017
06:20 AM
Based on the columns in spark dataframe need to do a lookup on another huge HBASE table. Is there any efficient way available to perform lookup operation in Spark Dataframe
... View more
Labels:
- Labels:
-
Apache Spark
02-23-2017
11:54 AM
I tried to follow the same steps mentioned above to override the log4j properties but still not working. My problem is I am running kafka streaming job in yarn cluster mode and when I go and see logs after one hour in web UI,the logs are grown big. I like to know the steps where I can write logs in local file system/hdfs so I would go and see the logs in unix terminal instead of using web UI.
... View more
02-23-2017
11:47 AM
Labels:
- Labels:
-
Apache Spark
-
Apache YARN