Member since
09-18-2015
3274
Posts
1159
Kudos Received
426
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2722 | 11-01-2016 05:43 PM | |
| 9345 | 11-01-2016 05:36 PM | |
| 5076 | 07-01-2016 03:20 PM | |
| 8519 | 05-25-2016 11:36 AM | |
| 4676 | 05-24-2016 05:27 PM |
02-04-2016
05:38 PM
@Greenhorn Techie spark client is client so there is no start and stop. As you can see from my screeen shot , when you stop spark , it will stop all those components. Now , for example when you run the following, it will launch a spark job and shutdown the job once it finishes. Run on a YARN cluster
export HADOOP_CONF_DIR=XXX
./bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
--master yarn-cluster \ # can also be `yarn-client` for client mode
--executor-memory 20G \
--num-executors 50 \
/path/to/examples.jar \
1000
... View more
02-04-2016
05:13 PM
@Pradeep kumar Please see this http://stackoverflow.com/questions/21432002/ntp-configuration-without-internet
... View more
02-04-2016
04:48 PM
@Greenhorn Techie Please see this and you can use that landing for all your future questions on manual install/use Now, going back to your requirement...Please take few minutes to read this http://spark.apache.org/docs/latest/spark-standalone.html sbin/start-all.sh - Starts both a master and a number of slaves as described above. sbin/stop-master.sh - Stops the master that was started via the bin/start-master.sh script. sbin/stop-slaves.sh - Stops all slave instances on the machines specified in the conf/slaves file. sbin/stop-all.sh - Stops both the master and the slaves as described above.
... View more
02-04-2016
04:22 PM
1 Kudo
@Greenhorn Techie
This is what it looks like When you stop through ambari then it will stop the following/all components. If just want to start or stop particular component then you can click that particular service and take the action. Click Spark Thrift Server and you can just start or stop that . I clicked thrift server and I can start from there...
... View more
02-04-2016
03:32 PM
@Davide Vergari Could you post it as an article? Thanks for sharing this.
... View more
02-04-2016
03:29 PM
1 Kudo
@Greenhorn Techie
You have to be careful before you switch users because you need to make sure that your id had proper permissions on HDFS to read and write. Answer is yes but again, make sure you do this su - hdfs hdfs dfs -mkdir /user/tom hdfs dfs -chown -R tom:hdfs /user/tom
... View more
02-04-2016
03:14 PM
@Pradeep kumar This is the exact root cause I checked the time on all the nodes. I found that, except the master node, time on all the other nodes were incorrect. So I manually corrected (ntpd was failing to connect to servers) the time on all the nodes. Do you know why ntpd is failing?
... View more
02-04-2016
11:23 AM
@nejm hadj Also, see this https://community.hortonworks.com/articles/961/a-collection-of-nifi-examples.html Search in HCC on Nifi examples
... View more