Member since
03-14-2016
4721
Posts
1111
Kudos Received
874
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2507 | 04-27-2020 03:48 AM | |
| 4974 | 04-26-2020 06:18 PM | |
| 4054 | 04-26-2020 06:05 PM | |
| 3287 | 04-13-2020 08:53 PM | |
| 5011 | 03-31-2020 02:10 AM |
06-13-2018
06:54 AM
@bharat sharma As we see the following error which indicates that you have not placed the hadoop-aws jars in the classpath: py4j.protocol.Py4JJavaError: An error occurred while calling o32.load.: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.s3a.S3AFileSystem not found<br> . So can you please check and download the aws sdk for java https://aws.amazon.com/sdk-for-java/ Uploaded it to the hadoop directory. please check your "spark.driver.extraClassPath" if it has the "hadoop-aws*.jar" and "aws-java-sdk*.jar" For more details please refer to : https://community.hortonworks.com/articles/25523/hdp-240-and-spark-160-connecting-to-aws-s3-buckets.html https://community.hortonworks.com/articles/36339/spark-s3a-filesystem-client-from-hdp-to-access-s3.html
... View more
06-13-2018
05:21 PM
It's already installed and the issue resolved now, tanx for you response.
... View more
06-01-2018
11:04 AM
1 Kudo
@Victor You can get detailed explaination about Yarn "Cluster Memory" in the following thread. https://community.hortonworks.com/questions/144325/why-my-cluster-memory-is-less-even-though-physical.html Regaarding 0 (Zero Memory utiliaztion). Until you run a job you wont see the memory being utilized. So please try running some job like following and then refresh the ambari Yarn dashboard. # su - spark
# export SPARK_MAJOR_VERSION=2
# cd /usr/hdp/current/spark2-client
# ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn-client --num-executors 3 --driver-memory 512m --executor-memory 512m --executor-cores 1 examples/jars/spark-examples*.jar 10
.
... View more
06-01-2018
05:48 AM
Okay ! after a bit of search on stackoverflow I came across the command `sudo chown -R $USER:$USER /home/rahul/nifi-1.6.0` that will give the permission to the folder to alter the changes with permission to read and write, and Bingo !!! it worked . And after firing the command mentioned in above, there is no error as following : Exception in thread "main" java.io.IOException: /home/nifi-1.6.0/run directory does not have read/write privilege Thanks @Matt Clarke @Jay Kumar SenSharma
... View more
05-17-2018
01:40 PM
1 Kudo
I figured it out, for that problem at least. I had to edit the pg_hba.conf file to add the IP range of 'host' machine.
... View more
04-24-2018
02:48 PM
@Jay Kumar SenSharma I just came across this one the website, does this mean there is a possible combination of both soon? https://hortonworks.com/ecosystems/
... View more
04-24-2018
01:42 PM
@Jay Kumar SenSharma yes that was a browser coloring issue ! Thank you ! Fixed.
... View more
04-24-2018
10:46 AM
@Jay Kumar SenSharma Hi, this worked! The error does not show anymore. Can you kindly explain what was that error about? and if it effected previously installed services.
... View more
04-23-2018
12:48 PM
Well I found a solution, took me a lot of time. Figured that some application was using port 8080 by: C:\WINDOWS\system32>netstat -anob | findstr "8080"
TCP 0.0.0.0:8080 0.0.0.0:0 LISTENING 4700
TCP 127.0.0.1:8080 127.0.0.1:49327 TIME_WAIT 0
TCP 127.0.0.1:8080 127.0.0.1:49328 TIME_WAIT 0
TCP 127.0.0.1:8080 127.0.0.1:49329 TIME_WAIT 0
TCP 127.0.0.1:8080 127.0.0.1:49399 TIME_WAIT 0
TCP [::]:8080 [::]:0 LISTENING 4700
TCP [::1]:8080 [::1]:65314 TIME_WAIT 0
TCP [::1]:8080 [::1]:65315 TIME_WAIT 0
TCP [::1]:8080 [::1]:65316 TIME_WAIT 0
TCP [::1]:8080 [::1]:65343 TIME_WAIT 0
TCP [::1]:8080 [::1]:65344 TIME_WAIT 0
TCP [::1]:8080 [::1]:65391 TIME_WAIT 0
TCP [::1]:8080 [::1]:65392 TIME_WAIT 0 So I kill that application with PID 4700: C:\WINDOWS\system32>taskkill /PID 4700 /F
SUCCESS: The process with PID 4700 has been terminated. Start VM from virtualbox again and I am happy and Ambari is up and running! Cheers.
... View more