Member since
02-01-2019
650
Posts
143
Kudos Received
117
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2612 | 04-01-2019 09:53 AM | |
1376 | 04-01-2019 09:34 AM | |
6473 | 01-28-2019 03:50 PM | |
1484 | 11-08-2018 09:26 AM | |
3609 | 11-08-2018 08:55 AM |
12-29-2016
12:13 PM
@nyadav If you run jobs in yarn-cluster mode then the above 3 should be set in all the nodes, you can add these through ambari so that it will push the changes to all the nodes.
... View more
12-29-2016
11:54 AM
2 Kudos
I've noticed that 24 hours after starting, the Spark Thrift server encounters authentication failures. By the way i am on HDP 2.4.2 [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
The message directly before this (usually several hours beforehand) is
WARN UserGroupInformation: Exception encountered while running the renewal command. Aborting renew thread. ExitCodeException exitCode=1: kinit: Ticket expired while renewing credentials
... View more
Labels:
- Labels:
-
Apache Spark
12-29-2016
11:46 AM
1 Kudo
@nyadav You need to add below options to your spark-env.sh.You should be able to run the pyspark jobs on 2.7. Let me know if you face any issue. export PYSPARK_PYTHON=/usr/local/bin/python2.7
export PYSPARK_DRIVER_PYTHON=/usr/local/bin/python2.7
export SPARK_YARN_USER_ENV="PYSPARK_PYTHON=/usr/local/bin/python2.7"
... View more
12-21-2016
07:54 PM
Very well explained @Rajkumar Singh !!
... View more
12-14-2016
07:40 AM
1 Kudo
@akshay singh Can you confirm if the HDP version and the dependencies version in your pom.xml are same ? Take a look at https://community.hortonworks.com/articles/27999/a-maven-pomxml-for-building-storm-topologies.html
... View more
11-28-2016
05:13 PM
Great thanks for sharing..
... View more
11-26-2016
07:00 PM
1 Kudo
@Nico Inhoffen I believe we have resolved this BUG. In kerberized cluster Storm-kafka spout will get the host and port values from the endpoints and the null values were due to a bug.
... View more
11-26-2016
06:55 PM
@Karan Alang
Here is the correct command : bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=<a href="http://sandbox.hortonworks.com/">sandbox.hortonworks.com</a>:2181 --list --topic kafka1_topic ref: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Authorization+Command+Line+Interface
... View more
11-26-2016
06:40 PM
@Huahua Wei Yes , you can safely kill the pid and delete the pid file from hdfs.
... View more
11-26-2016
06:14 PM
@Varma vetukuri This post should help you https://community.hortonworks.com/questions/44564/map-reduce-logging-using-log4j-and-yarn-log-aggreg.html
... View more