Member since
03-02-2021
43
Posts
1
Kudos Received
0
Solutions
06-14-2021
12:24 AM
I tried with the logs but unable to find any traces with the keywords. I even tried to check the logs by killing all jobs.
... View more
06-13-2021
09:31 PM
I am facing issue with Oozie server after restarting HDP stack. Oozie server service starts and stops after a minute or 2. I tried to follow the solutions provided to similar issue but not helping.
Following is the output in oozie-error.log file.
2021-06-06 15:14:41,087 WARN ConfigurationService:523 - SERVER[hdp-master.mydomain.com] Invalid configuration defined, [oozie.authentication.jwt.cookie]
2021-06-06 15:14:41,087 WARN ConfigurationService:523 - SERVER[hdp-master.mydomain.com] Invalid configuration defined, [hadoop.security.credential.provider.path]
2021-06-06 15:14:41,087 WARN ConfigurationService:523 - SERVER[hdp-master.mydomain.com] Invalid configuration defined, [credentialStoreClassPath]
2021-06-06 15:14:41,091 WARN Services:523 - SERVER[hdp-master.mydomain.com] System ID [oozie-oozi] exceeds maximum length [10], trimming
2021-06-06 15:14:41,208 WARN ConfigUtils:523 - SERVER[hdp-master.mydomain.com] Using a deprecated configuration property [oozie.service.AuthorizationService.security.enabled], should use [oozie.service.AuthorizationService.authorization.enabled]. Please delete the deprecated property in order for the new property to take effect.
2021-06-06 15:14:44,475 WARN SparkConfigurationService:523 - SERVER[hdp-master.mydomain.com] Deprecated property found in configuration: oozie.service.SparkConfigurationService.spark.configurations.ignore.spark.yarn.jarUse oozie.service.SparkConfigurationService.spark.configurations.blacklist instead.
2021-06-06 15:14:44,475 WARN SparkConfigurationService:523 - SERVER[hdp-master.mydomain.com] Spark Configuration could not be loaded for *: /usr/hdp/current/spark-client/conf does not exist
... View more
Labels:
06-13-2021
09:23 PM
As a work around I copied the existing directory structure on the server running HIVE server service and that helped. The query gets executed and the output is copied on HIVE server. I copy the output manually to the Edge node for developers. Not sure if there are any techniques so that we can copy the processed HDFS data directly on local fs of Edge node.
... View more
06-13-2021
09:20 PM
@Shelton Any hints on this issue? Thanks.
... View more
06-11-2021
12:32 AM
@shobikas @vidanimegh Pls suggest thanks.
... View more
06-10-2021
05:17 AM
Hello,
I am having HDP 3.1.4 cluster.
I have couple of hql scripts on the Edge node.
I am executing the hql scripts on Edge node. Queries are getting processed on HIVE server but unable to move the processed files from HIVE server to the Edge server.
How to make sure that after processing, the files are moved from HDFS fs on HIVE server to the Edge node? Thanks.
Regards,
Amey.
... View more
Labels:
06-09-2021
04:28 AM
I compared the hive set ; between HIVE 2.x & HIVE 3.x In HIVE 3.x following value is 'nonstrict'. Is that causing the issue? hive.exec.dynamic.partition.mode=nonstrict The problem is, if I execute the query which contains static & dynamic parameters I am getting the error. If I just keep it static or dynamic parameters, no problem. Some more parameters which might be causing the issue but I am not too sure = hive.optimize.dynamic.partition.hashjoin=true
hive.optimize.sort.dynamic.partition=false
hive.server2.support.dynamic.service.discovery=true
... View more
06-09-2021
04:24 AM
Problem is still there. 21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.0-1127.19.1.el7.x86_64
21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:user.name=eagledev
21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/user1
21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/user1
21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=hdp-slave1.mydomain.com:2181,hdp-slave2.mydomain.com:2181,hdp-master.mydomain.com:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@5ace1ed4
21/06/09 16:50:22 INFO zookeeper.ClientCnxn: Opening socket connection to server hdp-slave1.mydomain.com/10.200.104.188:2181. Will not attempt to authenticate using SASL (unknown error)
21/06/09 16:50:22 INFO zookeeper.ClientCnxn: Socket connection established to hdp-slave1.mydomain.com/10.200.104.188:2181, initiating session
21/06/09 16:50:22 INFO imps.CuratorFrameworkImpl: backgroundOperationsLoop exiting
21/06/09 16:50:22 INFO zookeeper.ClientCnxn: Session establishment complete on server hdp-slave1.mydomain.com/10.200.104.188:2181, sessionid = 0x279ef5fd2c3006b, negotiated timeout = 60000
21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Session: 0x279ef5fd2c3006b closed
21/06/09 16:50:22 INFO zookeeper.ClientCnxn: EventThread shut down
org.apache.curator.CuratorZookeeperClient.startAdvancedTracer(Ljava/lang/String;)Lorg/apache/curator/drivers/OperationTrace;
Beeline version 3.1.0.3.1.4.0-315 by Apache Hive
0: jdbc:hive2://hdp-slave1.mydomain.com:2 (closed)>
... View more
06-07-2021
10:02 PM
I ensured that the path exists on local system i.e. home/user1/user1_Run_Jar/AB_HIVE_1805 2021/neo4j-graph-preparation/data-files/user_stats & has 777 permissions recursively. But still same error.
... View more
06-03-2021
07:51 AM
I am trying to install CDH 5.16.2.4505 version. Getting following error while doing cluster setup = Any hints how to fix this issue? Thanks. I selected Hadoop core components only. Thanks & Regards, Amey.
... View more
Labels:
- Labels:
-
Cloudera Manager