Hello @Shelton,   There is no logs generated on that time interval. But i will check for yourquestions with my team and get back to you. Thanks for responding.   Best Regards, Vinod ... View more

Hi @Shelton,   We have disabled ranger authorization in Ambari & allow to run hive as end user instead of hive user. Still hiveserver2 is not coming up.   2020-03-26 05:36:05,838 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server <master node>:2181,<datanode1>:2181,<data node3>:2181 ls /hiveserver2 | grep 'serverUri=''] {} 2020-03-26 05:36:06,497 - call returned (1, 'Node does not exist: /hiveserver2') 2020-03-26 05:36:06,498 - Will retry 1 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)     Any clue on this ?       ... View more

Yes all were done. ... View more

Below are the steps to troubleshoot distcp:-  1. it is not problem with the hdfs or Kerberos or distcp but a MapReduce.  2. We tried to run a sample MR job to test, then it failed with the following exception Error: Java.io.IOException: initialization of all the collectors failed. Error in last collector was:java.io.IOException: Invalid “mapreduce.task.io.sort.mb”:3276. (The total amount of buffer memory to use while sorting files, in MB). It was expecting less than 2048. Changing this property able to run the distcp smooth.  I want to take a moment and say thanks to Shelton for responding it on time.  ... View more

@SwasBigData  I think you missed the outputyou  but you should still be able to locate the log. There is a step during the configure Ranger and DAS Databases that should have give you and output. After running the gen_embedded_ranger_db.sh located in /opt/cloudera/cm/bin on the Cloudera Manager host I will do a trial install tomorrow. ... View more

@desind  I can see the error  Authentication is not valid  but it seems you didn't use the format super:password->super:DyNYQEQvajljsxlhf5uS4PJ9R28=   instead, your input was as below according to the steps you shared. addauth digest super:password And then  delete  the znode  that should work [zk: xxx.unx.sas.com(CONNECTED) 2] deleteall /kafka-acl/Topic Please do that and revert         ... View more

Hello @Shelton, I have this same issue. I configured the metastore for your hive either MySQL.   But the below command returned [] ls /hiveserver2   Could you please help me with this. @north Do you recal how you configured this. ... View more

@eistre91 I have forwarded your post to my sales contacts. Let me know if you hear back from them.  ... View more

@stalsams  If you have the root access just  switch to user sqoop as root user # su - sqoop $ id uid=1020(sqoop) gid=1007(hadoop) groups=1007(hadoop)   Now you should be able to execute all sqoop commands with no need for a password     ... View more

I want latest file from hdfs dir.I am using below programm,but it get me all files into dataframe.I want to filter latest file and read only latest file.I am using scala val  fname=spark.read.csv("hdfs://ndwns001.ndw.leidos.com/automation_test/oozie/output/Bigdata_Counts/") ... View more

@Joe_Jim  By default, the broker binds to localhost. Can you share your server.properties and logs? I am of the opinion you need to update the server.properties listeners=PLAINTEXT://<IP:_TO_CHANGE>:9092 ... View more

@jking  I have a suspicion can you share a tokenized version of your  cluster /etc/hosts ? How many hosts do you have in your cluster and is the DNS working correctly? ... View more

Hi,     The issue has been resolved. I crossed checked my command and I was not passing the client certificate due to that I was getting this error.   Thanks to all for your support. ... View more

Hello,   Strangely package spark-atlas-connector_3_0_1_0_187 is missing in repository http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.1.0 ... So we must first bypass its installation so that the deployment continues then if necessary manually install spark-atlas-connector   # Fake rpm creation   [root@ambari ~]# yum install rpm-build [root@ambari ~]# mkdir -p rpmbuild/SPECS/ [root@ambari ~]# vi rpmbuild/SPECS/spark-atlas-connector.spec Name: spark-atlas-connector_3_0_1_0_187 Version: 3_0_1_0 Release: 187 Summary: - Group: - License: - URL: http:// Vendor: - Source: http:// Prefix: %{_prefix} Packager: Karthik BuildRoot: %{_tmppath}/%{name}-root %description Fake %prep %build %install %clean %files %changelog [root@ambari ~]# rpmbuild -ba rpmbuild/SPECS/spark-atlas-connector.spec [root@ambari ~]# mkdir -p repo/spark-atlas-connector [root@ambari ~]# cp rpmbuild/RPMS/x86_64/spark-atlas-connector_3_0_1_0_187-3_0_1_0-187.x86_64.rpm repo/spark-atlas-connector/ # Local YUM repository creation & registering   [root@ambari ~]# createrepo repo/spark-atlas-connector [root@ambari ~]# vi /etc/yum.repos.d/localrepo.repo [localrepo] name=spark-atlas-connector localrepo baseurl=file:///root/repo/spark-atlas-connector gpgcheck=0 enabled=1 [root@ambari ~]# yum makecache # Retry deployment   Have a nice day 🙂 ! ... View more

Hi,   Thanks for your response and sorry for the delayed answer. Your solution did not solve the issue because of the infrastructure our cloudera is installed on it. But we managed to solve it by backing up the needed folders, recreating the servers with same hostname & IP address but with newer OS and then we restored the folders and everything worked fine.   ... View more

Thanks for your response. Sorry for my the late response, but I wanted to pass this along in case someone runs into this issue. So I did some more testing and what I found is that I had an incompatible postgresql-jdbc driver. So I was upgrading from 9.6 to 10.x and it appears the jdbc driver I was using was pretty old and incompatible with version 10.x. Once I upgraded the postgresql-jdbc driver everything worked just fine. Here is what I did to resolve my issue:   wget https://jdbc.postgresql.org/download/postgresql-42.2.9.jar mv /usr/share/java/postgresql-jdbc.jar /usr/share/java/postgresql-jdbc.jar.old  (backing up the old driver) cp ~/postgresql-42.2.9.jar /usr/share/java/postgresql-jdbc.jar ambari-server setup --jdbc-db=postgres --jdbc-driver=/usr/share/java/postgresql-jdbc.jar     ... View more

@saivenkatg55    That could be a memory issue on your cluster. Can you share the below config set spark.executor.memory set yarn.nodemanager.resource.memory-mb set yarn.scheduler.maximum-allocation-mb Here are some links to help How to calculate node and executors memory in Apache Spark after adjusting that share the new output   ... View more

@Shelton  Please find the attach snap of error in hbase      ... View more

@Lewis @Shelton @Prakashcit  Hi All, I did ranger re-installation and now it is working, but if i integrate hive with ldap, again getting same null pointer exception.    Note: we integrated hive with ldap not ranger, just creating same ldap user in ranger (local) and trying to match it with LDAP user. whether it will create any problem? Kindly advice on this.   select * from asop.test; Error: Error while compiling statement: FAILED: NullPointerException null (state=42000,code=40000) ... View more

@kirkade  Unfortunately I have MySQL databases but if you could conserve that instalment I could try to reproduce your problem this weekend. I have been overloaded these past days ....To my experience reinstallation might not resolve the problem its usually a good learning curve to face a problem head one just imagine ypu were at a client site you wouldn't ask them to reinstall the environment. Just share the create and postgres load scripts and version so I can reproduce your problem.     ... View more