@jkibler  Your syntax looks okay save for a small change by fully qualifying the table name as the table doesn't belong to your schema?   SQL Server uses [dbo.tablename] so in the table parameter try --table dbo.table_name Hope that helps ... View more

@Manoj690    Please follow the below steps remember to skip steps you've already executed on the ambari database host.   # Install the mysql connector [optional if already executed] yum install -y mysql-connector-java ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar   # Enable auto start on boot [optional if already executed] sudo systemctl start mysqld [if not started] sudo systemctl enable mysqld I think you didn't create the Ambari backend database, please follow the steps below as root Assumption root password=w3lc0m31 ambari username = ambari ambari user password=ambari   mysql -u root -pw3lc0m31 CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambari'; GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%'; GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost'; GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'gaian-lap386.com'; GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'gaian-lap386.com' IDENTIFIED BY 'ambari'; FLUSH PRIVILEGES; # Log on as Ambari user and create the Ambari DB mysql -u ambari -pambari CREATE DATABASE ambari; USE ambari; SOURCE /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql; exit; After successfully running the below switch to the  ambari host CLI, the default mysql database port is 3306 anything else is not acceptable. The below command should pull parameters macting the below # grep 'jdbc' /etc/ambari-server/conf/ambari.properties server.jdbc.database=mysql server.jdbc.database_name=ambari server.jdbc.driver=com.mysql.jdbc.Driver server.jdbc.hostname=gaian-lap386.com server.jdbc.port=3306 server.jdbc.rca.driver=com.mysql.jdbc.Driver server.jdbc.rca.url=jdbc:mysql://gaian-lap386.com:3306/ambari server.jdbc.rca.user.name=ambari server.jdbc.rca.user.passwd=/etc/ambari-server/conf/password.dat server.jdbc.url=jdbc:mysql://gaian-lap386.com:3306/ambari server.jdbc.user.name=ambari # Start Ambari as root user # ambari-server start This time it should succeed. # To configure hive, oozie, ranger and rangerkms use this template mysql -uroot -pw3lc0m31 create database hive; grant all privileges on hive.* to 'hive'@'localhost' identified by 'hive'; grant all privileges on hive.* to 'hive'@'%gaian-lap386.com' identified by 'hive';   Substitute the values and names for the subsequent components after the Ambari startup ensure these values are matched in the subsequent component database setup Hope that helps ... View more

@Malthe Borch Nifi cluster coordinator relays on the zookeeper for the election. NiFi employs a Zero-Master Clustering paradigm. Each node in the cluster performs the same tasks on the data, but each operates on a different set of data. One of the nodes is automatically elected (via Apache ZooKeeper) as the Cluster Coordinator. All nodes in the cluster will then send heartbeat/status information to this node, and this node is responsible for disconnecting nodes that do not report any heartbeat status for some amount of time. Additionally, when a new node elects to join the cluster, the new node must first connect to the currently-elected Cluster Coordinator in order to obtain the most up-to-date flow. If the Cluster Coordinator determines that the node is allowed to join (based on its configured Firewall file), the current flow is provided to that node, and that node is able to join the cluster, assuming that the node’s copy of the flow matches the copy provided by the Cluster Coordinator. If the node’s version of the flow configuration differs from that of the Cluster Coordinator’s, the node will not join the cluster. Checklist Ensure you have 3 zookeepers (ensemble) to manage your Nifi cluster, and all should be up and running. Walkthrough Stop all the Nifi instances Ensure the 3 zookeepers are up and running Start the Nifi one at a time Validate that one nifi has been elected coordinator as is PRIMARY See screenshots Coordinator elected HTH ... View more

@Koffi Can you share your HDP setup? HDP Version, number of nodes, HA or not, Kerberized or not and share the exact error being thrown preferably logs ! Typically a node manager should run on the same node as the data node. HTH ... View more

@Sugumar Srinivasan My question is do you have something of value that you want to retrieve for that lost log? if so try the method below I had it documented but it has never occurred to me to use it Else the file will be recreated when that service restarts Recovery If a running program still has the deleted file open, you can recover the file through the open file descriptor in /proc/[pid]/fd/[num] . To determine if this is the case, you can attempt the following: $ lsof | grep "/var/log/Hadoop-yarn/yarn/yarn-yarn-timelineserver-hn0-myprha.log" If the above gives an output of the form: progname 5383 user 22r REG 8,1 16791251 265368 /path/to/file Take note of the PID in the second column, and the file descriptor number in the fourth column. Using this information you can recover the file by issuing the command: $ cp /proc/5383/fd/22 /path/to/restored/file HTH ... View more

@Swapnil Sonawane Your problem seems to be vast! The /etc/security/keytabs should be mounted as persistent volumes PVC stateful or use secrets to mount the files. You will need to share your deployment.yaml or whatever config HELM or other you are using, ... View more

@Surendaran Manickam Can you share your sqoop command the problem is the syntax URI. sharing your sqoop command could have helpyou only neede to masked sensitive data - HTH ... View more

@Pritam Konar Can you regenerate the keytabs and revert if the issue persists HTH ... View more

@Pritam Konar Can you check that below value in your Advancecore-site is set to TRUE hadoop.http.authentication.simple.anonymous.allowed=true Set I to this value hadoop.security.authentication=simple Please have a look at this document too Kerberos authentication on Windows HTH ... View more

@Jeff Yang If you found this answer addressed your question, please take a moment to log in and click the "accept" link on the answer. That would be a great help to Community users who'd have the same question in mind Thank you . ... View more

@Thomas Poetter it seems you DB isn't running can you just check as below can you ensure you install the mysql connector you can re-run this without any issues yum install -y mysql-connector-java Check MySQL status #  ps aux | grep mysql or # mysqladmin -u root -p status Enter password: Uptime: 1697  Threads: 15  Questions: 17283  Slow queries: 0  Opens: 73  Flush tables: 2  Open tables: 99  Queries per second avg: 10.184 if not then set it up manually as the root user please re-adapt create database rangerkms; grant all privileges on rangerkms.* to rangerkms@'localhost' identified by 'rangerkms'; grant all privileges on rangerkms.* to rangerkms@'%.[your_domain]' identified by 'rangerkms'; Then retry ... View more

@Jeff Yang My personal view after the merger it is ONLY and means ONLY serious bugs reported by clients can trigger some patch development by HWX as the teams are busy integrating the best of the 2 worlds into the new CDP, personally, I pray Ranger/Atlas wins the war against Sentry. There are 3 types of releases see below Major Release “Major Releases” (X.y.z) means Software releases that deliver major and minor feature development and enhancements to existing features. They incorporate all applicable Error corrections made in prior Major Releases, Minor Releases, and Maintenance Releases. Minor Release “Minor Releases” (x.Y.z) means Software releases that deliver minor feature developments, enhancements to existing features and Error corrections. They incorporate all applicable Error corrections made in prior Minor Releases and Maintenance Releases. For Hortonworks Products, Hortonworks typically has two Minor Releases per year. Maintenance Release “Maintenance Releases” (x.y.Z) means Software releases that deliver Error corrections that are severely affecting a number of customers and cannot wait for the next Major Release or Minor Release. They incorporate all applicable Error corrections made in prior Maintenance Releases. There is no predefined schedule for Maintenance Releases. They are released as needed based on customer feedback and outstanding Errors. My best bet is you could expect a maintenance release for the last HDP release the HDP 3.1 End of Full Support is set for December 2021 this show how serious Cloudera want to shelve the HWX brand and move on very fast I am sure that most customers will be encouraged to migrate to the new combined offering Cloudera Data Platform [CDP] ie HDF has been rebranded Cloudera DataFlow [CDF] Reference: https://www.cloudera.com/legal/hwx/support-services-policy.html ... View more

@Rajesh Reddy Any updates? ... View more

@Koffi On your edgenode or node where you copied/imported the keytab ensure you have install the kerberos client software, if it wasn't done do the below steps assuming you are on RHEL or Centos Remember to revalidate the krb5.conf it should be an exact copy as that on the KDC server. # yum install -y krb5-libs krb5-workstation Just for curiosity to be sure what keytab exactly did you copy can you share the name? Assuming your Kerberos realm is KOFFI.COM and suppose your new host is edgenode.koffi.com. Once the KOFFI.COM realm configuration file (/etc/krb5.conf) has been copied from the KDC to edgenode, use the kadmin protocol set up on the KDC to administer the Kerberos database remotely, directly from edgenode. Add a host principal for our new host and store the host’s secret key in the local keytab file. (kadmin can find the Kerberos admin server from the krb5.conf file you copied.) edgenode # kadmin -p admin/admin Authenticating as principal admin/admin@KOFFI.COM with password. Enter password: ****** kadmin:  ank -randkey -policy hosts host/edgenode.koffi.com Principal "host/edgenode.koffi.com@KOFFI.COM" created. kadmin:  ktadd -k /etc/krb5.keytab host/edgenode.koffi.com Entry for principal host/edgenode.koffi.com with kvno 3, encryption type     Triple DES cbc mode with HMAC/sha1 added to keytab WRFILE:/etc/krb5.keytab. kadmin:  quit Now when you run your connection on the beeline CLI shell> beeline -u "jdbc:hive2://<hostname>:10000/default;principal=hive/<FQDN@KOFFI.COM>" You shouldn't get "No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)" Please do that and revert ... View more

@Koffi In your initial posting, you shared this command shell> kinit mapr/<FQDN@REALM> -k -t /home/user1/Desktop/hive.service.keytab shell> beeline -u "jdbc:hive2://<hostname>:10000/default;principal=mapr/<FQDN@REALM>" Where is the mapr coming from , was this external host part of the cluster before kerberization? HTH ... View more

@Koffi While logged in as hive can you share the output of $ klist Bizarre ... View more

@Madhura Mhatre The hdfs balancer utility that analyzes block placement and balances data across the DataNodes. The balancer moves blocks until the cluster is deemed to be balanced. A threshold parameter is a float number between 0 and 100 (12.5 for instance). From the average cluster utilization (about 50% in the graph below), the balancer process will try to converge all data nodes' usage in the range [average - threshold, average + threshold]. In the current example: - Higher (average + threshold): 60% if run with the default threshold (10%) - Lower (average - threshold): 40% You can easily notice that the smaller your threshold, the more balanced your data nodes will be. For very small threshold, the cluster may not be able to reach the balanced state if other clients concurrently write and delete data in the cluster. A threshold of 15 should be okay $ hdfs balancer -threshold 15 or by giving the list of datanodes $ hdfs balancer -threshold 15 -include hostname1,hostname2,hostname3 HTH ... View more

@Hari Kishan Sangana With a one-line sentence, unfortunately, no member can help. We need basic info like HDP version/ the Jenkins integration with HDP, whether the cluster is kerberized or not and the OS. And the stack trace error being thrown some logs etc. just some tangible info that can help us HTH ... View more

@fares boudraa I think this should resolve your problem Hue is not allowed to impersonate hdfs Please revert if your problem persist or just accept and close the thread HTH ... View more

@Voruganti Vishwanath What is the HDP and zeppelin version? Can you share your cluster configuration? What values do you have for the below parameters? yarn.nodemanager.resource.cpu-vcores yarn.scheduler.minimum-allocation-vcores yarn.scheduler.maximum-allocation-vcores yarn.nodemanager.resource.cpu-vcores are by default ~80% of total vCPUs available on the machine. HTH ... View more

@Rajesh Reddy In addition to @Vipin Rathor I had a real production of setting up an HA KDC environment which almost is the case here. You will need to create a cron job that propagates users created on KDC1 to KDC2 and vice versa. If that makes sense to you here is the procedure My assumption: You are running 2 working KDC's on RHEL/Centos and you have different users on both HDP clusters but want them to access services on either cluster. Whether the krb5,conf file is managed by Ambari or not, the realm specification for your realm should look something like CUSTOMER.COM = {  kdc = kdc1.customer.com  kdc = kdc2.customer.com  master_kdc = kdc1.customer.com } Read carefully as this is a 2-way set up some steps have to be done on both KDC1 and KDC2 especially the crontab job should propagate in both ways as this creates the keytab from one KDC to the other. Host keytabs must now be created for the SLAVE KDC2. Execute the following commands from the Master KDC1:  # kadmin  kadmin: addprinc -randkey host/kdc1.customer.com  kadmin: addprinc -randkey host/kdc2.customer.com Extract the host key for the Slave KDC and store it on the hosts keytab file, /etc/krb5.keytab.slave: # kadmin: ktadd –k /etc/krb5.keytab.slave host/kdc2.customer.com Copy /etc/krb5.keytab.slave to kdc2.customer.com and rename the file to /etc/krb5.keytab Update /etc/services on each KDC host, if not present: krb5_prop       754/tcp               # Kerberos slave propagation Install xinetd on the hosts of the Master and Slave KDC, if not already installed, to enable kpropd to execute: yum install xinetd Create the configuration for kpropd on both the Master KDC1 and Slave KDC2 hosts: Create /etc/xinetd.d/krb5_prop with the following contents. service krb_prop {         disable         = no         socket_type     = stream         protocol        = tcp         user            = root         wait            = no         server          = /usr/sbin/kpropd } Configure xinetd to run as a persistent service on both the Master and Slave KDC hosts: # systemctl enable xinetd.service # systemctl start xinetd.service Copy the following files from the Master KDC host to the Slave KDC host: /etc/krb5.conf  /var/kerberos/krb5kdc/kadm5.acl  /var/kerberos/krb5kdc/kdc.conf /var/kerberos/krb5kdc/kpropd.acl /var/kerberos/krb5kdc/.k5.CUSTOMER.HDP Perform the initial KDC database propagation to the Slave KDC: Initially running this command by hand, you should see something similar to the following: # kdb5_util dump /usr/local/var/krb5kdc/slave_datatrans # kprop -d -f /usr/local/var/krb5kdc/slave_datatrans kdc2.customer.com  3234 bytes sent. Database propagation to kdc2.customer.com: SUCCEEDED The slave server will now synchronize its principal database with the master server. The Slave KDC may be started at this time: # systemctl enable krb5kdc  # systemctl start krb5kdc Script to propagate the updates from the Master KDC to the Slave KDC. Create a cron job, or the like, to run this script on a frequent basis. #!/bin/sh #/var/kerberos/kdc-slave-propogate.sh kdclist = "kdc2.customer.com" /sbin/kdb5_util dump /usr/local/var/krb5kdc/slave_datatrans for kdc in $kdclist  do    /sbin/kprop -f /usr/local/var/krb5kdc/slave_datatrans $kdc   done Hope that helps ... View more