I have the same issue as @lmdrone . Hadoop 'find' command only supports two expressions and cloudera has removed org.apache.solr.hadoop.HdfsFindTool utils. How do we filter files based on modified time? Please bring back "org.apache.solr.hadoop.HdfsFindTool" ... View more

Any solution this ... View more

A very late reply to this topic, just to document the similar error I had using a Kafka client from a different Kerberos realm.   [2020-07-13 09:47:08,678] ERROR [Consumer clientId=consumer-1, groupId=console-consumer-57017] Connection to node -1 failed authentication due to: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Fail to create credential. (63) - No service creds)]) occurred when evaluating SASL token received from the Kafka Broker. Kafka Client will go to AUTHENTICATION_FAILED state. (org.apache.kafka.clients.NetworkClient)   Debugging showed: error code is 7 error Message is Server not found in Kerberos database crealm is REALM1.DOMAIN.COM cname is rzuidhof@REALM1.DOMAIN.COM sname is krbtgt/REALM2.DOMAIN.COM@REALM1.DOMAIN.COM   Situation is a HDP cluster being access using a client on a host joined to a different (IPA) domain. No trust. This works without trust, I think trust is only needed to use accounts from a different domain but we used keytabs and interactive kinit from REALM1 in REALM2 to access services in REALM1.   All that was needed to get this to work was one additional line in /etc/krb5.conf on the REALM2 servers under [domain_realm] realm1.domain.com = REALM1.DOMAIN.COM   We already had under [libdefaults]: dns_lookup_realm = true dns_lookup_kdc = true We also arranged DNS forwarding, but no reverse lookups. ... View more

@bgooley    In CDH 6.3.x, this appears to have changed and the "https.py" file is slightly different now.  It accepts the cipher_list as a configuration item.  The way we secured Port 900 is by doing these steps:   1) Check to see if RC4 (and other weak ciphers) are open on Port 9000:   openssl s_client -cipher RC4 -connect <server>:9000 -msg   2) Edit the "/etc/cloudera-scm-agent/config.ini" file   3) Under the "[Security]" section of the config.ini file, we added these lines:   # Custom Cipher List to close vulnerabilities for port 9000 cipher_list=HIGH:!DSS:!DH:!ADH:!DES:!3DES:!SHA1:!RC4:!aNULL:!eNULL:!EXPORT:!SSLv2:!SSLv3:!TLSv1   4) Restart the Cloudera CM-Agent:     sudo service cloudera-scm-agent restart   5) Wait a minute or so and then rerun the OpenSSL command and RC4 (and other weak ciphers, if you test them) are closed:   openssl s_client -cipher RC4 -connect <server>:9000 -msg   It would be great if Cloudera could add this to their documentation on how to add this additional security to the CM Agent. ... View more

Hi,   You also need to check on below configuration (If any).   1. Dynamic Resource Pool Configuration > Resource Pools - Check if jobs are exceeding any max values respective of the queue it's being submitted.   2. Dynamic Resource Pool Configuration > User Limits - Check if the maximum number of applications a user can submit simultaneously is crossing the default value (5) or the specified value. ... View more

It's a bug in Oozie. CoordActionCheckXCommand doesn't take care of SUSPENDED state. It only handles SUCCEEDED, FAILED and KILLED. protected Void execute() throws CommandException { try { InstrumentUtils.incrJobCounter(getName(), 1, getInstrumentation()); Status slaStatus = null; CoordinatorAction.Status initialStatus = coordAction.getStatus(); if (workflowJob.getStatus() == WorkflowJob.Status.SUCCEEDED) { coordAction.setStatus(CoordinatorAction.Status.SUCCEEDED); // set pending to false as the status is SUCCEEDED coordAction.setPending(0); slaStatus = Status.SUCCEEDED; } else { if (workflowJob.getStatus() == WorkflowJob.Status.FAILED) { coordAction.setStatus(CoordinatorAction.Status.FAILED); slaStatus = Status.FAILED; // set pending to false as the status is FAILED coordAction.setPending(0); } else { if (workflowJob.getStatus() == WorkflowJob.Status.KILLED) { coordAction.setStatus(CoordinatorAction.Status.KILLED); slaStatus = Status.KILLED; // set pending to false as the status is KILLED coordAction.setPending(0); } else { LOG.warn("Unexpected workflow " + workflowJob.getId() + " STATUS " + workflowJob.getStatus()); coordAction.setLastModifiedTime(new Date()); CoordActionQueryExecutor.getInstance().executeUpdate( CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_MODIFIED_DATE, coordAction); return null; } } } ... View more

I was also facing same issue like you. Then I had followed these steps  and it worked for me :   set hive.support.concurrency=true; set hive.enforce.bucketing=true; set hive.exec.dynamic.partition.mode=nonstrict; set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager; set hive.compactor.initiator.on=true; set hive.compactor.worker.threads=2;   Then I changed added hive.in.test property=true in the hive-site.xml file in /usr/lib/hive location.   After that I restarted the Hive from HUE and then ran the update command and it worked for me. ... View more

Hi @Harsh J ,  I just deleted around 80% of my data with "DELETE from table_name where register <= '2018-12-31'" My disks are pretty full (around 90%). After the deletion nothing happened (about freeing space). I restart Cloudera (Kudu, Impala, HDFS, etc.) and nothing. I add this two lines to Kudu configuration (in "Master Advanced Configuration Snippet (Safety Valve) for gflagfile" and "Tablet Server Advanced Configuration Snippet (Safety Valve) for gflagfile"): ``` unlock_experimental_flags=true flush_threshold_secs=120 ``` After restart Kudu, wait for the 120 secs.. nothing. ... View more

It happened to me when I was installing cloudera 6.3.1, What solved to me was:   1. run:    sed -i 's/SELINUX=enforcing/SELINUX=permissive/' /etc/selinux/config     2. config /etc/hosts: (just an exemple, set the host of all machines)   hostnamectl set-hostname master1.hadoop-test.com echo "10.99.0.175 master1.hadoop-test.com master1" >> /etc/hosts sed -i 's/\r//' /etc/hosts echo "HOSTNAME=master1.hadoop-test.com" >> /etc/sysconfig/network     3.   reboot     then: 4.  wget <a href="https://archive.cloudera.com/cm6/6.3.1/cloudera-manager-installer.bin" target="_blank">https://archive.cloudera.com/cm6/6.3.1/cloudera-manager-installer.bin</a>     5.    chmod u+x cloudera-manager-installer.bin   6.    ./cloudera-manager-installer.bin     ... View more