Q&A

Kudu rebalance tool crashes - when I run it in command line as well as when I use Cloudera Manager UI. Here is the stderr displayed in the Cloudera Manager: + exec kudu cluster rebalance master1.domain.com,master2.domain.com,master3.domain.com --max_moves_per_server=5 --max_run_time_sec=0 --max_staleness_interval_sec=300terminate called after throwing an instance of 'std::regex_error' what(): regex_error*** Aborted at 1553857242 (unix time) try "date -d @1553857242" if you are using GNU date ***PC: @ 0x7fbfc3637207 __GI_raise*** SIGABRT (@0x3ec00005c50) received by PID 23632 (TID 0x7fbfc5c83a00) from PID 23632; stack trace: *** @ 0x7fbfc5642680 (unknown) @ 0x7fbfc3637207 __GI_raise @ 0x7fbfc36388f8 __GI_abort @ 0x7fbfc3f467d5 __gnu_cxx::__verbose_terminate_handler() @ 0x7fbfc3f44746 (unknown) @ 0x7fbfc3f44773 std::terminate() @ 0x7fbfc3f44993 __cxa_throw @ 0x7fbfc3f99dd5 std::__throw_regex_error() @ 0x931c32 std

We have already 10+ GB logs for Kudu Masters and 30+ GB logs for Kudu Tablet Servers. How to limit the number of log files to keep? The Kudu Configuration Reference mentiones only a limit on a log file size (--max_log_size), but no limit for the number of log files.

Short version   cloudera_scm_server is crashing during final configuration.  I reach Add Cluster - Configuration - Command Details and fail on Formatting the name durectories of the current Namenode and Creating Oozie Database Tables.   It's a hard failure - the cloudera_scm_server crashes.   I'm probably missing some little detail in my setup but haven't found anything yet in either this community or the usual onlne resources.   cloudera_scm_server.log shows:   2019-03-28 17:04:09,030 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:2019-03-28T17:04:09.030Z2019-03-28 17:04:09,734 WARN scm-web-92:com.cloudera.server.cmf.descriptor.components.DescriptorFactory: Could not generate client configs for service: YARN (MR2 Included) Caused by: com.cloudera.cmf.service.config.ConfigGenException: Unable to generate config of 'mapreduce.application.framework.path'   a

Hi All,I am having an issue in restarting Ranger Service via Ambari. I deployed a cluster via cloudbreak then added the Ranger service, I then had to make a config change and when I go to Ranger > Restart > Restart All Affected I get the attached issue:Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/RANGER/package/scripts/ranger_admin.py", line 241, in <module> RangerAdmin().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute method(env) File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 971, in restart self.stop(env, upgrade_type=upgrade_type) File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/RANGER/package/scripts/ranger_admin.py", line 61, in stop Execute(format('{params.ranger_stop}'), environment={'JAVA_HOME': params.java_home}, user=params.unix_user) File "/usr/lib/ambari-agent/lib/resource_management/core

How to denied use rm with -skiptrash on HDFS?I want to block any attempts to delete a file with the -skiptrash option.

Hi, I use Impala UI, under /queries tab, in order to get runtime metrics for queries. I'm a little bit confused about the 'duration' and 'waiting time' metrics. What is being computed and tracked in each of them? I have also timed how long it takes to run the query (since statement is submitted until all rows are fetched) from the client side (I use Ibis framework to submit queries to Impala) and I can see a much smaller runtime than the one reported in the UI. Why is that? Is perhaps the metrics reported an aggregated computing time across all impalad nodes? Thanks,Paulo.

Hi All - We restarted cloudera manager today with a change in mysql database. we see connnection refusedexceptions on all agent logs and cloudera manager as in below screenshot.Why this happens ?  

Good day guys, im newby in Cloudera and wanted to ask 2 questions. 1) I got 20TB of data and i should migrate it to 10 servers, do i need to have 20TB of disk on each server ?2) How do i organize the right HDFS model (NameNode, DataNode, SecondaryNameNone) on those 10 servers ? Thanks, i hope to receive the answer very soon )  

Hi All, i tired to start and stop my clodera manger, wheire i need to upgrader the VM with RAM,and storagefirst i tried to stop the clodera manger#service cloudera-scm-server stopclouderamanger is already stoppedbut still the UI i was able to access. so i found the process id and kill the process.  kill -9 pidand stoped the agent.service cloudera-scm-agent stop and shutdown the VM and allaocted the resourses . and started the VM . now i started the service .service cloudera-scm-server startservice cloudera-scm-agent start after startign the service . i see error in the logs2019-03-22 07:25:40,874 INFO 1894048261@scm-web-0:com.cloudera.server.web.cmf.AuthenticationFailureEventListener: Authentication failure for user: '__cloudera_internal_user__mgmt-ACTIVITYMONITOR-06a4fc9d25f0cecc59951cc2023da8a0' from 192.xxx.x.xx2019-03-22 07:25:41,317 INFO 1894048261@scm-web-0:com.cloudera.server.web.cmf.AuthenticationFailureEventListener: Authentication failure fo

Hi community,I am studying ksu for some use cases and found this link:https://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-user/ksu.htmlI have a user1 with KDC entry and keytab. Just before running ksu, I kinit user1 to get Kerberos ticket:[user1@server1 ~]$ klistTicket cache: FILE:/tmp/krb5cc_1003293697Default principal: user1@EXAMPLE.COMValid starting Expires Service principal02/18/2019 09:13:12 02/19/2019 09:13:12 krbtgt/EXAMPLE.COM@EXAMPLE.COMThen, I want user1 to ksu user2. For this to work, I have created a .k5login file on user2 home directory with user1@EXAMPLE.COM on its content.Than, I launch ksu with user1 but found this issue:[user1@server1 ~]$ ksu user2ksu: Server not found in Kerberos database while verifying ticket for serverAuthentication failed.Looking for an error on /var/log/krb5kdc.log, I found that one:UNKNOWN_SERVER: authtime 0, user1@EXAMPLE.COM for krbtgt/UNKNOWN_DOMAIN@EXAMPLE.COM, Server not found in Kerberos databaseAs the error states, service principal

Hello.Is there a way to change in Hue the default database for user to another one?

I created a UDA function which was written in C++. When I try to test it on some tables, it seems that when the table is stored as several parquet files, the UDA function will cause impala to crash. The error message is: Socket error 104: Connection reset by peer. However, the UDA function works fine with tables that stored in only one file.I checked the error logs but didn't find any useful information. The Version of Impala:Version 2.11.0-cdh5.14.4 Here is the UDA function:#include "funnel-uda.h"#include <sstream>#include <iostream>#include <algorithm>#include <vector>#include <string>#include <stdlib.h>using namespace impala_udf;using namespace std;template <typename T>StringVal ToStringVal(FunctionContext* context, const T& val) { stringstream ss; ss << val; string str = ss.str(); StringVal string_val(context, str.size()); memcpy(string_val.ptr, str.c_str(), str.size()); return string_val;}

Hi All,Trying to add an Azure Database for Postgres (v10). I am able to access it using psql from my host however when trying to add it to Hortonworks as an External Source gives the following issue:Kind Regards,Nathaniel Vala

Is CCA 175 Certification stopped. I already purchased but when I try to schedule the exam through local exam site, the customer support said "Cloudera is not launching any exams now" please check with them.  Do anyone have any idea about it?I already purchased the course not sure how can I get it back incase if they have stopped conducting exams?

Hi, I need to retrieve only the last entry in a given partition if there are multiple entries therein Assume I create an external table partitioned by date:create external table test_lb (field1 string, field2 string, field3 string)  partitioned by (year string, month string , day string, host string)  row format delimited fields terminated by ','Then I insert multiple records to same partition. i,e. same year,month,dayinsert into test_lb partition (year="2013", month="07", day="28") values ("foo1", "FOO2", "FOO3");insert into test_lb partition (year="2013", month="07", day="28") values ("foo4", "FOO5", "FOO6"); How do I retrieve just the most recent entry via a query... is there an inbuilt way to get only the latest values

Hi, We have HA enabled for the metastore and have 2 instances running.  Is it possible to configre a load balancer in front of these ? and if so is this normal practice ? Thanks 

I've got a new HDP 3.1.0 installation and I've started seeing failures in the hiveserver2.log about replication. Something is trying to run the following command and it isn't me. This is on a completely isolated system that isn't exposed outside of my vpn so I'm not sure what's going on.repl dump `*` from 71931 with ('hive.repl.dump.metadata.only'='true', 'hive.repl.dump.include.acid.tables'='true')The failure message is this which I'm assuming is normal because the NOTIFICATION_LOG doesn't keep records forever.2019-03-24T12:31:17,394 ERROR [HiveServer2-Background-Pool: Thread-209]: metastore.HiveMetaStoreClient (:()) - Requested events are found missing in NOTIFICATION_LOG table. Expected: 71932, Actual: 121582. Probably, cleaner would've cleaned it up. Try setting higher value for hive.metastore.event.db.listener.timetolive. Also, bootstrap the system again to get back the consistent replicated state.

I have a cluster with 3 tservers.  When running a workload that heavily reads from the cluster, 1 of the 3 tservers is reaching nearly 100% CPU utilization while the other two are less than 10%.  The tablets are equally balanced amongst the 3.   I am thinking that by chance, all my data used in this particular workload happens to reside on the 1 tserver.  thoughts?  How might I diagnose this further?

Hi, I'm following Douglas Eadline's tutorial http://www.informit.com/store/hadoop-and-spark-fundamentals-livelessons-9780134770864 and try to compile the WordCount example, but I fail. Eadline suggests to comile it with hadoop-core.jar, which was renamed in newer versions as I understand.I tried with javac -classpath /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core.jar -d wordcount_classes WordCount.java but I still get a load of errors. Any suggestions? Where can I find the correct jar?WordCount.java:27: error: package org.apache.hadoop.conf does not existimport org.apache.hadoop.conf.Configuration; ^WordCount.java:28: error: package org.apache.hadoop.conf does not existimport org.apache.hadoop.conf.Configured; ^WordCount.java:29: error: package org.apache.hadoop.fs does not existimport org.apache.hadoop.fs.Path; ^WordCount.java:30: error: package org.apache.hadoop.io does not exist

Hello everyone, please help!I have files whose name is formed by a date in the format "yymmdd", for example:TRB_ASE_D190125.txtTRB_TIT_D190125.txtI need to get only the files whose names match the current date. For example, if the date was "190125", the 2 subsequent files would match that criterion.I am using the ListFile processor to get the files and put this regular expression using the "now" function in the File Filter, but it does not work:^TRB_[A-Z]+_D${now():format('yymmdd')}.TXTThanks in advance!

Hi All - We have tried to move Cloudera manager mysql to other disk , when we made changes and restarted cloudera-scm-server logs says nav database not found. we created the nav database in mysql and restarted now it says authenctication failure for admin user. Would you please advise why this is happening and how to fix.Thanks in advance.

How to explain the createtime (1551243941685 ) and Timestamp ( Wed Feb 27 2019 16:05:41 GMT+1100 (Australian Eastern Daylight Time)?Thank you.

Hi at all,I have a JSON in my FlowFile content like this{   "headers": { "key_1":"value_1", ...... "key_n":"value_n"},   "info": {                "key_1":"value_1",                 ......                               "key_n":"value_n"},  "payload": {                "key_1":"value_1",                 ......                               "key_n":"value_n"}} my aim in the flow is:split the original JSON into three attributes (headers, payload and info)route on a global variable into two subflowsmerge headers+info+core attributesmerge headers+info+payload attributes add core

error_ambari.txtHi,I try to install ambari agent in 2 nodes with the ambari wizard but is fails, I review the logs and I saw the following error:Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)I attached the log file and hope some can help me with this Regards.Saul Ramirez