Solved

I took the Cloudera CCA 175 (CCA Spark and Hadoop Developer) exam on 12.10.2020 at 10.30 am IST. I was unable to use quotes and double quotes in the test environment. I tried various ways like copied it from spark documentation and used it but I cannot run it in the test environment. It throws the error illegal character '\u201d'. At last I lost my hope after 1.30 hours and requested for reschedule the exam.

Hello,  I have a cluster that currently runs with CDH and Cloudera Manager version 5.15.2 for both and I would like to upgrade only Cloudera Manager to 6.3.3. Is there any issue running the cluster with these different versions of CDH and CM? Thank you, 

hello comunnity. Currently I am going to implement a cluster of 20 hosts in total.The RDBMS is ORACLE, which will house the Cloudera scm, rman, nifi, etc. tables. What should be the minimum requirements for a database server.Oracle versionStorageRAMCPUThank you very much for your valuable help

Hi All - I'm new to Hadoop, recently installed HDP_2.5_virtualbox - trying to login to SQL - it prompts for a password, not sure the password - appreciate any help to login to Mysql. [root@sandbox ~]# Mysql -u root -p Enter password: 

Hello, We are using a beeline or impala-shell to extract data from hive tables as per requirement from the end-user. However, the request for data extraction is for high records for more than 1000 or sometimes more than 3000. It is very tedious to extract using the select query and dump to excel sheet. Is there any alternative way to take the output in a CSV file?. Like the output of select query moves to CSV file.  Please suggest.

While querying one of the Impala tables (out of hundreds of tables under the same database), I got the following exception: select count(*) from impala_table;Query: select count(*) from impala_tableQuery submitted at: 2020-10-23 03:12:42 (Coordinator: https://impalad:25000)ERROR: AnalysisException: Failed to load metadata for table: 'impala_table'CAUSED BY: TableLoadingException: Failed to load metadata for table: impala_db.impala_tableCAUSED BY: RemoteException: Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.checkOperation(StandbyState.java:88) at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.checkOperation(NameNode.java:1962) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOperation(FSNamesystem.java:1421) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3055) at org.apac

Hi Community. I am getting an error starting Hive in Cloudera Manager, CDP7. The error reads, "Command failed to run because service Hive has an invalid configuration. Review and correct its configuration. First error: Missing service dependency for Hive: YARN". How do I add the YARN dependency to an existing Hive service? Reinstalling Hive is not a really practical solution. Is there any way to add the dependency without reinstalling Hive? Thanks!

HDF 3.4.1  NIFI 1.9 - NIFI Provenance Repository filling disc 500GB I have a requirement to retain provenance for 5 days and made necessary changes  - provenance retaining hardly - 2 days and lesscontent_repo - 500GB   utilization 10%Provenence_repo - 500GB Utilization 98% flowfile_repo - 500GB   utilization 10% below are the configs  @MattWho @TimothySpann   Please advice 

Problem Description : Sqoop is pushing data daily from MSSQL to HDFS location in date format like year/month/day (one pull daily for delta of data) and on top of it, created external Hive table. But Past data at source RDBMS is getting updated frequently so sqoop has to pull and merge those changes with already pulled data in HDFS. With that, I want to delete data that is 60 days old. Is there any way to achieve it. Can partitions in Hive resolve it. 

Hi,I tried to get the CDP documentation in pdf using the pdf button on the online document. Its good, but I found out that some of the screenshot image became lot bigger and does not fit within the document(Some is perfectly fine). Is this can be repaired or I'm asking a bit too much.  In this example i used https://docs.cloudera.com/runtime/7.1.1/security-ranger-authorization/security-ranger-authorization.pdf Thank you very much.  the pdf button that i used this is the same page as the 1st picture 

We recently enabled kerberos and ldap to our cloudera 6.3.2 . And able to connect to Hive through JDBC connections successfully. But oozie/sqoop jobs that interact with metastore are failing with below error:-------------------------------------------------------------- javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)at java.security.Ac

Hi folks,  After install Cloudera CDP 7.1.3 we try to access on RANGER CONSOLE --> AUDIT --> ACCESS and we get this error/message"Error running solr query, please check solr configs. Collection not found: ranger_audits"Looking file catalina.out==================================================================================Oct 19, 2020 11:59:39 PM org.apache.ranger.server.tomcat.SolrCollectionBootstrapper connectINFO: Solr is in Cloud modeOct 19, 2020 11:59:39 PM org.apache.ranger.server.tomcat.SolrCollectionBootstrapper uploadConfigurationINFO: Config already exists with name ranger_auditsOct 19, 2020 11:59:41 PM org.apache.ranger.server.tomcat.SolrCollectionBootstrapper createCollection SEVERE: Error while creating collection in solr : org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://XXXXXXXXXXXX:8993/solr: Underlying core creation failed while creating collection: ranger_auditsOct 19, 2020 11:59:41 PM org.apache.range

Hi, Is there is any documentation available for OnPrem HDP to CDP Cloud Migration?Also let me know any document available for onprem HDP to OnPrem CDP migration. ThanksManish

Hello, I want to send and email using PutEmail with czech characters in email html body. I used UTF-8 charset, but it does not work, I'm keep getting emails with question marks instead of correct czech special characters. But subject text shows characters correctly. When checking data provenance, everything is ok.Can you please help me with processor configuration? What am I doing wrong?Screenshots of my setup below. Thanks a lotRgdsLudvik I'm getting message text in incorrect charset    

I have flowfile entries like the following:{\"CommandLine\":\"/bin/sh -c ( LC_MESSAGES=C;export LC_MESSAGES;df -k | awk 'NF == 1 { printf $0; next } { print }' ) > /opt/netiq/netiq/AM/tmp/netiqKykswU\",\"ConfigBuild\":\"1007.8.0010402.9\",\"ConfigStateHash\":\"788983332\",\"Entitlements\":\"15\",\"GID\":\"0\",\"ImageFileName\":\"/usr/bin/bash\",\"MD5HashData\":\"0883bc66ffeba0e89ee0a753ddbc1950\",\"NDRoot\":\"313466589192688920\",\"ParentBaseFileName\":\"agent\",\"ParentProcessId\":\"313467877387179481\",\"ProcessEndTime\":\"1602877133.023\",\"ProcessGroupId\":\"313466594618021306\",\"ProcessStartTime\":\"1602877133.016\",\"RGID\":\"0\",\"RUID\":\"1420\",\"RawProcessId\":\"52404\",\"SHA1HashData\":\"0000000000000000000000000000000000000000\",\"SHA256HashData\":\"c1c73170d33c0319a795bb05b1dbe7884b7582ec9f696d8a795e0ce692cb714e\",\"SVGID\":\"0\",\"SVUID\":\"1420\",\"SessionProcessId\":\"313467877387179481\",\"SourceProcessId\":\"313467877387179481\",\"SourceThreadId\":\"0\",\"TargetPr

I've recently created the following table in Impala....create table hpchistperf.oozie_job_stats(name string,start_time timestamp,end_time timestamp,rows_inserted integer,last_id_loaded integer,primary key(name,start_time))PARTITION BY HASH PARTITIONS 16stored as kudu; I read that there was no problem creating the table in Impala and it could be accessed in Hive since they share the same metadata. The problem is that when I run the query on Hive it gives me the following error:FAILED: RuntimeException java.lang.ClassNotFoundException: org.apache.kudu.mapreduce.KuduTableInputFormat

Hello, we are using Apache NiFi to process CSV data that contains time stamps without a specified time zone.  By default, it seems that NiFi considers timestamps parsed by CSVReader with a parse specification like "yyyy-MM-dd HH:mm:ss" to be in UTC, no matter what the user.timezone property of the JVM or the TZ environment variable is set to.  Is there a way to force the parsing to happen in a specific or the system time zone instead? Thanks,Hans

Hi all, I want to build a flow which brings up information about all versioned processgroup on this system. What I need is:- name of each processgroup- version of each processgroup- timestamp when this version was brought to this system Now I tested with NiFi-REST-API but I can not find the right request to do this.Could someone help me to get this works?Thanks.

Hello Team, We are planning to Impala UDFs and UDAs. To try things out we started by exploring the examples given in the cloudera github repo. After building the .so file using the make utility when I try to create the function using the following create function has_vowels (string) returns boolean location '/user/hive/udfs/libudfsample.so' symbol='HasVowels';     We are getting the following error ERROR: AnalysisException: Could not load binary: /<hdfs_path>/udfs/libudfsample.soUnable to load /var/lib/impala/udfs/libudfsample.9909.1.sodlerror: /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/impala/lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /var/lib/impala/udfs/libudfsample.9909.1.so)  Following are the error logs org.apache.impala.common.AnalysisException: Could not load binary: /user/parth.khatwani/udfs/libudfsample.soUnable to load /var/lib/impala/udfs/libudfsample.9909.1.sodlerror: /opt/cloud

Hi, I want to enable an on-demand metadata feature for impala as it gives a lot of improvement.   one of the features is that the cached metadata gets evicted automatically under memory pressure.    I want to know if this feature can be enabled in Cloudera CDH 5.14.4 version.   Has anyone implemented this on prod on CDH 5.14.4 version?

After running Impala query  select distinct(partition_date) parts from mddb_servt; I am getting below error: Execution time 10 seconds2) java.sql.SQLException: [Cloudera][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000, errorMessage:ExecQueryFInstances rpc query_id=937d334667doe010:4967d4b900000111 failed: Failed to get minimum memory reservation of 68.00 MB on daemon ec2-3-128-13.us-east-2:22000 for query 937d334667doe010:4967d4b900000111 because it would exceed an applicable memory limit. Memory is likely oversubscribed. Reducing query concurrency or configuring admission control may help avoid this error. Memory usage:Process: Limit=60.00 GB Total=50.68 GB Peak=50.70 GB  Buffer Pool: Free Buffers: Total=0  Buffer Pool: Clean Pages: Total=2.31 GB  Buffer Pool: Unused Reservation: Total=-2.29 GB  Free Disk IO Buffers: Total=1.10 GB Peak=1.15 GB  RequestPoo

Hi!I am running a scheduled job that consists of an insert-select query in hive 3.0/hdp 3 as the following Insert into table t1 as select * from t2 where timestamp > "predefined timestamp"The job was running flawless until out of sudden it started failing with the following error:Caused by: java.lang.IllegalArgumentException: bucketId out of range: 4147[2020-10-13 06:58:12,214] INFO - at org.apache.hadoop.hive.ql.io.BucketCodec$2.encode(BucketCodec.java:94)[2020-10-13 06:58:12,214] INFO - at org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.<init>(OrcRecordUpdater.java:271)[2020-10-13 06:58:12,214] INFO - at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat.getRecordUpdater(OrcOutputFormat.java:278)[2020-10-13 06:58:12,214] INFO - at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getRecordUpdater(HiveFileFormatUtils.java:350)[2020-10-13 06:58:12,214] INFO - at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getAcidRecordUpdater(HiveFileFormatUtils.java:336)[2020-10

So, we recently moved to using spark as HIVE engine instead of mr and we are seeing some significant improvements in certain queries which needs intermediate tables/storage to process. Can someone provide a complete or optimum list of configuration that we should be using that will not cause memory issues but still be able to get the best out of using spark as engine?Nodes : 30Cores: 16Memory: 112GB /nodeHadoop 2.6.0-cdh5.13.0Hive 1.1.0-cdh5.13.0

I am trying running nifi on AWS with a ELB fronting it.  I use ListenHTTP (port 7777)and need to do a health check via ELB.  It seems listenhttp only supports HEAD and POST.  ELB only supports health check via GET.  Any ideas how I can accomplish this to determine health status of port 7777 using ELB health checks?

hi all   We are performing now the change hostname configuration on production cluster according to the document - https://docs.cloudera.com/HDPDocuments/Ambari-2.7.0.0/administering-ambari/content/amb_changing_host_names.html The last stage is talking about – "in case NameNode HA enabled , then need to run the following command on one of the name node"  hdfs zkfc -formatZK -force  since we have active name node and standby name node we assume that our namenode is HA enable ?  but we want to understand  what are the risks when doing the following cli on one of the namenode hdfs zkfc -formatZK -force is the below command is safety to run without risks ?