Q&A

On CDP Private Cloud 7.1.7, and running without rack awareness (an intentional choice due to some of the older servers running with smaller data disks), I am (as would be expected) seeing this spammed to the namenode logs: 2022-08-24 09:41:50,253 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Not enough replicas was chosen. Reason:{NO_REQUIRED_STORAGE_TYPE=1} Because I'm not logging INFO messages, I can eliminate those entries with something like: hadoop daemonlog -setlevel nnserverhost:9870 org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy WARN What's the best way to persist that across a restart?

Good morning,trying to install Cloudera Manager agents in a RH7.9 (this should be supported as seen in the Cloudera support matrix)I realized that this is not supported ¿!!!?The packages installed in the server are newer as expected (they are from rh7.9) and the cloudera manager is looking for older versions...What crazy is having a support matrix if this is not applied  any idea? downgrade dependencies are not considered. 

Hi , I'm trying to create a host and after enabling auto-tls .browser shows that certificate is invalid and not secure. I got failed to receive heartbeat from agent in the step of installing agent.  

Hi ,We are having four nodes CDP 7.1.7 SP1 cluster. Currently , when a normal hive user creates a database, the database ownership is getting assigned to hive:supergroup  in /external/hive which is default. We want the /external/hive database ownership to be assigned to the user who is creating a database. For example:User "Test " is creates one database, the ownership should be like:drwxr-xr-x - test supergroup 0 2022-07-05 17:35 /warehouse/tablespace/exteranl/hive/test_db Kindly let us know if this is possible.

Hi Cloudera gurús, This is my CDP. 3 Master Nodes+3 Worker Nodes HA enabled and testing it. Here is the issue: when I shut down Master 2 some queries are randomly failing showing this: # impala-shell -i haproxy-server.com -q "use dbschema; select * from table_foo limit 10;"Starting Impala Shell without Kerberos authenticationWarning: live_progress only applies to interactive shell sessions, and is being skipped for now.Opened TCP connection to haproxy-server.com:21000Connected to haproxy-server.com:21000Server version: impalad version 3.4.0-SNAPSHOT RELEASE (build 0cadcf7ac76ecec87d9786048db3672c37d41c6f)Query: use dbschemaQuery: select * from table_foo limit 10Query submitted at: 2022-03-23 11:28:37 (Coordinator: http://worker1:25000)ERROR: ImpalaRuntimeException: Unable to initialize the Kudu scan nodeCAUSED BY: AnalysisException: Unable to open the Kudu table: dbschema.table_fooCAUSED BY: NonRecoverableException: cannot complete before timeout: KuduRpc(method=GetTableSchema, tablet=

Hello,i'm using a small 3Master+3workers cluster [Cloudera Manager 7.3.1 (Runtime 7.1.6)]Postgresql HA configured (not embedded)The thing is that I need to restart a tablet server if hte CM host is down, and obviously I can't do it using the CM.is there any way to restart these Kudu tablet servers without CM?NOTE: I've already tried sudo service kudu-tserver stopbutFailed to stop kudu-tserver.service: Unit kudu-tserver.service not loaded.

Im trying to run Apache NiFi on my laptop. I can start the run-nifi.bat But when I try to access http://localhost:8443/.  "This page isn’t working localhost didn’t send any data. ERR_EMPTY_RESPONSE"On the same laptop, I'm accessing different NiFi instances running on AWS. With different ports (Production).System: Windows10Please help to connect

Hi Team,we are trying to integrate kafka and hbase into atlas 2.2.0we are not using embedded kafka and hbase , seperately installed in same machine.kafka version : 2.13-2.5.0atlas version : 2.2.0/home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh --topic Maestrotopic3>>>>> /home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh>>>>> /home/ranger/apache-atlas-sources-2.2.0Using Kafka configuration directory [/data/kafka_2.13-2.5.0/config]Log file for import is /var/log/atlas/import-kafka.logPicked up _JAVA_OPTIONS: -Xms1024m -Xmx1024m -XX:MaxPermSize=512mOpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/data/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/data/kafka_2.13-2.5.0/libs/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/St

We are using Nifi TailFile to tail multiple files in a directory and state location is set to "remote". It looks like Nifi is keeping the file state even after the file is deleted. is there a way to remove deleted file from the state, manually or automatically?

Hi all, I am new to Hive, was told below parameter used to improve hive performance, if i were to set and run those code sequence as below, does the sequence matters and correlated to each other? Do we need to put up these code whenever run the query? Or execute once will be sufficient? Thanks.  Set hive.exec.parallel = true;set hive.vectorized.execution.enabled = true;set hive.vectorized.execution.reduce.enabled = true;set hive.cbo.enable=true;set hive.compute.query.using.stats=true;set hive.stats.fetch.column.stats=true;set hive.stats.fetch.partition.stats=true;set mapred.compress.map.output = true;set mapred.output.compress= true;SET hive.exec.dynamic.partition = true;SET hive.exec.dynamic.partition.mode = nonstrict;set hive.auto.convert.join = false;set hive.exec.max.dynamic.partitions=100000;set hive.exec.max.dynamic.partitions.pernode=10000;   

my cluster is CDH 5.16.2 Please help ##log## [xxx@hdpwxxx1.true.care]:/reserve1>hdfs dfs -put 30.img /tmp/41.img22/08/12 17:49:55 INFO hdfs.DFSClient: Exception while adding a blockorg.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.NotReplicatedYetException): Not replicated yet: /tmp/41.img._COPYING_at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:3688)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3477)at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:694)at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.addBlock(AuthorizationProviderProxyClientProtocol.java:219)at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:507)at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.

Hi guys:It's possible to run Oracle stored proc that return a cursor as SP parameter using only NiFi processors?

I'm installing CDH 6.2 on rhel7 and using Oracle19C as metastore and backend db for all components wherever required. All services start normally, however when trying to login to hue, the page throws error. CDH Parcel Version: CDH-6.2.0-1.cdh6.2.0.p0.967373Oracle DB: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - ProductionOracle Client: 19.3.0.0.0 Tried with three cx_Oracle versions: cx_Oracle-5.2.1-py2.7 / cx_Oracle-5.3-py2.7 / cx_Oracle-7.2.0-py2.7 Environment:Request Method: GETRequest URL: http://<URL>:8888/hue/accounts/login?next=/Django Version: 1.11Python Version: 2.7.5Installed Applications:['django.contrib.auth', 'django_openid_auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.staticfiles', 'django.contrib.admin', 'django_extensions', 'babeldjango', 'desktop', 'axes', 'about', 'beeswax', 'filebrowser', 'help', 'impala', 'jobbrowser', 'jobsub', 'metastore', 'oozie', 'pig', 'proxy', 'rdbms'

hello cloudera community, we identified that there is tablet_id with different volume among the cluster's tablet servers there is tablet_id with 700MB on one tablet server and the same tablet_id with 190MB on another tablet server how can we equalize this volumetry between tablet_id on all tablet servers we believe that with this different volumetry between the tablet_id, some hosts are using a lot of memory, practically 90% of memory of what was defined. 1 - we are using cloudera express 5.16.x2 - a rebalance has already been performed on kudu recently 

hello cloudera community,   we have installed livy directly on a cloudera cluster host and we are now trying to connect to livy now using kerberos but we are getting the following error:   <html><head><meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/><title>Error 401 </title></head><body><h2>HTTP ERROR: 401</h2><p>Problem accessing /sessions. Reason:<pre> Authentication required</pre></p><hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.24.v20180605</a><hr/></body></html>   how can we make this connection to retest livy?   we are testing the connection with python3 script as follows:     import json, pprint, requests, textwrapfrom requests_kerberos import HTTPKerberosAuthhost='http://localhost:8998'data = {'kind': 'spark'}headers = {'Requested-By': 'livy','Content-Type': 'application/json'}auth=HTTPKerberosA

Hey everyone,I want to make an API request for every single day in a period. I have defined a start date and an end date in the Generate Flow File Procesorthe api structure is as followshttps://example.com/interfaces?session=${SessionId}&date_start=${since}&date_end=${since}and now I want to loop with route on attribute and update attribute to request all days in this time period.now the question how can I, the attribute of the start date always increase by one day. with the following expression in the update attribute processore I get an empty field as result${since:toNumber():plus(86400000):format('yyyy-MM-dd')}ResultDoes anyone have an idea what adjusted we 

I know Hive only provide equi join. For example, below sql statement.select *from A join B        on A.c1 = B.c2where 1=1; But I want to execute Like join Query in Hive. For example, below sql statement.select *from A join B        on A.c1 like B.c2where 1=1; Please let me know if you know the solution in Hive.

Hi all, I want to access Nifi REST API from a different client machine which is in a different domain. The REST API is up and running but gives me a CORS error when I try accessing the API from a client in a different domain. Please see below -Access to XMLHttpRequest at '...' from origin 'null' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. I understand that I need to change Jetty's web.xml to allow cross origin requests. I can't seem to find web.xml in my nifi install. This is a nifi docker container.Any help I can get in this is much appreciated. Please also let me know if there's a better solution than changing web.xml. I tried CORS browser plugins but haven't been able to get them to work. Thanks in advance.

Which versions and distributions of Hadoop are compatible with Spark 3.3?

Hi team I am new to CDP , I have a problem with the ldap changes in the cloudera manager service.It requests a restart of the service after applying the changes , this restart is applied but the changes are not applied.Can you help me , to identify the problem .

On tableau dev server, jobs are failing with below error, whereas its getting successful from tableau desktop. we use MIT kerberos authentication to connect from Tableau to Cloudera. Same config works in our tableau prod machineError:This job failed on Aug 12, 2022, 8:45 AM after running for 0.0 min because of: java.lang.RuntimeException: [Cloudera][ThriftExtension] (9) Error occurred while authenticating via SASL. Error details: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credential cache is empty) Integrated authentication failed

I'm connecting to Hive using Cloudera's ODBC driver version 2.6.9 and I'm getting a weird behavior that ruins the user experience. Any help to solve or debug this problem would be greatly appreciated. I'm using DBeaver to connect to it. All my databases just display a few of it's tables. Here is an example. When I open DBeaver, The dabase just shows the table `localizacao`:  This database has 3 tables. If I run the command show tables from raw_aneel, DBeaver interface will display:  The error message in english would be:    org.jkiss.dbeaver.model.exec.DBCException: SQL Error [S1090]: [Microsoft][ODBC Driver Manager] Invalid string or buffer length  Here is the complete stack trace displayed in the Error Log. The error message above is in Brazilian Portuguese: org.jkiss.dbeaver.model.exec.DBCException: SQL Error [S1090]: [Microsoft][ODBC Driver Manager] Comprimento inválido de cadeia de caracteres ou de bufferat org.jkiss.dbeaver.m

Hi All,I want to put the error message or want to attach the log file from local location to the nifi flow. My flow is as below:Getfile -> EvaluateJSON -> ExecuteStreamCommand->RouteAttribute-> PutEmail. Please help me for this. Thank you in advance!!

Hi Team,we have done the patching on the development it has failed document is attached and the UAC team suggested to remove symlink /usr/bin/python, on rhel8 it should not be set. can you please suggest what we can do.Below are the errors in logfile :-fatal: [clrv0000239897.ic.ing.net]: FAILED! => {"changed": false, "cmd": "dnf install -y python2-dnf", "msg": "Could not import the dnf python module using /usr/bin/python (2.7.18 (default, Feb 10 2022, 14:26:12) [GCC 8.5.0 20210514 (Red Hat 8.5.0-10)]). Please install python2-dnf package or ensure you have specified the correct ansible_python_interpreter.", "rc": 1, "results": [], "stderr": "Error: Unable to find a match: python2-dnf\\n", "stderr_lines": ["Error: Unable to find a match: python2-dnf"], "stdout": "Last metadata expiration check: 0:00:26 ago on Tue 02 Aug 2022 07:05:15 PM CEST.\\nNo match for argument: python2-dnf\\n", "stdout_lines": ["Last metadata expiration check: 0:00:26 ago on Tue 02 Aug 2022 07:05:15 PM CEST.", "

Hello,I'm new to nifi so my problem may be dumb. Sorry if it seems really basic.I'm trying to use nifi to :1/ Read a JSON response from a REST GET endpoint.2/ Parse the JSON and split some JSON array into JSON objects3/ insert objects in SQL.I'm stuck at Step 2 because my SplitJSON doesnt return anything. No error as well.Here is a sample of my JSON : { "context":"q%3D%2523all%26q.restriction%3Dclass%253Acr_hashtag%26of%3Djson", "nhits":1000000, "nmatches":1000000, "start":0, "ellql":"#query{nbdocs=26966678, text_relevance.expr=\"@term.score*@proximity+@b\", noproximity, term.score=NO_RANKING}(#and(#true() #category(categories,\"Top/datamodelclass/cr_hashtag\") ))", "executor":null, "estimated":false, "autocorrected":false, "spellCheckSuggestions":{ "origStr":null, "suggestions":[ ] }, "hits":[ { "did":1, "url":"ID=10055&", "buildGroup":"bg0", "source":null, "slice":3, "score":0,