Q&A

hi We have recently upgrade from CDH to CDP 7.1.7. Now we want to disable Kerberos on CDP cluster. Is it feasible to do so. I know it is not recommended by Cloudera to disable Kerberos. But still we want to do that to implement one use case. Could someone tell me how to disable kerberos for whole cluster in CDP 7.

I am trying to learn how to use the RestLookupService to make REST API calls. But since it is a controller service, not a processor.. so I need some guidance on which processor I should use.. where I can configure this service. I want to save the response that this service returns to somewhere else..let's say to the local file system...using PutFile.

eating bean with name 'auditDataSource': FactoryBean threw exception on object creation; nested exception is org.h2.jdbc.JdbcSQLNonTransientConnectionException: File corrupted while reading record: null. Possible solution: use the recovery tool [90030-199]        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:342)        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:113)        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1707)        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1452)        at org.springframework.beans.factory.support.AbstractAutowir

Hello,I'm facing a problem with HDFS is in bad state because of Canary test failed.  ERROR com.cloudera.cmon.firehose.polling.hdfs.HdfsCanary: (9 skipped) com.cloudera.cmon.firehose.polling.hdfs.HdfsCanary@70164e31 for hdfs://nameservice1: Failed to write to /tmp/.cloudera_health_monitoring_canary_files/.canary_file_2022_06_29-15_20_26.3f6b5657894eb2c0. Error: {}java.io.IOException: Could not get block locations. Source file "/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2022_06_29-15_20_26.3f6b5657894eb2c0" - Aborting...block==null at org.apache.hadoop.hdfs.DataStreamer.setupPipelineForAppendOrRecovery(DataStreamer.java:1491) at org.apache.hadoop.hdfs.DataStreamer.processDatanodeOrExternalError(DataStreamer.java:1271) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:667)WARN org.apache.hadoop.hdfs.DataStreamer: Could not get block locations. Source file "/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2022_06_29-15_24_31.b

Hi I am having a challenge with the access control for credentials while trying to execute the spark process in Zeppelin, can anyone help? ERROR idbroker.AbstractIDBClient: Cloud Access Broker response: { "error": "There is no mapped role for the group(s) associated with the authenticated user.", RegardsLakshmi Segu 

Hello,I want to graph number of HBase regions per regionserver. In Ambari there was a metric regionserver.Server.regionCount, that provided just that. In CM the closest query I could find is "select regions_across_regionservers" that shows me an average number of regions per server. Average is useful, but I want to track per server data.I know how to get it from command line, or from master web gui.  I am looking specifically for datasource query so I can graph it with grafana.Thank you!

Is there any way to e.g. keep the ttl for the service HDFS at 90 DAYS, and set the ttl for the other services to 14 DAYS? ---Related articles:Support Question "How to define the retention period for Ranger Audits in CDP INFRA Solr?"https://community.cloudera.com/t5/Support-Questions/How-to-define-the-retention-period-for-Ranger-Audits-in-CDP/td-p/318759 solrctl Referencehttps://docs.cloudera.com/runtime/7.1.0/search-managing/topics/search-solrctl-ref.html

HelloI'm using Postgresql with NIFI, how can I specify the schema I want to use from Postgresql  There is no option for the schema in the DBCPConnectionPoll.  and there is no option for schema also in the lookup service   

Hello everyone, i need your help. I am relatively new to nifi and need your help. I am using the processor ConsumeKafkaRecord_2_6 1.15.3 with the record writer json set record writer. The output is then an array without object name. Which processor or setup can I use to add an array name to the json output? I get this Output  [ { "moving_id" : 17078440, "store_id" : 9, "gtin" : 4064813288877, "qty" : -1.0, "condition_indicator" : 40, "issue_date" : 1656585240000, "received_date" : 1656547200000, "update_date" : 1656578057133}, { "moving_id" : 17078441, "store_id" : 9, "gtin" : 4064813288907, "qty" : -1.0, "condition_indicator" : 40, "issue_date" : 1656585240000, "received_date" : 1656547200000, "update_date" : 1656578057133} ] and i need json file in this structur {"stocks":[ { "moving_id" : 17078440, "store_id" : 9, "gtin" : 4064813288877, "qty" : -1.0, "condition_indicator" : 40, "issue_date" : 1656585240000, "received_date" : 165654720

Hi Team,We are using guava-15.0.jar as part of cloudera-manager-api dependency version 6.3.0 in build.gradle file of cloudera-manager client module.But while deploying it is giving the below error -java.lang.NoClassDefFoundError: com/google/common/base/Objects$ToStringHelperWe faced the above error even with guava-30.0-jre.I tried deploying by excluding the above guava jar but that again gave error -Exception in thread \"main\" java.lang.NoClassDefFoundError: com/google/common/cache/CacheLoader   Please find below the build.gradle file dependencies-dependencies {compile group: 'com.cloudera.api', name: 'cloudera-manager-api', version:'6.3.0'compile group: 'org.slf4j', name: 'slf4j-log4j12', version:'1.7.21'compile group: 'com.github.stefanbirkner', name: 'system-rules', version:'1.3.0'compile group: 'org.eclipse.persistence', name: 'eclipselink', version:'2.4.2'compile group: 'org.apache.cxf', name: 'cxf-rt-transports-http', version:'2.7.7'compile group: 'org.json', name: 'json',

Hello Experts, I am trying to upgrade nifi from 1.12.1 version to 1.16.3.I have a script 'ECMAScript' which was working fine in 1.12.1and now behaving different in 1.16.3.Below is the portion of code giving error in 1.16 version when 'xyzId' is null.But it just logs null as expected in 1.12 version var xyzId = flowFile.getAttribute('xyzId');log.info(xyzId); Below is the error in 1.16 version when null value received for above line (log.info() line throws below error)imer-Driven Process Thread-8] o.a.nifi.processors.script.ExecuteScript ExecuteScript[id=bae38131-0181-1000-29af-196312cdd9d9] ExecuteScript[id=bae38131-0181-1000-29af-196312cdd9d9] failed to process due to org.apache.nifi.processor.exception.ProcessException: java.lang.RuntimeException: java.lang.NoSuchMethodException: Can't unambiguously select between fixed arity signatures [(org.apache.nifi.logging.LogMessage), (java.lang.String)] of the method org.apache.nifi.controller.TerminationAwareLogger.info for arg

Hi, I have successfully installed Cloudera Manager on RHEL 7 Azure VM but I'm not able to access the web ui. I have tried after adding 7180 port as inbound port in Network Security Group in azure portal.

Hi Team, I have the below flow to get the ticket details through Rest Api call i want to execute it will atomatically execute after every 60 min but scheduling not working.Please support how can execute below flow after every 60 min.   Note:API's working fine through postman able to execute flow      I have set below property but flow is not executed after 60 min.    

Hi , getting this error while connecting with secured nifi instance.I tried adding certs to Keystore of java.  /bin/cli.sh nifi current-user -p nifi.propertiesERROR: Error executing command 'current-user' : PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Hi, I took the CDP generalist exam few weeks ago,and I can't find a place to download its invoice to hand it to my company for a refund.anyone can help? Regards,

Hi Everyone,  I'm trying to schedule a impala query using Oozie scheduler but it keeps failing or gets stuck at a percentage for hours.  I created a workflow and then scheduled which did not work.   Does anyone any expierence scheduling a impala query using Oozie?  Could you please provide step by step directions or share a link that would walk me through the process?  Thank you in advance.  

The following attached error during Kerberos setup with Ambari 2.7.5  

Hi,I have some schedulers in OOZIE through hue and some of them some times fail. However when I run them manually after, they end successfully. Is there any way to put retry policy in my WFs? Here is the error that I am taking:Exit code of the Shell command 1<<< Invocation of Shell command completed <<<java.lang.reflect.InvocationTargetExceptionat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410)at org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:422)at org.apa

HELLO,  IM TRYing to refresh automatically via ssis package a cube ssas tabular which is connected to hive via ODBC cloudera latest driver. When I´m manually processing my cube It works without any problems , but when I´m using SSIS to automate the process I have this error message (It worked in ssis fine just this 4 days I have  this problem)   Analysis Services Execute DDL Task] Error: The JSON DDL request failed with the following error: Failed to save modifications to the server. Error returned: 'OLE DB or ODBC error: [DataSource.Error] ODBC: ERROR [HY000] [Cloudera][ThriftExtension] (14) Unexpected response from server during a HTTP connection: SSL_read: errno = 9. ERROR [HY000] [Cloudera][ThriftExtension] (14) Unexpected response from server during a HTTP connection: SSL_read: errno = 9.. OLE DB or ODBC error: The command has been canceled.. OLE DB or ODBC error: The command has been canceled.. OLE DB or ODBC error: The command has been canceled

I am learning the ExecuteScript processor. I want to understand, how to use python script to fetch a file from local, similar to what the GetFile processor does.  Thanks

sql("show databases").show() sql("show tables").show()error 

Hello Team, we are doing CDC by pushing data to Kafka and another pipeline will be reading data from Kafka. Whenever we restart the second pipeline (read from Kafka to Kudu), I notice there are thousands of records coming. I would like to know how Kafka keeps the checkpoints? Is there any setting to change it? Thanks,Roshan

Hello! So I am trying to figure out how to work with the EnforceOrder processor.I have 200 flowfiles and every flowfile has attribute partition_number.  Many flowfiles has the same filename. I wanna use EnforceOrder to merge them accroding to filename and partition_number attributebut non of the flowfiles ever go to success relationship queue.Here's my settings:What am I doing wrong?

I want to insert splitted records from splitjson processor in database table using nifi.For eg: Below is splitjson processor output. I want to push all the flow files in database table having same fragment.identifier.table columns are mentioned below:identity_id|json|person_refwhere identity_id should be auto sequence number json would  flow-file contentperson_ref is the foreign key having int data typeso  table will have 13 entries and identity_id column will be unique numbers in sequence, json column will have different flow file contents and person_ref column should be same for all 13 entries lets say 111.   

I have a pyspark dataframe with names likeN. PlainfieldNorth PlainfieldWest Home LandNEWYORKnewyorkSo. PlainfieldS. PlaindieldSome of them contain dots and spaces between initials and some do not. How can they be converted to: n Plainfieldnorth plainfieldwest homelandnewyorknewyorkso plainfields plainfield(with no dots and spaces between initials and 1 space between initials and name)I tried using the following but it only replaces dots and doesn't remove spaces between initials:names_modified = names.withColumn("name_clean", regexp_replace("name", r"\.",""))After removing the whitespaces and dots is there any way get the distinct values.like this.north plainfieldwest homelandnewyorkso plainfield