Solved

I have a CDH instance which is comes bundled from Oracle Big Data Appliance. Its on CDH CM 6.3.4. I know its already EOS. License expired is soon at 30 March 2024.Can CDH license renewal still applicable or does subscription to CDP license would work on CDH instance?

Hi, I am trying to set up a workflow in Nifi, where I need to process Flowfiles in a specific order. To my understand, this is what the EnforceOrder processor was designed for.The part I am having problems with looks like this:In Set Merge Priority I am setting a Merge Priority attribute (1 for the left node, 2 for the right node). I am also setting an Enforce property to group the EnforceOrder by:My EnforceOrder processor is configured as followed:Only flowfiles with merge_priority = 1 will be successfully routed, while the flowfiles with merge_priority = 2 are stuck in "Wait". Files with merge_priority = 2 to arrive at the EnforceOrder processor before the flowfiles with merge_priority = 1. To my understanding, the flowfiles should be re-processed as soon as a file with the current merge_priority has been processed?

Hello I'm running apache zookeeper 3.8.4.File zookeeper.log does not get created.Log files are empty except the following message:SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".SLF4J: Defaulting to no-operation (NOP) logger implementationSLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.SLF4J: Failed to load class "org.slf4j.impl.StaticMDCBinder".SLF4J: Defaulting to no-operation MDCAdapter implementation.SLF4J: See http://www.slf4j.org/codes.html#no_static_mdc_binder for further details.How can I configure zookeeper with logs working ?According to the links, this is due to slf4j not detecting logback.The process runs with the following options in the commandline (extract)The classpath does seem to include slf4j and logback jars, as well as the conf directory containing logback.xml/apps/dis/java/jdk/bin/java-Dzookeeper.log.dir=/apps/zookeeper/log-Dzookeeper.log.file=zookeeper.log-cp.../apps/zookeeper/bin/zkcurrent/bin/../lib/sl

The repositoryhttps://repo.hortonworks.com/content/repositories/releases/is not working for 17 days now.  What is happening?Related to https://community.cloudera.com/t5/Support-Questions/Hortonworks-maven-repo-broken/m-p/398655

Hi All, We are in process of migrating HDP cluster to CDP. As we would like to retain Atlas lineage in CDP from HDP, we have exported them from HDP cluster which gave us a .zip file. Extracted it and remediated the contents of .json files to match the CDP cluster name and server names, then zipped it back to a new_name.zip file. Now when running Atlas import API command to import remediated Atlas data to CDP Atlas, we are getting below error for most of the Atlas imports. {"errorCode":"ATLAS-404-00-007","errorMessage":"Invalid instance creation/updation parameters passed : type_name.entity_name: mandatory attribute value missing in type type_name"} Also, have some more errors, which certainly need to be looked into but not super priority.{"errorCode":"ATLAS-400-00-08A","errorMessage":"Attempting to import empty ZIP file."}:{"errorCode":"ATLAS-500-00-001","errorMessage":"java.lang.NullPointerException","errorCause":null} Having tried to figure this out since a w

Hello,I am trying to delete a custom Type, but I receive the error "ATLAS-409-00-002: Given type InformationAssets has references".  This type has no entities and no relationships associated with it. I created a second custom Type (which I can delete) and compared the API results from: /api/atlas/v2/types/typedef/name/But both look the same.  I'm not sure what the dependency is?  Is there anyway to find out what references are out there for this custom Type?

Hello,I have a custom Type that has an attribute that references another Type by mistake.  I have tried to delete this attribute using a PUT api call to: /api/atlas/v2/types/typedefsBut got the error:ATLAS-400-00-00D: attribute delete not supported.I even tried to to update the attribute to not longer reference the Type but instead have "typeName": "string", but I get the error:ATLAS-400-00-029: Data type update for attribute is not supportedIs there a way to delete an attribute from a custom Type?

Hi,When I try to download custom parcel on Cloudera Manager, it cannot do it anymore.Older parcels can still be downloaded.We have a workaround on which we upload new parcels to bypass the issue.Following the error returned by LOG when we try to download custom parcel on CM.     Unable to dowload size for parcel : name.parcel     java.lang.NumberFormatException: null          at java.lang.Long.parseLong(Long.java:552)          at java.lang.Long.parseLong(Long.java:631)Do you have any idea to solve it ?

org.apache.hadoop.hdfs.server.datanode.DataNode: Unsuccessfully sent block report 0x687e2eff1ccff4e2 with lease ID 0x2e42f5718005b5ef to namenode: hadoop-standby/:8020, containing 1 storage report(s), of which we sent 0. The reports had 659503 total blocks and used 0 RPC(s). This took 83 msecs to generate and 56 msecs for RPC and NN processing. Got back no commands.2025-01-02 19:28:31,810 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: RemoteException in offerServiceorg.apache.hadoop.ipc.RemoteException(java.io.IOException): java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.runBlockOp(BlockManager.java:5558)at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.blockReport(NameNodeRpcServer.java:1651)at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:182)at org.apache.hadoop.hdfs.protocol.proto.DatanodeP

I have a use case scenario coming up next week where I will need to process some Parquet to CSV.   I created a few demos with nifi 1.10 but unfortunately it is not possible to use this version of NiFi on the customers environment.   I know I can still satisfy the actions I need to take with 1.9 without the new 1.10 features for Parquet, but I know these new features would be more efficient.   Is it possible to drop the required artifacts into 1.9 and achieve Parquet Reader functionality?

Hello. I am using nifi cluster with 3 nodes. I tried to Impala Query with query, it was success.   But when I tried to use ListSFTP and FetchSFTP, It occurs Error. The Error was   ERROR [Timer-Driven Process Thread-6] o.a.nifi.processors.standard.ListSFTP ListSFTP[id=c486c6ce-016e-1000-0000-000037351532] Failed to properly initialize Processor. If still scheduled to run, NiFi will attempt to initialize and run the Processor again after the 'Administrative Yield Duration' has elapsed. Failure is due to java.lang.IllegalArgumentException: A HostProvider may not be empty!: java.lang.IllegalArgumentException: A HostProvider may not be empty!java.lang.IllegalArgumentException: A HostProvider may not be empty!at org.apache.zookeeper.client.StaticHostProvider.init(StaticHostProvider.java:136)at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:87)at org.apache.zookeeper.ZooKeeper.createDefaultHostProvider(ZooKeeper.java:1312)at org.apache.zooke

I'm currently facing an issue with implementing OpenID Connect (OIDC) authentication in Apache NiFi. And I am putting my NIFI in the kubernets. so, I am with the NIFI containers.Error Overview:I receive the following error in the NiFi logs:2025-01-02 10:29:05,167 INFO [NiFi Web Server-52] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 172.24.0.3 [<oncloudtemuser@sossourabh7687gmail.onmicrosoft.com><CN=172.24.0.3, OU=NIFI, O=NIFI, L=HYDRABAD, ST=TELANGANA, C=IN>] GET https://172.24.0.3:8443/nifi-api/flow/current-user2025-01-02 10:29:05,173 WARN [NiFi Web Server-52] o.a.n.w.s.NiFiAuthenticationFilter Authentication Failed 172.24.0.3 GET https://172.24.0.3:8443/nifi-api/flow/current-user [Untrusted proxy CN=172.24.0.3, OU=NIFI, O=NIFI, L=HYDRABAD, ST=TELANGANA, C=IN]2025-01-02 10:29:05,207 ERROR [NiFi Web Server-190] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.io.UncheckedIOException: Read Current User Entity failed.

Hello,In NiFi Toolkit 2.0, the nifi create-reg-client command does not provide a parameter to specify the registry client type (GitLabFlowRegistryClient 2.0.0 ). Are there any workarounds for setting up a registry client in the apache/nifi:2.0 Docker image for a CI pipeline?Best regards Wojtek  

Hello, I'm planning to implement new cluster + Data services, I'm planning to use below versionsCM 7.11.3 CHF 11 Cloudera Runtime 7.1.9 SP1Cloudera Data services 1.5.4 SP1I have checked the compatibility matrix all the releases are compatible with each other, but when I read this article I can't find 7.1.9 SP1https://docs.cloudera.com/cdp-private-cloud-data-services/1.5.4/release-notes/topics/cdppvc-service-packs-1-5-4-SP1.html#ariaid-title2 Any one tried this path? 

When I install hue with cloudera manager it stops, when searching the log it shows me a permissions error. ======log cloudera manager======2021-04-19 10:30:55,858 INFO CommandPusher-1:com.cloudera.server.cmf.CommandPusherThread: Acquired lease lock on DbCommand:15463385272021-04-19 10:30:55,860 INFO CommandPusher-1:com.cloudera.cmf.service.AbstractOneOffHostCommand: Unsuccessful 'HueTestDatabaseConnection'2021-04-19 10:30:55,861 INFO CommandPusher-1:com.cloudera.cmf.service.AbstractDbConnectionTestCommand: Command exited with code: 12021-04-19 10:30:55,861 INFO CommandPusher-1:com.cloudera.cmf.service.AbstractDbConnectionTestCommand: + '[' syncdb = is_db_alive ']'+ '[' ldaptest = is_db_alive ']'+ exec /opt/cloudera/parcels/CDH-7.1.5-1.cdh7.1.5.p0.7431829/lib/hue/build/env/bin/hue is_db_aliveTraceback (most recent call last):File "/opt/cloudera/parcels/CDH-7.1.5-1.cdh7.1.5.p0.7431829/lib/hue/build/env/bin/hue", line 9, in <module>from pkg_resources import load_entry_pointFile

Mates,Need help or references of how to configure REST API on NiFi 2.0, unable to generate token as we were doing on 1.x.  Have NiFi on plan Linux not on any cluster

Hi,I can't seem to figure out why I'm unable to reach my NiFi GUI from computers on our intranet other than where it's installed.nifi.properties has these entries (nothing on the http settings)nifi.web.https.host=ourMacMini20nifi.web.https.port=9444nifi.web.https.network.interface.default=nifi.web.https.application.protocols=h2 http/1.1nifi.web.jetty.working.directory=./work/jettynifi.web.jetty.threads=200nifi.web.max.header.size=16 KBnifi.web.proxy.context.path=nifi.web.proxy.host=nifi.web.max.content.size=nifi.web.max.requests.per.second=30000nifi.web.max.access.token.requests.per.second=25nifi.web.request.timeout=60 secsnifi.web.request.ip.whitelist=nifi.web.should.send.server.version=truenifi.web.request.log.format=%{client}a - %u %t "%r" %s %O "%{Referer}i" "%{User-Agent}i"From the computer itself where the installation occurs this works:https://ourmacmini20:9444/nifi/For other computers (where certificates are installed) this message is shownThis site can’t be reachedourmacmini20

I am using CSVReader and CSVRecordSetWriter using the Infer schema setting. But when I have values such as "030", even though the all the values has double quotes enclosed. When I write the schema into avro.schema, I see it is considered as type 'int'. But I want to treat it as string. Because of this our output after the processor looks like 30 and the first 0 is omitted. I want to use Infer schema property only because, I want to read the csv files dynamically without hardcoding the schema and use that.

Hi team,I'm struggling while converting nested JSON to Flat JSON using JOLT.The Nested JSON is in the below format: { "data": { "getCarListing": { "edges": [ { "node": { "id": "1001", "identifier": 500001, "carClusterIds": [ { "element": { "name": "SEDAN1000", "businessRelations": [ { "countries": [ { "countryCode": "US" } ], "code": "US01" }, { "countries": [ { "countryCode": "CZ" }, { "countryCode": "SK" } ], "code": "CZ01" }, {

I have recently upgraded from CM 7.6.7 to CM 7.11.3 and CDP 7.1.7 SP2 to CDP 7.1.7 SP3.HDFS Datanode, Impala Daemon, Yarn Resource Manager, and Hbase Region Server are showing unhealthy web server on Cloudera as shown below. After checking one of the agents log, I found the following error.  18/Nov/2024 09:09:09 +0100] 2414 GM IMPALAD throttling_logger ERROR Error fetching metrics at 'https://host.domain.com:25000/jsonmetrics?json'Traceback (most recent call last): File "/data/cloudera/cm-agent/lib/python3.8/site-packages/cmf/monitor/generic/metric_collectors.py", line 224, in _collect_and_parse_and_return opened_url = urlopen_with_retry_on_authentication_errors( File "/data/cloudera/cm-agent/lib/python3.8/site-packages/cmf/util/url.py", line 339, in urlopen_with_retry_on_authentication_errors return function() File "/data/cloudera/cm-agent/lib/python3.8/site-packages/cmf/monitor/generic/metric_collectors.py", line 244, in _open_url return self._urlopen_ca

Hi Community,I have recently upgraded from CM 7.6.7 to CM 7.11.3 and CDP 7.1.7 SP2 to CDP 7.1.7 SP3.There are many services that are showing web server error on Cloudera as shown below. One of those services is HDFS. When I checked the cloudera-scm-agent log, I found the following error.  [21/Oct/2024 09:09:09 +0100] 2414 GM IMPALAD throttling_logger ERROR Error fetching metrics at 'https://host.domain.com:25000/jsonmetrics?json'Traceback (most recent call last): File "/data/cloudera/cm-agent/lib/python3.8/site-packages/cmf/monitor/generic/metric_collectors.py", line 224, in _collect_and_parse_and_return opened_url = urlopen_with_retry_on_authentication_errors( File "/data/cloudera/cm-agent/lib/python3.8/site-packages/cmf/util/url.py", line 339, in urlopen_with_retry_on_authentication_errors return function() File "/data/cloudera/cm-agent/lib/python3.8/site-packages/cmf/monitor/generic/metric_collectors.py", line 244, in _open_url return self._urlopen_call

Hi Everyone, I have the below Input JSON and would like to flatten the data into the below mentioned output JSON format. I am new to the JOLT, I tried to transform (its incomplete) but could not achieve the result. Kindly help me to get the desired output, thanks in advance. Input JSON:           { "data": { "getItemListing": { "edges": [ { "node": { "id": "2940126", "identifier": 3119650, "parentItems": [ { "id": "364270", "parentIdentifier": 371153 }, { "id": "364258", "parentIdentifier": 371147 } ], "treatmentClusterIds": [ { "metadata": [ { "treatmentClusterIDs": "528" } ], "element": { "name": "GOAT02", "labelingClusters": [

In Apache Spark, spark_shuffle and spark2_shuffle are configuration options related to Spark's shuffle operations, which can be set to start auxiliary services within the Yarn NodeManager. But what is the difference between these two?

I need to authenticate to a Kafka Broker using OAuth. I am looking at the Apache NiFi issues pages and see this URL to add support for SASL/OAUTHBEARER to the Kafka 3 processors which now have an allowable values list of SASL mechanisms - https://issues.apache.org/jira/browse/NIFI-7421Does anyone know if this will ever be implemented?Otherwise, does anyone have any list of ideas I could use to achieve this requirement?@MattWho @SAMSAL 

Hello Experts,I am using Wait - notify processors for my flow, once in a while I am seeing this error in Notify processor.java.lang.RuntimeException: Unable to communicate with cache when processing.The screen shot attached. Any idea what is the reason and how to resolve this ?Thanks in advance.Mahendra