Solved

Hi All:I'm new to Apache NiFi. I am able to download and have the apache nifi working perfectly but when it comes to apache NiFi registry, I am getting an error preventing me to start the registry. I ran the batch file after adjusting the port for NiFi registry.run-nifi-registry.bat it gave me an error. I'm guessing some classpath setup issue... not sure C:\Apache nifi-1.12.1\nifi-registry-0.8.0\bin>run-nifi-registry.batcmd.exe /C "C:\Program Files\AdoptOpenJDK\jdk-8.0.242.08-hotspot\bin\java.exe" -cp C:\Apache nifi-1.12.1\nifi-registry-0.8.0\bin\..\conf;C:\Apache nifi-1.12.1\nifi-registry-0.8.0\bin\..\lib\*;C:\Apache nifi-1.12.1\nifi-registry-0.8.0\bin\..\lib\shared\*;C:\Apache nifi-1.12.1\nifi-registry-0.8.0\bin\..\lib\bootstrap\* -Xms512m -Xmx1024m -Dorg.apache.nifi.registry.bootstrap.config.file=C:\Apache nifi-1.12.1\nifi-registry-0.8.0\bin\..\conf\bootstrap.conf org.apache.nifi.registry.NiFiRegistry runError: Could not find or load main class nifi-1.12.1\nifi-reg

 [mirrormaker-thread-0] Mirror maker thread failure due to java.lang.RuntimeException: { is an invalid regex. at kafka.utils.TopicFilter.<init>(TopicFilter.scala:38) at kafka.utils.Whitelist.<init>(TopicFilter.scala:46) at kafka.tools.MirrorMaker$ConsumerWrapper$$anonfun$init$2.apply(MirrorMaker.scala:316) at kafka.tools.MirrorMaker$ConsumerWrapper$$anonfun$init$2.apply(MirrorMaker.scala:314) at scala.Option.foreach(Option.scala:257) at kafka.tools.MirrorMaker$ConsumerWrapper.init(MirrorMaker.scala:314) at kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:215) please help! 

Hello,I have a file in .xlsx format that needs to be transofmerd to .csv format with broken pipe as separator. When I enter broken pipe (¦) as Value Separator in ConvertExcelToCSVProcessor, in CSV file appears "?" as separator instead of "¦". Is there a way to convert .xlsx to .csv with broken pipe delimiter?

Hello, After we upgraded from CDH 5.15 to CDP 7.1.6 runtime. The HUE SAML login got broken. It gives an error below. Any ideas?  ValueError at /saml2/login/Could not deserialize key data.Request Method: GETRequest URL: http://xxxxx.com:8889/saml2/login/?next=/Django Version: 1.11.29Exception Type: ValueErrorException Value:Could not deserialize key data.Exception Location: /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/cryptography-2.9-py2.7-linux-x86_64.egg/cryptography/hazmat/backends/openssl/backend.py in _handle_key_loading_error, line 1382Python Executable: /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/bin/python2.7Python Version: 2.7.5Python Path:['/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/desktop/libs/libsaml/attribute-maps',

I am trying to get all documents within a time range using RunMongoAggreation.In console I tried with below query and it worked fine.aggregate([{"$match": {"modified_at": {"$gte": ISODate("2017-02-08T12:10:40.787")}}}])Flowfile has below propertiesstartTimestamp: 2017-02-08T12:10:40.787Below is the query being used in RunMongoAggreation[{"$match": {"modified_at": {"$gte": "${startTimestamp}"}}}]Its not return result

I have some impala queries in a file and I want each time I am running the queries to set the mem_limit = 3gb but in the command not in the file inside impala-shell -f /path/ (and add here the mem_limit) Is this possible?

Hi, I have been struggling to get my oozie jobs that run a single spark action to run through without an error. I am using client mode. When I run the spark job via the command line with spark-submit there is no issue and from what I understand this is because I am able to set the driver memory settings but via Oozie the launcher hosts the driver and it so this new container now needs to match the memory settings of my the driver or it will fail. There are many blogs and articles I have read but none of these have helped:https://stackoverflow.com/questions/24262896/oozie-shell-action-memory-limithttps://community.cloudera.com/t5/Support-Questions/Oozie-Spark-Action-getting-out-java-lang-OutOfMemoryError/td-p/39732http://www.openkb.info/2016/07/memory-allocation-for-oozie-launcher-job.htmlhttps://stackoverflow.com/questions/42785649/oozie-workflow-with-spark-application-reports-out-of-memoryWhere the issue comes in partly is I don't know where in the XML these settings sh

Hello Everyone, I would like to understand how I can define the retention period for Ranger Audits in CDP Infra Solr. Ranger Audits are filling up the disk space on my nodes and I would like to configure a retention period for the same. Is there any configuration setting in CDP-INFRA-SOLR or Ranger Service in Cloudera Manager that can be configured? Thanks,Megh

Dear experts, I built an application that runs on HDF-3.4.1.1, to be accurate, a jar file runs in Storm supervisor(s), so I need to get the Ambari 2.7.3.0  with the supported HDF stack and packs binaries, apparently, all the binaries that were for Hortonworks changed with the new links that can't be accessed publicly this is my problem. may you kindly help me in this to provide an alternative so I can get the binaries?   Appreciated in advance @HDF  @GangWar @MattWho  @smdas @Shelton    

I have 5 Data nodes, where 1 Data node crashed. Name node is still replicating the data blocks of the crashed Data node. What happens when I recover the crashed Data node. The data blocks which is under replicated will be reported but what happens to already replicated blocks. Will they be marked as corrupted blocks or will be deleted/removed since there will be 4 replicas?

I'm sending data from a MiNiFi agent to a NiFi Cluster. I set port 1026 for the nifi.remote.input.socket.port in the site to site properties section of the nifi.properties file in the NiFi cluster to get these data. I want to add another MiNiFi agent for sending some different data to the previously mentioned NiFi Cluster. So I want to add another port in the nifi.remote.input.socket.port for distinct receiving values in the NiFi cluster from MiNiFi agents. How can I add multiple values for nifi.remote.input.socket.port and if it is not possible, how can I handle receiving different values in the same NiFi cluster from MiNiFi agents.Nifi version: 1.8.0MiNiFi version: 0.5.0 java

Hi team, currently I have chosen the HDP Version 3.1.5 to gain insights from structured and unstructured data, but I feel confused because of Cloudera announcement about moving to a subscription model, Does All of technology about HDP remain, free and open source?  if I have a cluster running HDP 3.1.5 without a subscription but have the local repo, does it allow me to use it for free? 

Hello, I am trying Apache Nifi for testing purpose. I create a flow that need to unzip a file that contains an .xml file. I chain those processors: ListFIle => IdentifyMimeType => UnpackContent. All files I have test are failled. And I have a java error on my logs: 2021-04-09 17:56:40,839 ERROR [Timer-Driven Process Thread-7] o.a.n.processors.standard.UnpackContent UnpackContent[id=b746ede2-0178-1000-dd20-22c2c827cbb7] Unable to unpack StandardFlowFileRecord[uuid=cfe7807c-d6ad-4127-b779-75b2f57c0ba6,claim=,offset=0,name=data.zip,size=0] due to org.apache.nifi.processor.exception.ProcessException: IOException thrown from UnpackContent[id=b746ede2-0178-1000-dd20-22c2c827cbb7]: net.lingala.zip4j.exception.ZipException: Could not fill buffer; routing to failure: org.apache.nifi.processor.exception.ProcessException: IOException thrown from UnpackContent[id=b746ede2-0178-1000-dd20-22c2c827cbb7]: net.lingala.zip4j.exception.ZipException: Could not fill bufferorg.apache.ni

Hello Team, We have scheduled the command below kinit -R -kt /opt/striim/streamset.keytab streamset/RB-HADOOP-03@INNOV.LOCAL Does this refresh the ticket? We have an ETL tool that will replicate data to Kudu. The user on the ETL tool will receive authentication keys and save this in "keytab"? then Ranger will allocate this user a ticket? How do we analyze this ticket? What is its purpose? Thanks, Roshan

Dear all, After migrating an historical flow from 1.11.4 to 1.13.2, without any changes, after a while, the limits are raised but the content repository is never cleaned. I restart nifi, and the content-repository is cleaned. Thx for your helpJean-Louis OS : Ubuntu : 20.04.2Java : OpenJDK Runtime Environment (build 1.8.0_292-8u292-b10-0ubuntu1~20.04-b10)Nifi properties: # Content Repositorynifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepositorynifi.content.claim.max.appendable.size=10 MBnifi.content.claim.max.flow.files=100nifi.content.repository.directory.default=/data3/nifi_repositories/content_repositorynifi.content.repository.archive.max.retention.period=12 hoursnifi.content.repository.archive.max.usage.percentage=50%nifi.content.repository.archive.enabled=truenifi.content.repository.always.sync=falsenifi.content.viewer.url=/nifi-content-viewer/nifi.content.repository.encryption.key.provider.implementation=. 

Hi,I've been using a flow to get events from Azure Event Hub. I have decided to move from GetAzureEventHub to ConsumeAzureEventHub. But ConsumeAzureEventHub returns following: "failed to process session due to EventHub names must conform to the following rules to be able to use it with EventProcessorHost: Must start with a letter or number, and can contain only letters, numbers, and the dash (-) character. Every dash (-) character must be immediately preceded and followed by a letter or number; consecutive dashes are not permitted in container names. All letters in a container name must be lowercase. Must be from 3 to 63 characters long." The EventHub name contains underscore which is OK for EvenHub itself and for GetAzureEventHub as well. I'm unable to change it because I'm just a "reader" ... Is there any possibility how to use ConsumeAzureEventHub for such name?

We want to hide templates and workflows created by one user to the other users in the system. So the workflow should be visible to only its owner.

How to do error handling for ListSFTP processor (eg. if it fails to connect to SFTP server due to any reason)?  The only connection it shows is "Success".

Hello,I am getting the below error when trying to check logs via CM for Hue Kerberos Ticket Renewer.  CM / CDH - 6.3.3I checked / /var/log/hue but did not find kt_renewer.log. Appreciate any assistance to fix this. ThanksAmn

On a kerberized HDP cluster when I try to run dfsio it fails with below error##########################################org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:563)at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:422)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)Caused by: org.apache.hadoop.security.AccessControlException: User hdfs does not have permission to submit application_1622150236559_0013 to queue defaultat org.apache.hadoop.yarn.server.resourcemana

Hello Can we install the below Cloudera applications on the IBM Power System with RHEL 7.x or 8.x OS?  1) Spark Streaming 2) Sqoop 3) Nifi 4) Impala Best regardsKannan

I am using cloudera quickstart 5.13.This is a phenomenon that the start and restart buttons of CM disappear as soon as you press add service to use kudu table and download kudu parcel(s).If you click the service arrow such as HDFS or Impala and click restart, the following error occurs.How can I solve this?Help Me!

Hi All, We are getting the below error intermittently for some tables and we have hive 3.1 . I am passing a sample query like insert into table test as select * from some table in Nifi PutHiveSQL processor .  Hive target table is managed table and stored as avro format Error :PutHive3QL[id=d1652e76-54f7-30cf-8dc8-b1934cee3c26] Failed to update Hive for StandardFlowFileRecord[uuid=7447e7f0-4d1e-40fa-9965-5c98f7d11341,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1618924236521-4, container=default, section=4], offset=154416, length=3631],offset=0,name=20210421_test_branch_info,size=3631] due to java.sql.SQLException: Error while compiling statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.StatsTask. MetaException(message:Cannot change stats state for a transactional table ra_test.test_branch_info without providing the transactional write state for verification (new write ID -1, valid write IDs null; current state

CDP 7.1.6 with Isilon OneFS v8.2.2.0,  AD kerberos enabled. While installing RangerKMS service, it failed to start up with the following errors:   [root@n04 ~]# less /var/log/ranger/kms/ranger-kms-n04.gz.local-kms.log2021-06-11 08:30:32,179 INFO org.apache.hadoop.crypto.key.kms.server.KMSWebApp: -------------------------------------------------------------2021-06-11 08:30:32,181 INFO org.apache.hadoop.crypto.key.kms.server.KMSWebApp: Java runtime version : 1.8.0_232-b092021-06-11 08:30:32,185 INFO org.apache.hadoop.crypto.key.kms.server.KMSWebApp: KMS Hadoop Version: 3.1.1.7.1.6.0-2972021-06-11 08:30:32,185 INFO org.apache.hadoop.crypto.key.kms.server.KMSWebApp: -------------------------------------------------------------2021-06-11 08:30:32,208 INFO org.apache.ranger.plugin.classloader.RangerPluginClassLoaderUtil: getFilesInDirectory('/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/ranger-kms/ews/webapp/WEB-INF/classes/lib/ranger-kms-plugin-impl'):

Hello Team, Not able to change my company's name in my profile. By mistake during account creation, selected incorrect company's name. Request you to kindly fix it. Thanks,Alex