Solved

I am trying to run Nifi in a docker container on an EC2 instance.  However, whenever my container starts I am seeing the following error.  2022-06-14 02:23:51,093 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.java.io.IOException: Failed to bind to ec2-xx-xx-xx-xx.us-east-2.compute.amazonaws.com/xx.xx.xx.xx:8082at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:349)at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:310)at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234)at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)at org.eclipse.jetty.server.Server.doStart(Server.java:401)at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)at org.apache.nifi.web.server.JettyServer.start(JettyServer.j

Hello! So I can't access NiFi UI because the server if probably down exactly because of amount of files that have been stuck at queue. So the question is how can I find out the size of the queue for the moment using may be logs or maybe there are some other ways to do so? Thank you beforehand!

Hello, On our use case, we're receiving data in JSON format and it must be converted into Avro using a schema that has X and Y includes mandatory fields and everything else (Z and W) should go into a Custom map. Here is our Avro schema:"fields": [{"name": "X", "type": "long"},{"name": "Y", "type": "string"},{"name": "Custom", "type": { "type": "map", "values": "string" } }] And an example of data coming in:{"X":123,"Y":"ABC","Z":"zzz","W":"www"} We have implemented a JoltTransform to do the needed transformation to get Z and W into a Custom array.Now the issue is on doing the conversion from JSON into Avro since we get an error stating it cannot convert an java.lang.Object into a Map. How can we properly process this JSON and put that array into a Map so it can be properly stored using that Avro schema? Thank you.Best regards,Miguel

HI , I have a requirement where I need to do case-insensitive joins across the system and I don't wish to apply upper/lower functions. I tried setting TBLPROPERTIES('serialization.encoding'='utf8mb4_unicode_ci') at Table level but still the comparison is happening considering case sensitivity.  PSB - drop table test.caseI;create table test.caseI(name string, id int)TBLPROPERTIES('serialization.encoding'='utf8mb4_unicode_ci');insert into test.caseI values ('hj',1); drop table test.caseI_2;create table test.caseI_2(name string, id int)TBLPROPERTIES('serialization.encoding'='utf8mb4_unicode_ci');insert into test.caseI_2 values ('HJ',1); select * from test.caseI iinner join test.caseI_2 i2 on i.name=i2.name;--No ResultTried with encoding 'SQL_Latin1_General_CP1_CI_AI' but got same result as above.Any help would be appreciated, thanks!

Hello everyonei have a content in Json (an array of id)[{"id":7532233},{"id":755163}] and i need to convert this whole array into one attribute so that i can after that refer to it in an ExecuteSql processor with statement :update column 1  where id exists(${attributeName})

I am using the latest version of impala JDBC Driver 2.6.27.1032, when my sql is  "upsert into my.testtable(id,name,insert_time) values(?,?,?)" and using preparestatement, then the column "insert_time" will be replaced to "upsert_time", so my sql be changed to "upsert into my.testtable(id,name,upsert_time) values(?,?,?)"  and i get a error "errorMessage:AnalysisException: Unknown column 'UPSERTby' in column permutation",。How can I solve this problem？please!

Hi I have created a new entity and I would like to show all the attributes by default while doing the 'Create new entity' process. At present, the UI mask shows only the Required attributes. How can I do this?

Hi,I want to insert data to Ms. Sql's server from nifi flow.Where can I find a sample or a guide?Also, where have I must install sqljdbc-4.0.jar?

We recently added a two nodes to our cluster through ambari wizard, we installed datanode, nodemanager, Metrics Monitor, LogFeederThe datanode/nodemanager are starting correctly by not livetopology_mappings.data was updated in both mnode and cnodes cat /etc/hadoop/conf/topology_mappings.data[network_topology]cnode2.2b87d4bc-6cf3-4350-aaf7-eff7227d1aef.datalake.com=/default-rack10.1.2.172=/default-rackcnode5.2b87d4bc-6cf3-4350-aaf7-eff7227d1aef.datalake.com=/default-rack10.1.2.169=/default-rackcnode4.2b87d4bc-6cf3-4350-aaf7-eff7227d1aef.datalake.com=/default-rack10.1.2.175=/default-rackcnode3.2b87d4bc-6cf3-4350-aaf7-eff7227d1aef.datalake.com=/default-rack10.1.2.67=/default-rackcnode1.2b87d4bc-6cf3-4350-aaf7-eff7227d1aef.datalake.com=/default-rack10.1.2.188=/default-rackcnode6.2b87d4bc-6cf3-4350-aaf7-eff7227d1aef.datalake.com=/default-rack10.1.2.9=/default-rack  datanodes have 2 external disks to store hdfs data  [root@node6 ~]# df -hFilesystem Size Used Av

Hi Team,From last few days, data03 node was not working, so we moved it to decomissioned state and restarted all our services. Now i need to recomission and start the services here as well.I selected the host and under actions clicked on End maintenace, but it is failing.Please help me out with what shall i do to resolve this?

Can someone point me how to get it configured?  What's the reason HDP/Cloudera dropped the embedded view from Ambari to begin with?  It was a useful tool to see the details of a hive/tez job, good for troubleshooting.

We are planning a production cluster and right now I already have the Mysql being replicated but only have one Cloudera Manager . However we need to patch etc and sometimes have to take downtime of the CM . Is there a way to add a 2nd cloudera manager to the same database ? 

Hi Team,Can anyone please help me with the below file:           cloudera-manager-agent-6.3.0-1281944.el7.x86_64.rpmi am unable to find it as the below site is giving 404 error:https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/Regards,Jessica Jindal

Hi,after having deleted tera bytes of data from HDFS (1/4 of the total capacity), the block count among data nodes did not decrease as expected. It is still over the critical threshold.How could it be solved?Thank you 

I was able to replicate my problem using the following configuration:FlowThe MergeContent configuration is as followsMergeContent PropertiesMergeContent SchedulingAlthough the conditions of the MergeContent processor are met, it does not concatenate the incoming flowfiles.The last version in which the above flow worked was version 1.15.3.Any suggestions on how to solve this? Is this a bug?P.S.: The flow is so designed due to a dependency on only writing one file per day.

Hi all,i'm searching for a safe procedure to add a new journal node to a Cloudera CDP cluster. Now we have only two master node and so we would like to add the third masternode and deploy the third journal node on it. Is it enough to deploy the role using the cloudera manager add role function? Or is it necessary to take further action? Thank you.

Hello,due to a problem with a script, we have almost saturated the HDFS available space.I suppose this is caused by temporary Hive files that have not been cleaned up due to the abnormal termination of the script.I would like to check the /tmp/hive folder on HDFS but my user, that has administrative privileges, cannot access the /tmp/hive folder.Is there a way to check and clean such folder? Any help would be appreciated. KRs,Andrea

hello, please help with below questions: 1. I have nifi cluster with 2 nodes and running ExecuteGroovyScript processor to create text formatted file generated from flow data content and save it on local filesystem but it is saving file only on one nifi node, is there any way to replicate this file on all nodes at a time? 2. Above file formatting by ExecuteGroovyScript processor as text in Prometheus format and using such flow http_request-> fetchfile-> http_response processors to create custom Prometheus healthcheckmetric web page, it worked for single node but with 2 nodes all data balanced randomly between 2 nodes so I have different content in 2 web pages ( http://node1/healthcheck/ and http://node2/healthcheck/ ) so monitoring not working properly, could you help how to add custom Prometheus metric so it will be available like regular nifi prometheus web page /metrics/ with same content on both nodes? may be other solutions?&n

Ranger is not updating the access audits in ranger after a certa date solr logs are giving the below error:ERROR (qtp246273275-120) [c:ranger_audits s:shard1 r:core_node2 x:ranger_audits_shard1_replica_n1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Exception writing document id 8cb43886-8056-46b2-8b24-0ead48c8280b-1 to the index; possible analysis error: number of documents in the index cannot exceed 2147483519 how can this be troubleshooted?

Hi Community, We are working on a new requirement where I need to disable table-level TTL and rather use cell-level TTL for one of our HBase tables. I can disable the table-level TTL by altering the table property and set cell-level TTL for new incoming data. However, I can't find anything on how can I set cell-level TTL for the data / rows that are already existing in the table. Any suggestions will be very helpful. Thank you,Snehasish

Hello,We are observing Impala CatalogException: Error refreshing metadata for table due to lock contention errors on our Cluster.Came across this article - https://community.cloudera.com/t5/Support-Questions/Impala-Error-updating-the-catalog-due-to-lock-contention/td-p/329310 upon checking the suggestions made by Cloudera Tech - load_catalog_in_background is already unchecked nor do we find any error for JVM Pause in Catalogue LogsJava Heap Size of Catalog Server in Bytes = 28GCDH 6.3.3Would appreciate any help in understanding why this happens and ways to fix this.

So I am trying to implement this script  from datetime import datetime, timedelta, date flowfile = session.get()current_date= datetime.today().date()if flowfile != None: src_table_name = flowfile.getAttribute('src_table_name') date_filter = str(current_date.replace(day =11)) if src_table_name in ['Income_project_BP', 'Plan_final_scenario', 'Plan_primary_scenario', 'ProductionCalendar', 'Subcontracting_BP']: flowfile = session.putAttribute(flowfile, 'date_filter', date_filter) session.transfer(flowfile, REL_SUCCESS) else: session.transfer(flowfile, REL_FAILURE)   And I've tried a lot of different ways but when it comes to "replace" method I'm always getting this error. Same thing when I am trying to use "strftime" method What should I do? I really need this method.Thank you beforehand!

Hello folks! I have a cluster with 9 machines, running on CDH 6.2 (OnPremise). I have 3 master, 1 edge and 5 workers. I am not able to up 2 of 5 NodeManager on workers. 3 of them are ok, and 2 of them give me a follow log (attach), without error but a Warning with "NullPointerException":. When I put the NodeManager to run, on Cloudera Manager it doens't fail, but I got two alerts, as follow:- NodeManager can not connect to ResourceManager- ResourceManager could not connect to Web Server of NodeManager Also, I can't access the /jmx of the server. And, when I run NodeManager by Cloudera Manager, my CPU going to use of 100%. On that 2 workers, I have RegionServer and DataNode working fine, the problem is only with NodeManager. Please, any suggest?     

Good morning everyone,   I have a question regarding the Failure of the ExecuteSQL processor.   I would like to know if when accusing a failure in this processor I can give a Retry in the Processor with its own Failure.   Thanks

Working on a wait-notify setup, but it is running into issues. It is constantly being populated with an "Address already in use" error although there is just the one set of DistributedMapCacheServer and DistributedMapCacheClient controller services. This oddly doesn't immediately cause an issue when these are first created and the wait-notify actually works fine, but on any subsequent runs with the same input file (same Release Signal Identifier) the files do not wait at all and move through immediately.