Solved

Hi All,I have created a profile with organization email but i have selected the wrong company. Now I would like to change the company.Any help would be appreciated. Thanks,Rajan

My previous processor group is for update and insert a table. After that, I want to use ExecuteSQL processor.My question is: How to clear all flowfiles and proceed to the ExecuteSQL processor, so the ExecuteSQL processor only run once.For example, my pic above, there are 3 flowfiles queued, it means that the ExecuteSQL processor will run 3 times? (I think). how to clear them and only run once

As an admin of a CDH cluster , some query has submitted  to the hiveserver2 ,  but the query still  in the hiveserver make the explain stage , which does not submit to the yarn cluster and has no application id on yarn . And the query make the hiveserver2 wrong  which  has a too long query such as select * from aaa where code in ('xxx','xxx1','xxx3',.......'xxx2000000') ;A sql has more than milion row  may make the hiveserver2 corrupt .  The 10002 web page  seems does not have some action button like yarn web the deal with the query . The cdh version is 6.3.2 . hive version 2.1.1 .When this situation occur , I have to restart the hiveserver2 .I want to know if there is some way to kill a query through hive queryid or hive sessionid o instead of yarn applicationid . It is also usefull when someone query the metastore which multithread such as "use hadoop " sql with 30 or more active connect  ,  The adminastritor

Greetings,I'm currently grabbing data from a website, "splitting" it into its separate components and trying to route them so I can save them into a single zipped file.My workflow currently works as follows:InvokeHTTP --> SplitText --> RouteText ...The original Dataset is a bunch of satellite data (example below).  I’m then splitting it into separate 3 line records. I then run RouteText and try and grab specific records.  For example, if I wanted TDRS 3 and AMSC 1 and SKYNET 4C, how do I pull them out with its respected 2 lines?  I tried adding a Property:  SatelliteData1 = ${literal('TDRS 3')}.  This only gives me the first line of the Split record (I need all 3).  I tried OR-Statement, but don't think I'm using it right. ${literal('TDRS 3''):or(${literal('AMSC 1')}):or(${literal('SKYNET 4C')})}Any suggestions will be greatly appreciated.  Thanks in advance.Original Data:Splitting:  

Hi, I have a Hadoop 3.1.1 cluster and recently I found that the NameNode heap memory usage in the cluster is very high. I saw 629973631 file objects in the cluster through WebUi, so according to my calculations, it should occupy no more than 90GB of memory, right? Why is the current memory usage consistently above 140GB? Is this related to me enabling erasure codes? 

Hello everyone,in the execution of a Spark script, which invokes several applications, we have different timings in each of the runs performed, in fact it went from about 8h on 18/11/2023, to about 17h on 09/12/2023 to about 33h on 02/12/2023.Since this is a very heavy process, all daily schedules were disabled to give the application a chance to allocate all resources.The application is run sporadically during weekends, and we do not know exactly when the next run will be.We have noticed that when running Spark applications that are part of the script, a delay of about 3 to 4 hours is encountered between the execution of one job and its next.Do you have any suggestions regarding a possible cause of the anomaly encountered? Thank you

Hello Community,I am getting below error on my HBase table. The table has been fine for close to a year until 2 days ago. What's the cause?org.apache.nifi.hbase.PutHBaseRecord PutHBaseRecord[id=ee1b356c-aa58-13f5-ac63-84a796230216] Failed to put records to HBase.org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1 action: org.apache.hadoop.hbase.exceptions.FailedSanityCheckException: Requested row out of range for batchMutate on HRegion USAGE:DATA_TRANSPARENCY,a11ced068cb301a6c6c502932d686a05_20200621142500_189290_366,1676533563461.bbbebf59c6e4e9750e9f5825850d6f4f., startKey='a11ced068cb301a6c6c502932d686a05_20200621142500_189290_366', getEndKey()='a11f2100d30ecc142b4c0466c01797ad_20230202125000_129718_339', row='a1200e570c1a24fee95b1004dbff50eb_20240108193317_2079_8316'at org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:1041)at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicBatchOp(RSRpcServices.java:959)

Hi, I have email archived in .MSG format files in folder location which is mentioned in SQL database table.I need to get the file from folder location , read the file , check if there is any attachment in msg file , if yes then need to extract that attachment and make the entry of that file into database tableI am new to NIFI , So anyone can help me with some sample flow for this requirement? Thanks In advance , Prats 

As seen from the image above. My requirement is for the "ExecuteSQL" processor to run only once after all the queued files are processed via "PutDatabaseRecord" processor.Right now it will run 190+57 times but I want to run after all the queued files are processed through. Is there any way to do this in Nifi ?

Hello Everyone,We are facing below issue in our CFM.Background: We have NiFi Flow which uses GetHDFS processor to fetch all the HDFS files and performs few transformation and put the data back to HDFS using putHDFS processor. The CRON used for this was: 0 0/45 06-19,21-05 1/1 * ? *Now: Now based on the requirement we have changed the CRON schedule to "0 30 10-19 1/1 * ? *" (0 30 10-19 * * ? *) , the getHDFS processor runs at CRON schedule and also process the data on the day of change. But from next day/tomorrow onwards the processor is running/threads are intiated on all nodes based on CRON schedule however it does not process the data and gives a below warning.Warning: Error while retrieving list of files due to DestHost:desport <master_hostname>:<port>     LocalHost:localport <worker_node>:<port>Failed on Local Exception: java.io.IOException: Couldn't setup connection for <nifi_user>@principal to <master_hostname> Doe

Hello,it's not clear to me how users are managed in CDP Public Cloud / Machine Learning / Cloudera Cloudera Data Visualization.It's ok I can create a new user from the the DataViz menu, but in this case, how are they linked to users that I manage from Management Console / User Management in Control Pane?In DataViz Users & Groups, do I have to create a new user with same workload username and password assigned to the user in the User Management section?Are these users synchronized somehow?I'm asking because once a user has been created in DataViz, I am not able to change its password, so I'm wondering if somehow they are synchronized and can be managed from the User Management section in Management Console.Thank you,Andrea 

Hi all,My nifi crash for some reasons that I could not figure out, can anyone helps?1. A chain of lookups that I have, the table being looked up contains around 50,000,000 rows2. After running some node, it crashed like this:3. After this, I relogged into NiFi to see what error happened:My Lookup service setup:I can provide more details upon request.Please help!!!Thanks  

Hello about log4j2 failure (CVE-2021-44228) in cloudera there is the KB: https://my.cloudera.com/knowledge/Resolution-for-TSB-2021-545---Critical-vulnerability-in-log4j2?id=332012 instructs to run the script to remove a class in the jar etc. our doubt is, do we need to run this script in any version of CM / CDH or just the version from 6.x.x?

My environment is testing on Windows 10. I installed version 1.14 and started. The chrome 91.0.4472.124 says it is not secure and I confirm to go to the site "https://localhost:8443/" not secure. The address changes to "https://localhost:8443/nifi/login" and asks for username and password. What username and password should I use? Where can I generate this username and password?

Hi: From google : If you are using NiFi 1.0 you can use the GenerateTableFetch processor. It allows you to choose the "page" (aka partition) size, and will generate SQL statements, each of which will grab one "page" of data. You can route those into ExecuteSQL and it will retrieve smaller sets of results at a time. If you have a NiFi cluster, you can route GenerateTableFetch into a Remote Process Group that points at an Input Port on the same cluster, (which will distribute the SQL statements across the cluster), then the Input Port can be connected to the ExecuteSQL. This allows you to fetch rows from a table in parallel I have three nodes on my nifi cluster, I follow the post, put GenerateTableFetch on primary node(test01) to execute, and then send to remote process group on the same cluster. then output port to ExecuteSQL. But the actual behavior is ExecuteSQL was execute just in one node (test02, or test 03, or tes01). My question is how to fetch rows from a table in parallel(test0

Hi I got a random message on telegram asking if I was available wen I asked for wot I got told I’d receive a WhatsApp message , I did from a man saying he wanted to give me details on a job anyway wen I didn’t rely he kept midering so I asked for details  he told me I needed to register see first photo wen I clicked on it and looked at the terms it says (see second photo) so I told him it looked like a scam and he told me to just register so I looked into it and joined u to find out wot is going on   

Hello! I am trying to install Nifi 2.0.0 M1 on Ubuntu 20.04. I downloaded Nifi from the official website and removed all HTTPS-related configurations. When I run Nifi with Python disabled, it works fine. However, when I enable Python, the following error occurs, and I'm not sure how to fix it. 😟 Does anyone have any knowledge or experience with this? (Python 3.10 is installed.) 

When upgrading our docker container from 1.19 to 1.22 with the same nifi.properties and flow.xml.gz and repo's we get the following error. Also the same if we tried to migrate to 1.23. Any one any idea's? Google and Chatgpt can't seem to help out 😅 Thank you in advanced. 2024-01-10 09:54:49,115 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFiJan 10 09:54:49 awsinstancename docker[21712]: java.lang.VerifyError: Bad return typeJan 10 09:54:49 awsinstancename docker[21712]: Exception Details:Jan 10 09:54:49 awsinstancename docker[21712]: Location:Jan 10 09:54:49 awsinstancename docker[21712]: com/fasterxml/jackson/databind/cfg/MapperBuilder.streamFactory()Lcom/fasterxml/jackson/core/TokenStreamFactory; @7: areturnJan 10 09:54:49 awsinstancename docker[21712]: Reason:Jan 10 09:54:49 awsinstancename docker[21712]: Type 'com/fasterxml/jackson/core/JsonFactory' (current frame, stack[0]) is not assignable to 'com/fasterxml/jackson/core/TokenStreamFactory' (from met

Hi,Im trying to connect LDAP from Nifi.I've done most of the instructions in the official documentation and read most posts on the Cloudera Community. However, I couldn't connect to my company's LDAP system. I configured login-entity-providers and authorizers for LDAP.When I start Nifi, it starts successfully and gives no error. When I try to log in with my initial admin it gives me "Unable to validate the supplied credentials. Please contact the system administrator." error. My questions are:1- When we are setting up initial admin, we are not setting password of that user. How do we log in without password?2- I'm using USE_DN for identity strategy for ldap-provider. My initial admin name and ldap user are same. So I'm trying to login with my user like: Username: CN=K015576,CN=Users,DC=tcmb,DC=gov,DC=trPassword: "my_ldap_password" What path should I follow? Is there a mistake in my configure files?I have added my conf files below: nifi.properties, login-entity-

Dear Sir,I am using apache nifi for  Getting file from SFTP serverBackup files another SFTP serverDelete the  all files from where getting locationsprocess file to call invokeHttp RequestSFTP images using winscp:After login folder locations:  1. Here is the diagram:2. ListSFTP properties:3. FetchSFTP properties3. Put SFTP propertiesI follow processors  bellow sequences: ListSFTP--> FetchSFTP-->PutSFTP-->UpdateAttribute --> InvokeHttpI did not get file from sftp server. What is the wrong of my configuration?   

I have a API which I need to make a call to get the response using InvokeHttpProcessor. Incase if response status code is not 200's then I need to do a retry for 3 times ( incase of 400's and 500's). I see that I can set a retry on Failure, NO retry and Retry relationships but that is not working as expected. How to configure the relationships such that there I can achieve retry of 3 times on Failure, Retry and NoRetry relationships. Also is there a way to see how many retries are performed and is nifi logging this information some where?

Hi guys! I have a question about Avro schema.My input flowfile are in json format and I want to convert it into xml. The method I use is using ConvertRecord Processor. In terms of ConvertRecord Processor, I use JsonTreeReader as record reader and XMLRecordSetWriter as record writer. Following picture is the properties of JsonTreeReader I set it up. Here is the json I want to convert: { "article":{ "article-meta": { "art_id": "me08001", "title" : [ { "lang" : "en", "title" : "A" }, { "lang" : "hr", "title" : "B" } ] }, "journal-meta": { "journal_id": "me9", "type" : "journal" }} If I have a lot of flowfile that contain this kind of format Json file, what Avro schema should I put inside AvroSchemaRegistry. I try a lot of online converter and it doesn't satisfy AvroSchemaRegistry when I do validation. I need the Avro schema so that the ConvertRecord can read my JSON flowfile.Thank you in advance   

Hi,  I am trying to download any version of CDH. As noted on the download page I must include a valid username and password associated with an active Cloudera subscription. Now, I am sure, that my company is a Cloudera Partner, so how do I know which of my colleagues has received the license key file? We are a big company so I am not sure where to start from the inside. Is this information available on the website? Thanks,Cisco

Hello, Studying the documentation below I found a good example of how to use Spark with JDBC to connect to external databases. https://spark.apache.org/docs/latest/sql-data-sources-jdbc.html I already tested the example and it worked very well. However, when I looked carefully I saw that the password (its text) was being manipulated directly by the program. This way, from the tests I did, the password is open both in the Spark log and the connection, if it is intercepted on the network, the password will be open in open text. I'm looking for a way to make this connection safely. One mechanism that comes to mind is the hadoop credential provider that I already used with good old sqoop. So in sqoop the password was protected and no visible in any form, even if the connection was intercepted. Could anyone tell me what mechanisms I can use to make this connection securely, without exposing credentials? It can be a credential provider or something else.I didn't find anything like that in th

Hello,I've noticed how the CML Terminal lets you install Python packages on the 'Net using pip.(A gcc compiler is even available!)Isn't that a security risk?Is there a way to only allow package installation from an in-house repository?Many thanks.