Q&A

Hello all,I am getting issues after starting NIFI (LDAP enabled). I started successfully to log in my NIFI, but all components are disabled as below.I guess the problem may be in setting at authorizers.xml, but I couldn't find anything from there. Below is my authorizers.xml. Could you please give me any ideas for this issue? Thank you in advance.    <?xml version="1.0" encoding="UTF-8" standalone="yes"?><authorizers> <userGroupProvider> <identifier>file-user-group-provider</identifier> <class>org.apache.nifi.authorization.FileUserGroupProvider</class> <property name="Users File">./conf/users.xml</property> <property name="Legacy Authorized Users File"></property> <property name="Initial User Identity 1">cn=admin,dc=ananas,dc=com</property> </userGroupProvider> <accessPolicyProvider> <identifier>file-access-policy-provide

SuccessAccess Granted: Certificate authenticated. home This is what I see when I login with certificate. After pushing home button it redirects me again to login page. I have checked the logs couldn't find any error in server generated fils. Can anyone help me to troubleshoot it?

I'm getting the "Transaction already in progress error message" when attempting to do anything in the NiFi UI. Adding a user, processor, label, etc. The only thing the UI lets me do is manage my nodes in the cluster via the cluster tab.Some things to note:Version: 10.11.0My nifi.web.https.host value in nifi.properties lists the same value on all three nodes. (Example: user-nifi) Instead of (user-nifi1, user-nifi2, user-nifi3).My nifi.web.proxy.host value lists all three nodes in the instances, in addition to localhost

Dears, We have two node using embedded Zookeeper. The first node ran okay but the second node it always disconnects from the cluster with this error:  CONNECTING to DISCONNECTED due to Proposed configuration is not inheritable by the flow controller because of flow differences: Found difference in Flows: Local Fingerprint: s.email.ConsumeIMAPNO_VALUEorg.apache.nifinifi-email-nar1.13.210 sec30 sec1 secWARNfalseTIMER_DRIVENPRIMARY0Mark Messages as Read=truefolder=Inboxhost=outlook.office365.commail.imap.socketFactory.clas Cluster Fingerprint: s.email.ConsumeIMAPNO_VALUEorg.apache.nifinifi-email-nar1.13.210 sec30 sec1 secWARNfalseTIMER_DRIVENALL0Mark Messages as Read=truefolder=Inboxhost=outlook.office365.commail.imap.socketFactory.class=ja   BTW, using Nifi 1.13.2

Hi Team ,    Unable to change interpreter  by user , need to enable multiple user support .one admin user to be activate. Error attached on interpreter

I need a small help in modifying the JSON file by a content of its own object. My Json array looks like below [ { "firstName": "John", "lastName": "Wick" }, { "firstName": "Lisa", "lastName": "Tandon" }]   The final output expected is as below   [ { "firstName": "John", "lastName": "Wick", "fullName": "John Wick" }, { "firstName": "Lisa", "lastName": "Tandon", "fullName": "Lisa Tandon" }]   I have tried with ReplaceText, but not able to access content inside Json. PS: Won't be able to make Json to attribute and back to AttributeToJson as data is huge considering number of parameters (all not listed above)   Can anyone help me what should be replaceText formula or Jolt formula

How to suppress below INFO and WARNING messages using log4j.properties log4j:WARN No appenders could be found for logger (com.ibm.connect.shadow.parquet.crypto.EncryptionPropertiesFactory).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. INFO org.apache.hadoop.io.compress.CodecPool - Got brand-new compressor [.snappy] 

I am trying to create a table using the CREATE TABLE statement, in hue it works perfectly but when I use the dbexecute function in r, I get an error. How could it be solved.   library(DBI)library(odbc)con_Hive <- dbConnect(odbc::odbc(), "Hive_produccion", database = "cld_ml_bi_eng", encoding = "latin1")con_HivedbExecute(conn = con_Hive, statement = "CREATE EXTERNAL TABLE iris(sepal_length double, sepal_width double, petal_length double, petal_width double, species varchar(10))LOCATION '/sandbox/CLD_ML_BI_ENG/iris'")Error: nanodbc/nanodbc.cpp:1655: 42000: [Cloudera][Hardy] (80) Syntax or semantic analysis error thrown in server while executing query. Error message from server: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [RDIAZ] does not have [CREATE] privilege on [default/iris] <SQL> 'CREATE EXTERNAL TABLE iris(se

Hi Guys, I'm pretty new in this world of CDP, and I would like to know if I have a CDP Private Cloud license, but in a short time, I will migrate to a CDP Public cloud.The same License that I have purchased for Private cloud, works for Public Cloud?Or I need to purchase a new license for CDP Public Cloud?Thanks in advance,Alfonso C.

Hello everyone, I am new for big data framework. Currently I am working for collect many type of files from ftp/sftp server and put in HDFS. I just research about apache Nifi it good to go, but the interface look not good to monitor because all flow in the same place. Anyone have more experience about this could you recommend the best framework for this task?   Thanks,  

Hi,is it possible to get the pdf version of CDP Documentation?https://docs.cloudera.com/cdp/latest/index.html Thank you, best regards 

Hello, I'm using Hue to visualize document indexed in SolR.I would like to Hide from Hue some field that are part of the SolR collection schema.The image below is what I see from Hue when I open a indexed document, I would like to know if it is possible (and how), hide the yellow fields from Hue. This is the schema used to create the SolR collection:I've tried to set stored=false and indexed=false but this seems to have no effect. Could you please provide some hint? Thanks

Hello! I'm getting files from a SMB with GetSMBFile, eventually they'll be placed on S3 with PutS3Object, the problem is the path contains "\" on it and I need to replace all of those to "/", I attempted a simple ${path:replaceAll('\','//')}/${filename} on the Object Key but it didn't work. What's the proper way to do this on NiFi?

I'm loading the pyspark(spark-2.4 & python-3.7.4) data frame to Hive using Hivewarehouseconnector. Hive tables are partitioned and orc formatted. Data frame can contain multiple partition values. I need to load those partition values in single load and overwrite if that partition was already exists.   df.select(columns).write\.format(HiveWarehouseSession().HIVE_WAREHOUSE_CONNECTOR)\.mode('overwrite')\.option('inferSchema', 'true')\.option('table','tablename')\.option('partition','partition_column')\.save()   Got the below error. I'm passing right partition column name.Caused by: java.lang.IllegalArgumentException: Invalid partition spec: partition_column. Even in append mode, it's not working. I'm getting same error. How to accomplish the load? Kindly help. 

In my project I am synchronizing neo4j data with mysql by using nifi to connect to kafka and neo4j plugin Streams to bring the data from kafka to neo4j. Inside nifi I have this structureQuerydatabasetable -> UpdateAttribute -> PublishKafkaRecordIt triggers on any change and gets updated data, but to be able to optimize my neo4j code I need to know the type of query triggered in sql (insert, update, delete). It seems that I can't get it with Querydatabasetable processor, so which processor I should use for that purpose?

Hello,Following https://docs.cloudera.com/data-engineering/cloud/api-access/topics/cde-api-get-access-token.html I am unable to get a response from the curl command, just hanging. Maybe it would finish with a timeout.Seems this endpoint is not accessible. Any help to troubleshoot is welcomed.

Hello,I'm trying to figure out whether Azul Zulu versions 8.58.0.13, 11.52.13 or 17.30.15 are supported by Cloudera and compatible with CDP Private Cloud 7.1.7, CM 7.4.4 and CDSW 1.9.0 ? Thanks and regards,Olek

Hello, I have completed registration for CDP Public Cloud(60 days trial) But still I am waiting for access details from Cloudera end. How much time it will take to get the required details? Can anyone please help me with this. Thanks 

I want to copy a table from HBase to HBase across clusters by using copytable command, by default it is set to 1 mapper and scans all rows which cause a timeout. Are there any options available for the HBase copytable command in a way to optimize performance? without specifying any parameter to HBase-site.xml. hbase org.apache.hadoop.hbase.mapreduce.CopyTable --peer.adr=myserver:/hbase --new.name=<<tablename>>  <<tablename>> 

Hi all, I am passing a FlowFile with several custom attributes to my ExecuteScript processor (running python). I want the ExecuteScript to create two new FlowFiles, each having their own set of attributes (the values are to be retrieved from the input FlowFile). Problem is, I am unable to add any attributes to the two new flowfiles and it throws the error "org.apache.nifi.controller.repository.StandardFlow" object has no attribute "putAttribute".  flowFile = session.get()if flowFile != None: ... destFlowFile = session.create(flowFile) # doesn't work destFlowFile.putAttribute(destFlowFile, "logMsg", "Testing Msg") destFlowFile.putAllAttributes(destFlowFile, backupAttributes)  I've followed the examples listed in this guide https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-1/ta-p/248922, hence I wasn't expecting this to not work. Is it just not possible to add attributes to created FlowFiles and if so, are there any a

Hi,Wanted clarification/guidance related to finalize process for HDFS. Could we keep using the cluster while the finalize upgrade process is ongoing e.g., data is being written to hdfs while finalize process continues in the background OR would we need downtime for the same? ThanksWert

I am trying to get the data from elasticsearch from processor group via port. I want to apply some python code on the output data. How can I read the input and execute the code?

Hello,after some operation in NiFi GUI (parameter value change in a parameter context), the GUI got sort of stuck.A controller service show status disabled in GUI, however, I cannot enable or delete a controller service, as the NiFi claims it is running on a node.Do you have some Idea how to get rid of this situation?Best regardsJaro

Hello allI have a NiFi data flow that has flowfiles containing multiple json records (with exactly the same received time) and I'm trying to extract the timestamp as a flowfile attribute as follows:All the received times match as I'm using a PartitionRecord processor prior to EvaluateJsonPath, I have confirmed they are identical.I am aware that I can split > extract > merge but I am trying to avoid that due to the high volume of data passing through this flow, is there any way to just extract the first timestamp for all the records in a single flowfile using EvaluateJsonPath? or another record processor? I am also trying to avoid having to use ExtractText and regex to pull the timestamp, again due to the volume of data in the flow. There is an existing flow working in our production environment using plain/text and regex etc. I am trying to re-design using record processing to improve efficiency.  any assistance welcome. 🙂   

Hello experts. I have a text file reading into Nifi flows. Part of my flow is splittext > extracttext. I have split the text as line by line using SplitText Processor. Daily file is generated with the process output reads as below: 20211129-04:00:26 RG1287.kla EOF mark not found!20211129-04:00:55 RG9625.kla EOF mark found!....and so on How can i configure extracttext processor to result values as attributes for the each flow file.The values would be:date:20211129 Group 1:RG1287.klaGroup 2:EOF mark not found! Appreciate the help. Thanks!