Q&A

Hello.I am trying to connect to an Cloudera server, from another server (its an php web server).Everything has been setup correctly on the php server side. However, i'm still getting the error mentioned in the subject when trying to connect: [28000][unixODBC][Cloudera][ThriftExtension] (9) Error occurred while authenticating via SASL. Error details: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (No Kerberos credentials available (default cache: KEYRING:persistent:0))[ISQL]ERROR: Could not SQLConnectMy authentication is local, so i've set AuthMech to 1.No SSL for connecting into the server (not needed)I can connect from the php server to the cloudera host (tested with an telnet)The credentials are okay (since i can connect using the web browser).The php server is an RHEL Version 7.6And the log isn't working. I manually created the file, changed its permissions, but nothing is being logged.I'll put my configurations here. A

Trying to setup integration with AWS S3 bucket with Nifi Registry in ECS FARGATE, while looking to the cloudwatch logs found that the providers is not valid, below is the provider.xml file used with nifi image version 1.28.1<?xml version="1.0" encoding="UTF-8" standalone="yes"?><providers> <flowPersistenceProvider> <class>org.apache.nifi.registry.provider.flow.FileSystemFlowPersistenceProvider</class> <property name="Flow Storage Directory">./flow_storage</property> </flowPersistenceProvider> <extensionBundlePersistenceProvider> <class>org.apache.nifi.registry.aws.S3BundlePersistenceProvider</class> <property name="Region">us-east-1</property> <property name="Bucket Name">s3://nifi-registry-test/</property> <property name="Key Prefix">flow/</property> <property name="Credentials Provider">DEFAULT_CHAIN</property>

Hi everyone,I hope everything is fine with you.I'm having an issue with my nifi flow, I'm using nifi 1.26.0 and a PutFTP processor.In queue, my files have attributes and everything is ok, but when the file goes to PutFTP it seems like it loses his attributes.My processor gives this errorPutFTP[id=0193100b-ea3b-104f-002c-f975c98b8819] Processing halted: yielding [1 sec]: java.lang.NullPointerException: Cannot invoke "String.startsWith(String)" because "fullPathWithSlash" is null Does anyone face something like that? Thanks in advance.

Hi, I am Cloudera admin for a client in Riyadh Saudi Arabia. They purchased the the highest level of support  and have not been able to access the support portal all this time, I created for them a myCloudera account for them in order to be able to raise support tickets but for some reason I cannot access the Support Portal as it shows me that my access is restricted. Has anyone faced this before and how did you resolve it?

Hi We are currently in the process of upgrading our NIFI version to 2.0.0. But as per the below NIFI jira task PrometheusReportingTask has been completely removed from version 2.0.0 and added as a NIFI rest api endpoint.https://issues.apache.org/jira/browse/NIFI-13507The issue we are currently facing is that with old version 2.0.0-M2 metrics was exposed through PrometheusReportingTask which can be accessed without the need for any authentication and we were able to scrape the metrics to Prometheus.But with the latest versions from 2.0.0 we have to provide the Bearer Token(Using OIDC) to access the metrics as it is a Rest API endpoint which is causing difficulties to integrate with Prometheus as we need to add an extra layer to get the token.So my question here is, If there is any way to access the nifi-api/flow/metrics/prometheus without the need to provide any authentication which will solve our issue.Any Suggestions are appreciated. 

Hi NIFI Team,I am currently working with Apache NIFI 2.4.0 and have successfully enabled Active Directory (AD) authentication. My AD users can log in, and I can assign policies to each user.I have another application that also uses AD authentication. However, when I try to authenticate NIFI using the token generated by this application, I receive a "User does not have permission" error—even though the user is an admin.Upon comparing the tokens, I noticed that the "sub" field in NIFI's token contains the user's email ID, whereas the token from my other application uses a unique ID from Azure. However, the "preferred_name" field is the same in both tokens.Could you help me resolve this issue? Additionally, is there an official chat channel or forum where I can seek assistance for NIFI? I am new to NIFI and would appreciate any guidance.I have attached my "nifi.properties" and "authorizers.xml" files for verification. Please let me know if my configurations a

Hello,in hive i am unable to establish a connection,beeline> !connect jdbc:hive2://localhost:10000/Connecting to jdbc:hive2://localhost:10000/Enter username for jdbc:hive2://localhost:10000/:I don't know what to put for username and password, it is not accepting ENTER Key or my device login details.Output: 24/05/20 01:19:08 [main]: WARN jdbc.HiveConnection: Failed to connect to localhost:10000Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status. Enable verbose error messages (--verbose=true) for more information.Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0)Please help.

@Shelton My team and I have been running Apache Ranger Admin v2.4.0 in Docker. We recently started an effort to try and upgrade Apache Ranger Admin v2.5.0. In v2.4.0, we set up the install.properties file to correctly point to our LDAP server for logging in to the Ranger Admin UI and everything was working well. One thing to note, is that in v2.4.0 we didn't have usersync running, and we were still able to login.After upgrading to Apache Ranger Admin v2.5.0, we ran into an issue trying to login to the Ranger Admin UI. In the logs this is what it showed...ranger-admin.log -2025-02-12 20:14:42,853 [https-jsse-nio-6182-exec-8] INFO [SpringEventListener.java:76] Login Successful:drew.nicolette | Ip Address:xxxxxxxxx | sessionId=5F137022A5347056C441709DDC19A26F | Epoch=17393912828522025-02-12 20:14:42,890 [https-jsse-nio-6182-exec-8] ERROR [SessionMgr.java:486] Error getting user for loginId=drew.xxxxxxcatalina.out -java.lang.RuntimeException: Failed to create user drew.xxxxx

I'm encountering challenges when using the InvokeHTTP processor in Apache NiFi to send GraphQL POST requests. Specifically, I'm trying to execute GraphQL mutations via POST but facing difficulties in configuring the InvokeHTTP processor to handle these requests properly.Problem:I've set up an InvokeHTTP processor to send GraphQL mutation requests via POST to a specific endpoint. However, the requests don't seem to be reaching the GraphQL server as expected. Details:The InvokeHTTP processor is configured with the following properties:HTTP Method: POSTRemote URL: myURL...!Headers: Content-Type: application/json, Accept: application/jsonSend Parameters in: BodyThe body content is constructed in JSON format as required by the GraphQL server. It includes the mutation query and variables. Request for Help:I'm seeking guidance on how to properly configure the InvokeHTTP processor in NiFi to successfully send GraphQL POST requests. If anyone has experience using InvokeHTTP for GraphQ

I am running nifi locally using docker, I tried to generate token using the following url and obtained token successfullyhttps://localhost:8443/nifi-api/access/tokenbut when I tried to make POST call to the following URL, I received 403 error code. I used same token generated with above request, can anyone suggest what am I doing wrong?https://localhost:8443/nifi-api/process-groups/d9a891ee-017e-1000-4067-96bceee7d5e3/template-instance 

I have few tables in hive which I need to access in my SAS platform.I have created a Pass through query present below (marked some paths in *) but it is throwing error.  PROC SQL;    CONNECT TO HADOOP(    host = "bigdataplatform-gcg-&amb..lac.nsroot.net"    schema = &ESQUEMA.    HDFS_TEMPDIR = "/data/&HDFS./work/hive/&HDFS._ofc/"    DBMAX_TEXT = *****    uri = "jdbc:hive2://bigdataplatform-gcg-&amb..lac.nsroot.net:10000/&ESQUEMA.;principal=hive/*****T;ssl=true"     scratch_db = "&HDFS._ofc"subchar=questionmark    );      CREATE TABLE Applicant AS    SELECT     * FROM CONNECTION TO HADOOP ( WITH json_extract AS (  SELECT     get_json_object(xml_data, '$.app.Id') AS ID,    get_json_object(xml_data, '$.app.apply[0].flag') AS Flag,    get_json_object(xml_data, '$.app.apply[0]

How to download the installation package for deploying CM6.3.1 on Ubuntu system? The dep file converted from the rpm file applicable to the RedHat system may encounter issues during installation. How to solve it??

Hello everyone.I'm trying to connect to an Cloudera culster - using Impala.My certificate is valid (i can OpenSSL it to the host)(this is an excerpt of  the openssl command output) CONNECTED(00000003)depth=1 C = BR, ST = **, L = CITY_NAME, O = ****** ***, OU = IT, CN = Hadoop CA Authorityverify return:1depth=0 C = BR, ST = **, L = CITY_NAME, O = ****** ***, OU = IT, CN = impala.******.***.brverify return:1---Certificate chain 0 s:/C=BR/ST=**/L=CITY_NAME/O=****** ***/OU=IT/CN=impala.******.***.br i:/C=BR/ST=**/L=CITY_NAME/O=****** ***/OU=IT/CN=Hadoop CA Authority---  My DSN is  also correct (i think):[ImpalaDSN]Driver=Cloudera Impala ODBC DriverHost=host.xzv.jkjPort=21050AuthMech=3UID=theuserPWD=thepasswDatabase=dataLogLevel=3LogPath=/tmp/odbc_trace.logUseSASL=1SSL=1SSL_VERIFY_SERVER=0SSLTrustStore=/etc/pki/CA/certs/RootCA.pemAllowSelfSignedServerCert=1AllowInvalidCACert=1ThriftTransport=2SSL_VERIFY_CLIENT=0  Also, i can connect to this host using Wi

I am running 3 nifi nodes using AWS ECS, while the development team is working over the nifi console, we are getting an error related to the unable to sync. So the node is being disconnected.My mind is this issue is happening due to that the changes are not being made over the primary node.How can I enforce that the changes are applied directly to the primary node to avoid this issue?Can any one help?

My NiFi (1.24) is running using the AWS ECS, 3 nodes, and connected via ALB as a cluster.So, my case is how to make sure that the opened node by the users/developers is the primary one (When accessing the ALB URL).This is to avoid allowing them to apply their changes over the non-primary one, as almost with every new flow update by the developer, I am getting one of my nodes disconnected and unable to sync with the primary one! 

I am fairly new to Nifi and I am using 2 GetSFTP processors to get file from two remote servers, these are application.properties files. Now I want to compare the contents of these files and check for any differences.Need a little guidance on how I can achieve this in NiFi.

Hello,I need your guidance on the following scenario:We have a SIEM (QRadar) infrastructure where Event Collectors receive logs from various data sources. These logs are correlated based on SIEM rules and use cases.we plan to send logs to the SIEM while also storing a copy in a Data Lake.Current Approach:We have structured the workflow as follows:DATA SOURCES → NiFi → Store in DATA LAKE & Forward to SIEM (using PUTTCP processor)Questions:Does NiFi allow segregation of data sources?Handling LEEF logs:logs reach NiFi with the original source IP but leave NiFi with NiFi’s source IP.Since QRadar first looks for the hostname in the payload (and if absent, uses the source IP), this could cause misidentification.Can NiFi be configured to retain the original source IP while forwarding logs, without modifying the original log (to comply with legal requirements)?Log Integrity & Authenticity:Does NiFi ensure log integrity and authenticity for legal and compliance purposes?LEEF Parsing:Is

I am unable to understand as I have just started with data engineering and saw a course using the HDP. I am not able to launch the image properly.Log link : https://drive.google.com/file/d/1MxawAFc68796kkh4V8I_dbILdH6bHOxQ/view?usp=sharing Configs:I changed the custom16 TCP port, host, and guest, to 1024. Please help me!

The JSON is example is :{"app": {"Id": "123","apply": [{"flag": "1","Product": [{"Code": "DZ9e","Line": "10000","status": "C",}{"Code": "EZ9e","Line": "19000","status": "C",}]}}I need output likeIDFlagCodeLineStatus1231DZ9e10000C1231EZ9e19000C I have used lateral explode/lalteral inline explode . But it is not giving value in separate rows.Anybody has any solution

Is there a way to overwrite an existing file with PutFile processor?Nifi version is 1.23.2

Hello, I'm facing some issues trying to install CM 7.13.1 on Ubuntu containers (happens in both 22.04 and 20.04) using a Postgres 10 14 server running on its own container.The install of both CM server and CM agents work fine, but when I start the cluster creation it fails during the CM agents startup. The following error is reported in the agent logs (cloudera-scm-agent.log): [11/Feb/2025 09:32:33 +0000] 8471 MainThread agent ERROR Heartbeating to node-1.cluster:7182 failed.Traceback (most recent call last): File "/opt/cloudera/cm-agent/lib/python3.8/site-packages/cmf/agent.py", line 1642, in _send_heartbeat response = self.requestor.request('heartbeat', heartbeat_data) File "/opt/cloudera/cm-agent/lib/python3.8/site-packages/avro/ipc.py", line 132, in request self.write_call_request(message_name, request_datum, buffer_encoder) File "/opt/cloudera/cm-agent/lib/python3.8/site-packages/avro/ipc.py", line 174, in write_call_request self.write_req

Hi Team,We want to integrate Nifi with 2 LDAP servers and both have different manager DN, we are using below config in "login-identity-providers.xml" file  <provider> <identifier>ldap-provider-1</identifier> <class>org.apache.nifi.ldap.LdapProvider</class> <property name="Authentication Strategy">SIMPLE</property> <property name="Manager DN">CN=yaqootnifi,OU=Groups,OU=UATOCP,OU=OpenShift,DC=CORP,DC=SA,DC=ZAIN,DC=COM</property> <property name="Manager Password">*********</property> <property name="TLS - Keystore"></property> <property name="TLS - Keystore Password"></property> <property name="TLS - Keystore Type"></property> <property name="TLS - Truststore"></property> <property name="TLS - Truststore Password"></property> <property name="TLS - Truststore Type"></proper

Hello,On the CDP gateway page a number of API services are listed among which Impala. The endpoint has the format https://host:port/gateway/cdp-proxy-api/Impala. My question is can this endpoint be used to submit actual queries? And, if yes, is there any documentation available? Thank you!

Hi, We are running nifi 1.24, some times when developers apply the changes it causes a discount for one of the nodes, and it is unable to sync with the other nodes.How can I have a re-sync button to re-sync the flows, to avoid the need to delete the flow directly from the affected node?

Hi everyone,I would like to ask for technical support in Apache NiFi 2.2.0. In my project, I want to integrate Apache NiFi 2.2.0 to Postgres, which is my data source using DBCPConnectionPool. I use ExecuteSQL processor to interact with PostgreSQL, but I encounter the following error when verifying.Some solutions suggest me to check if DBCPConnectionPool is enabled or not. I go to check, it is already enabled. Please help me.Thank you in advance.