Solved

We have a delimited flatfile which has special characters ingested into HDFS. We have created a Hive External table on top of the HDFS path and visualized the data in Hue & Beeline. But the special characters are displayed as question mark (?). Please advise the solution.

I have a flowfile that is a list of files,that gets split into individual flowfiles (10 in this case)I am using FetchSFTP to fetch the file from a sftp location, the content of this fetched file is then split and is processed and stored.In this example: I want to start processing ep.txt only after i have processed bulk.txt after that names.txt and so on.I was referring to these links:NiFi workflow monitoring – Wait/Notify pattern with split and merge – Pierre Villardand How to wait for all fragments to be processed, then do something? (ijokarumawak.github.io)I was able to implement something like this. but this is not what I was looking for, I want to implement a wait before fetchSFTP I tried release signal identifier with filename , but that was stopping any flowfile from moving forward. Please advise, thank you 

After deleteing all version of a flow from nifi registry, restarting a node cannot connect cluster, It always display below error. How can I resolve it? 2021-12-16 22:13:17,520 ERROR [Timer-Driven Process Thread-149] o.a.nifi.groups.StandardProcessGroup Failed to synchronize StandardProcessGroup[identifier=d059cc81-0176-1000-0000-0000089d93e4,name=整机CMES] with Flow Registry because could not retrieve version 3 of flow with identifier 5d63465f-9893-4f93-8d27-b8c258519d68 in bucket fbcc346b-ee7c-4ec4-ba7d-68c0f4c306b1org.apache.nifi.registry.client.NiFiRegistryException: Error retrieving flow snapshot: The specified flow ID does not exist in this bucket. at org.apache.nifi.registry.client.impl.AbstractJerseyClient.executeAction(AbstractJerseyClient.java:117) at org.apache.nifi.registry.client.impl.JerseyFlowSnapshotClient.get(JerseyFlowSnapshotClient.java:97) at org.apache.nifi.registry.flow.RestBasedFlowRegistry.getFlowContents(RestBasedFlowRegistry.java:213)

Does HDFS perform load balancing on the storage of data blocks when the disk space of each node is different? 

Hello, We are facing issues regarding KDC,  on CDP HBase and Impalad wont start and here below the logs: The Keytab file is generating kerberos ticket correctly  (using the command line kinit) and the service Keytabs have been regenerated several times but still the same issue. This issue is blocking: Impala, HBase and Yarn Nodemanagers Caused by: KrbException: Server not found in Kerberos database (7) - LOOKING_UP_SERVERat sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73)at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:226)at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:237)at sun.security.krb5.internal.CredentialsUtil.serviceCredsSingle(CredentialsUtil.java:477)at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:340)at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:314)at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:169)at sun.se

I am executing python script in ExecuteStreamCommand and my code last line dumps the json key values. My goal is to read those key values into PutDatabaseRecord processor and insert the values in my database. However, I am unable to read content of the flow file of ExecuteStream. I have used ConvertAvrotoJson but it is showing output as "not a avro data file". Is there any workarounf of it? Thanks.

Hello,we are performing Hive queries with PySpark using the HWC in JDBC_CLUSTER mode.Everything is running fine and we get the results for the queries, but we also receive a warning message saying that connection has been closed:23/08/16 09:59:05 WARN conf.HiveConf: HiveConf of name hive.masking.algo does not exist23/08/16 09:59:05 WARN transport.TIOStreamTransport: Error closing output stream.java.net.SocketException: Connection or outbound has closedat sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1181)at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)at java.io.FilterOutputStream.close(FilterOutputStream.java:158)at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)at org.apache.thrift.transport.TSocket.close(TSocket.java:235)at org.apache.thrift.transport.TSaslTransport.close(TSaslTransport.java:400)at org.apache.thrift.transport.TSaslC

Hi,how to define "Table Name Pattern" for more than 1 table in ChangeCaptureMySql processor configuration file.  I tried below patterns where "product_types" and "user_subscription" table names. I want to read data only from these tables.[product_types|user_subscription]_ or [product_types|user_subscription]% 

Hi all. I can't figure out why the MergeRecord processor converts fractional values ​​to integers. Reading origin from json file.

I am trying to configure CDP 7.1.8 with Active Directory. Once the Kerberos Configuration is enabled, during the last step where the services are started, the Zookeeper service is failing to start with the following exception. I had checked the encryption types enabled on the Active Directory as well as the encryption types with which the principal are created. Also, I could see that the kinit using the generated keytabs works fine for the zookeeper service and other services as well. Is there anything that is missing in the configuration ?--- 5:01:27.233 AM WARN SaslServerCallbackHandler No password found for user: null5:01:27.235 AM ERROR QuorumPeerMain Unexpected exception, exiting abnormallyjava.io.IOException: Could not configure server because SASL configuration did not allow the ZooKeeper server to authenticate itself properly: javax.security.auth.login.LoginException: No password provided at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(

If yes, please provide references to the documentation... Can't seem to find any.

Hi,   After Multi Delimiter Issue is solved. I got an error with data insertion. So, after the data successfully inserted to text file format table with rowserde hive.contrib.serde2.MultiDelimitSerDe. Then i need to insert those data to parquet format table. But when i ran the query to insert data, it failed.   Error message: "[FATAL] 10:25:08 vaproject.xxx tHiveRow_1 Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask".   I used this script: "INSERT OVERWRITE TABLE ParquetTable PARTITION (datadt=dateint) SELECT * FROM TextFileTable"   What is wrong or what am i missing? Please need your help. Thank you.     Sincerely,     Gideon Maruli IT Data Management

In a GenerateTableFetch i have set as Maximum-value Columns the column updated_at. In addition i have define the property "initial.maxvalue.updated_at" with an hard-code initial value. However i do not want to hardcode manual the initial value but instead grab it dynamically, like performing an SQL statement to get the max updated value from the table. How i could feed such information in "initial.maxvalue.updated_at" ?

Hello,I am trying to setup a HTTPS on NiFi Registry and am receiving SSL_ERROR_BAD_CERT_ALERT when attempting to access the URL after loading the certificates.The way I am setting it up is that I create a CSR, have the CA sign it (AD Domain Controller) and import that certificate into the keystore.   I have generated the keystore, the truststore imported the certificates, and when I restart the NiFi Registry service the logs show no errors so I am not sure what I am doing wrong. Below are some sanitized outputs of some commands I've ran to show what I could potentially have done wrong.Curl Output:curl --cert-type P12 --cert ./keystore.p12:Password --cacert /root/ssl/ADcerts/ad1-ca.cer -v https://nifiregistry.our.network.com:8443/nifi-registry* Trying 123.4.56.78:8443...* Connected to nifiregistry.our.network.com (123.4.56.78) port 8443 (#0)* ALPN, offering h2* ALPN, offering http/1.1* CAfile: /root/ssl/ADcerts/ad1-ca.cer* TLSv1.0 (OUT), TLS header, Certifica

Hi Everyone, I have Ranger 2.4 installed. On the ranger ui, when I give a allow policy to my user, the policy is getting applied and my user is able to see the desired table. But when I give the same policy to a whole group instead of a single user, the users in that group are not able to see the desired table.Any idea how can I resolve this?

Profile is displaying as LONDON STOCK EXCHANGE GROUP PLC. But I need to change it REFINITIV US LLC

I has NiFi 1.11.4 installed on 3 nodes cluster. I installed the new version of NiFi 1.23.0 in a nearby directoryMoved config files:nifi.properties rolled over Ansibleautorizations.xml copied from the old versionjks files copied from the old versionflow.xml.gz copied from the old versionUpdated sensitive properties: sudo ../bin/nifi.sh set-sensitive-properties-key 'key'sudo ../bin/nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256 What does this mean when I do update flow with sensitive props? Flow Configuration Property not specified [null]And now both my versions of NiFi could not initialize it flow, no exceptions or critical errors in logs i've found, some was fixed but nothing helpsCluster coordinator is elected successfully, Zookeeper is available.No any logs in nifi-processors.logs, no any updates on directories for provenance, flowfile, content.I enabled TRACE logs and on Initializing I see only this continuously:2023-08-23 14:03:20,513 TRACE [Load-Balan

Dear Team, I am trying to create ADF Linked service to CDP Hive server. CDP kerberized and ssl enabled cluster. Cloud you share the configurations like below: {"name": "HiveLinkedService","properties": {"type": "Hive","typeProperties": {"serverType": "HiveServer2","thriftTransportProtocol": "SASL","serviceDiscoveryMode": true,"authenticationType": "UsernameAndPassword","httpPath": "jdbc:hive2://******;principal=hive/_HOST@******;serviceDiscoveryMode=zooKeeper;ssl=true;zooKeeperNamespace=hiveserver2","host" : "******","port" : "10000","allowSelfSignedServerCert" : true,"allowHostNameCNMismatch": true,"enableSsl": true,"useSystemTrustStore": true,"trustedCertPath": ".cer path file","zooKeeperNameSpace": "hiveserver2","username" : "******","password": {"type": "SecureString","value": "******"}}}} 

Some hdfs operations (hsync, append, ... ) are not supported with the erasure coding (ec) policy. Does hbase work well when the data folders of hbase are set to the erasure coding policy?Is Hbase compatible with HDFS-EC instead of HDFS-replica ?   

I'm running RHEL 7.9. I'm using free trial. While installing agents I got error message like this below  Please help give soliution. Thank you Regard,Doan Gia Hy

I am trying to connect Qlik to hadoop (cloudera ODBC Driver for Apache Hive) , but i am facing the following error. could someone suggest how to resolve the issue please. Error: failed! [cloudera][thriftextension] (14) unexpected response from server during a http connection: connect() failed: errno = 10060. Looking for your support please to find a solution.   Thanks

Hi  TeamI need my company name to be changed to Refinitiv LLC

I have flowfile which needs to be sent to two different processors which does transformation on the Json. Flow will be split into two parts which are independent of each other so they can be performed parallel but the input for these two parts is the same flow file. how can I send a flowfile in a queue to two different processors at same time.

Dears, After installing CDP Base on premise, I'm deploying NIFI services.For this reason as recommanded, I created the Data Context and created compute cluster with two hosts and installed activated the Parcels CDH 7.1.5 by default a CFM 2.1.5, and when arriving to the Service selection (Custom Services) I don't see the NIFI services, all services belongs to the CDH parcelsHow can I get them ? Thank you in advance

Hi, We are getting an error when initializing the zk Failover Controller here the error below: WARNING: HADOOP_ZKFC_OPTS has been replaced by HDFS_ZKFC_OPTS. Using value of HADOOP_ZKFC_OPTS.log4j:ERROR Could not instantiate class [com.cloudera.cmf.eventcatcher.client.logs.ExceptionForwarderAppender].java.lang.ClassNotFoundException: com.cloudera.cmf.eventcatcher.client.logs.ExceptionForwarderAppenderat java.net.URLClassLoader.findClass(URLClassLoader.java:387)at java.lang.ClassLoader.loadClass(ClassLoader.java:418)at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)at java.lang.ClassLoader.loadClass(ClassLoader.java:351)at java.lang.Class.forName0(Native Method)at java.lang.Class.forName(Class.java:264)at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198)at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327)at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124)at org.apache.log4j.Propert