Member since
01-30-2021
22
Posts
0
Kudos Received
0
Solutions
11-07-2023
05:24 AM
Dear Experts, I want to transfer data from NiFi to Elasticsearch using the PutElasticsearchHTTP processor. However, I am encountering the error "Elasticsearch returend code 400 with message bad request" The setup, which I have depicted in the image below, was working previously. How should I resolve this problem? Regards
... View more
Labels:
- Labels:
-
Apache NiFi
11-06-2023
05:16 AM
Dear Experts, I want to transfer data from NiFi to Elasticsearch using the PutElasticsearchHTTP processor. However, I am encountering the error "javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target" The setup, which I have depicted in the image below, was working previously. How should I resolve this problem? Regards
... View more
Labels:
- Labels:
-
Apache NiFi
05-14-2023
02:29 AM
Dear @SAMSAL , Because we have semi-structured JSON data in the "other" field, we do not have a fixed schema to set in the JsonTreeReader. We want a solution for batch processing of the flowfiles with that assumption.
... View more
05-13-2023
07:35 AM
Hi @SAMSAL , We have high TPS so our constraint is using batch processing. Therefore, we do not want to split records line-by-line and we have already split flow files to 1000 records. Therefore, we need a solution for batch processing in NIFI.
... View more
05-13-2023
01:10 AM
Dear Experts, We want to learn how to perform batch processing on semi-structured JSON flow files in NiFi. We know that we can use a Jolt Transform Record, an Update Record processor, and a Query Record processor, for which we need to set reader and writer controller services. However, we are unsure how to set up reader and writer controllers for semi-structured data schemas. In the text below, we have a multi-line JSON flow file in which the "code" field is static and the "other" field is an unstructured nested JSON. We want to query the "code" field in batch mode and include the "other" field information in the resulting flow files for processing in subsequent processors. How should we set the reader and writer controllers to achieve this? Regards, {"code": "6", "other": {"feild1": "data1"}} {"code": "7", "other": {"feild2": "data2"}} {"code": "8", "other": {"feild3": "data3"}}
... View more
Labels:
- Labels:
-
Apache NiFi
08-23-2022
05:15 AM
Dear Experts, We have some files in our HDFS server and want to get these files and put them into Isilon. We use GetHDFS and PutHDFS for this purpose. But, we get the lease and lock errors, as shown in the picture below. How can we fix these problems? Regards
... View more
Labels:
- Labels:
-
Apache NiFi
09-11-2021
04:53 AM
I found the below problem in the minifi-app.log: 2021-09-08 15:00:04,696 WARN [Timer-Driven Process Thread-3] o.a.nifi.processors.standard.TailFile TailFile[id=1697d3af-4b0c-3a38-0000-000000000000] Unable to open file /log/MyLog/myLog_120_20210908_125952.log; will attempt to access file again after the configured Yield Duration has elapsed: java.nio.file.NoSuchFileException: /log/MyLog/myLog_120_20210908_125952.log: In addition, I found that the line below is repeated regularly in the journal file of this directory: ./minifi-0.5.0/state/local/partition-0/ 3877966761 file.205.filename \/log/MyLog/myLog_120_20210908_125952.log.tar.gz file.331.checksum 967031169 file.420.checksum
... View more
09-07-2021
04:16 AM
I have a MiNiFi agent that tails from a file and sends it to a NiFi Cluster. After a while, The state directory takes 27 Gigabytes of the disk and the content_repository only takes 34MB of the disk. Any idea about this problem?
... View more
Labels:
- Labels:
-
Apache MiNiFi
-
Apache NiFi
06-09-2021
12:17 AM
I'm sending data from a MiNiFi agent to a NiFi Cluster. I set port 1026 for the nifi.remote.input.socket.port in the site to site properties section of the nifi.properties file in the NiFi cluster to get these data. I want to add another MiNiFi agent for sending some different data to the previously mentioned NiFi Cluster. So I want to add another port in the nifi.remote.input.socket.port for distinct receiving values in the NiFi cluster from MiNiFi agents. How can I add multiple values for nifi.remote.input.socket.port and if it is not possible, how can I handle receiving different values in the same NiFi cluster from MiNiFi agents. Nifi version: 1.8.0 MiNiFi version: 0.5.0 java
... View more
Labels:
- Labels:
-
Apache MiNiFi
-
Apache NiFi
05-02-2021
04:25 AM
I'm sending data from MiNiFi to the NiFi Cluster. Is there any way to keep MiNiFi backpressure data in the disk rather than the memory? I've also disabled the Swap feature for increasing performance. I'm worried about consuming lots of the memory by MiNiFi. Nifi version: 1.8.0 MiNiFi version: 0.5.0 java
... View more
Labels:
- Labels:
-
Apache MiNiFi
-
Apache NiFi