Member since
09-23-2022
4
Posts
0
Kudos Received
0
Solutions
06-20-2023
10:21 AM
It just gave this information in DEBUG. "PutAzureDataLakeStorage DataLakeServiceClient is not found in the cache with the given credentials. Creating it."
... View more
06-20-2023
09:12 AM
Hi, I am connecting to ADLS from NiFi using PutAzureDataLakeStorage 1.21.0 How do I create a directory using this processor? The document says a directory will be created if it dont exist. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-azure-nar/1.12.0/org.apache.nifi.processors.azure.storage.PutAzureDataLakeStorage/index.html But my processor is not creating the directory but its failing with error. I am providing the directory as "devfolder" "PutAzureDataLakeStorage PutAzureDataLakeStorage[id=ac451284-0924-383a-9840-06131cd20808] Failed to create file on Azure Data Lake Storage reactor.core.Exceptions$ReactiveException: io.netty.channel.ConnectTimeoutException: connection timed out:" do I have to do something special to create directories. Also, if I set the directory to empty string the file gets created in the root of my ADLS container. Regards.
... View more
Labels:
- Labels:
-
Apache NiFi
09-27-2022
03:20 PM
Hi ,
we are consuming and publishing from MQ.
Consumer is fast enough, the publisher is slow and it’s builds a lot of back pressure.
we have a 4 node 20 core each * 4 total 80 cores.
memory 16GB each 64GB total.
publish is running at 200 concurrency.
I did system diagnostics cores and memory are free, then what is causing the slow publish and build back pressure.
I wanted to try and increase run duration to increase the throughput but this processor don’t seem to have the option to modify run duration.
what other option there to increase throughput.
let me know if you need more information .
Regards
... View more
Labels:
- Labels:
-
Apache NiFi
09-23-2022
03:05 PM
Hi Team, We have a 4 node cluster with 64 cores. but when we do system diagnostics we see only average usage of 5 cores. also the data is queuing up and processing slowly. regards, samyan
... View more
Labels:
- Labels:
-
Apache NiFi