Member since
01-08-2021
12
Posts
2
Kudos Received
0
Solutions
04-12-2021
12:15 AM
Hi, thank you. However, this is the mechanism of every List processor: it keeps track of the files already processed. Instead, I was wondering if there was a way to start a trigger or a notification to the flow when new ones arrive in the blob. Thanks
... View more
04-09-2021
08:18 AM
@ApacheNifi I have this flow that take files from Azure Blob Storage (with ListAzureBlobStorage processor). Now.. is there some trigger or notification mechanism that indicates that new files to be processed have been uploaded to the Azure blob?? Thank you so much
... View more
Labels:
- Labels:
-
Apache NiFi
04-09-2021
01:29 AM
Hi @ApacheNifi, I have implemented this flow in Nifi. There are two types of files in the blobstorage (a and b). The ListAzureBlobStorage processor finds only the b files (which are few) on restart. To solve the problem I always have to delete the processors (List and RouteOnAttribute) and create a new ones with the same old settings. Why does this happen? does anyone have the same problem? Thank you so much
... View more
Labels:
- Labels:
-
Apache NiFi
03-08-2021
10:10 AM
yes it's quite right! i add a volumes on docker compose that connect directory on VM to docker-compose. Thank you so much!
... View more
03-08-2021
07:47 AM
No.. it's on virtual machine and not included into dockerfile of docker-compose.yml
... View more
03-08-2021
02:58 AM
Hi @Nifi ! I created a flow in local nifi, now i ve to do the deployment in virtual machine azure. I use docker compose to do that and it works but the ListFile processor to ingest the file doesn't accept the directory path of virtual machine. I insert the path that i find from virtual machine terminal (with the command pwd) but doesn't work. How can i ingest the file that i ve load on virtual machine? thanks
... View more
Labels:
- Labels:
-
Apache NiFi
01-19-2021
03:05 AM
Hi @ApacheNifi, i have i extracted the attributes from the filename and used AttributesToJson to create a json file to upload into elasticsearch (to create a catalog). It gives me many problems, what can I do? I also noticed that from updateattribute to attributetojson it adds "\ n" in two fields. This is my flow: UPDATEATTRIBUTE: I have correctly configured updateattribute and this is the result Setting: Result: ATTRIBUTESTOJSON: Configuration: Result: as you can notice in jsonattributes it adds a "\ n" at the end of the strings "giri" and "id-riduttore" fields PUTELASTICSEARCHHTTPRECORD: Configure: Error: How can i do this? thank you so much
... View more
Labels:
- Labels:
-
Apache NiFi
01-10-2021
10:30 AM
1 Kudo
I used the PutParquet processor with CSVReader to compress files.txt in parquet format and moving them to HDFS. Checking from the Browse Directory of Hadoop however the files keep .txt Is it normal? Are they saved in parquet format? Thank you! @ApacheNifi Configuration of PutParquet processor: Hadoop:
... View more
Labels:
- Labels:
-
Apache NiFi
01-09-2021
02:03 AM
1 Kudo
Hi, thank you so much for your answer. I understand that i had to treat the data as CSV format with a “tab” delimiter rather than a “,”. For my project i use this flow: GetFile -> UpdateAttributo -> PutParquet but something go wrong. The error that compare is: "Unable to create record reader". This is my processor configuration: THANK YOU @ApacheNifi
... View more
01-08-2021
09:08 AM
Hi, i can't compress so many files.txt into a Parquet format to save in HDFS. How can i do that? @ApacheNifi
... View more
Labels:
- Labels:
-
Apache NiFi
-
HDFS