Created 12-28-2017 05:49 PM
Hi ,
When I am trying to Overwriting File then getting below error
is there any alternative solution to "replace"(Delete existing "parquet" file and put new file )
Created on 12-28-2017 06:41 PM - edited 08-17-2019 05:35 PM
We cannot change the existing filename in HDFS but we can do alternate solution in NiFi as below.
You need to use UpdateAttribute processor before PutParquet processor.
Update Attribute
we are going to update the filename before putparquet processor so that everytime when file goes to putparquet processor will have same file name every time.
Add new property to Update attribute processor
filename
desired_parquet_filename.prq
Configs:-
PutParquet:-
So we are going to have same filename everytime and in this processor we need to change
Overwrite Files property to True //if the same filename exists in the directory processor will replace the existing file with new file
Configs:-
Flow:-
1.GetFile 2.UpdateAttribute //change the filename by adding filename property 3.PutParquet //change the Overwrite files property to true
If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of errors.
Created on 12-28-2017 06:41 PM - edited 08-17-2019 05:35 PM
We cannot change the existing filename in HDFS but we can do alternate solution in NiFi as below.
You need to use UpdateAttribute processor before PutParquet processor.
Update Attribute
we are going to update the filename before putparquet processor so that everytime when file goes to putparquet processor will have same file name every time.
Add new property to Update attribute processor
filename
desired_parquet_filename.prq
Configs:-
PutParquet:-
So we are going to have same filename everytime and in this processor we need to change
Overwrite Files property to True //if the same filename exists in the directory processor will replace the existing file with new file
Configs:-
Flow:-
1.GetFile 2.UpdateAttribute //change the filename by adding filename property 3.PutParquet //change the Overwrite files property to true
If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of errors.