Member since
11-22-2017
10
Posts
0
Kudos Received
0
Solutions
03-12-2018
07:03 AM
Thanks all. One more question: How does PutParquet Processor connect to HDFS? By WebHDFS , HttpFS or Native ?
... View more
03-09-2018
06:49 AM
I have a special case which is required to write out Parquet file into local directory. I tried not to enter 'Hadoop Configuration Resources' but it will instantly throw exceptions. If I entered "Hadoop Configuration Resources", then the output path is actually points to path inside the HDFS cluster. So, I would like to know, Does PutParquet processor support writing to Local Folder?
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache NiFi
11-22-2017
08:41 AM
@Shu Thanks for the reply, however using UpdateAttribute processor will have multiple parquet files output. What I want to achieve is: N files under directory > 1 .parquet file
... View more
11-22-2017
07:07 AM
I have a process group as follow: ListFile > FetchFile > mergeContent >convertCSVtoAvro > PutParquet On 1st execution, everything works fine, which all 15 files in the directory are written into parquet. After that, if new file is added in the directory, it will be ingested, but the original parquet file was overwritten. What I want is `append` contents of the new file into the parquet file, but not 'overwrite' it. I would like to know are there any approach/ processor to resolve this issue?
... View more
Labels:
- Labels:
-
Apache NiFi