About ctgj0004

ctgj0004 · ‎03-12-2018

Thanks all. One more question: How does PutParquet Processor connect to HDFS? By WebHDFS , HttpFS or Native ?

ctgj0004 · ‎03-09-2018

I have a special case which is required to write out Parquet file into local directory. I tried not to enter 'Hadoop Configuration Resources' but it will instantly throw exceptions. If I entered "Hadoop Configuration Resources", then the output path is actually points to path inside the HDFS cluster. So, I would like to know, Does PutParquet processor support writing to Local Folder?

ctgj0004 · ‎11-22-2017

@Shu Thanks for the reply, however using UpdateAttribute processor will have multiple parquet files output. What I want to achieve is: N files under directory > 1 .parquet file

ctgj0004 · ‎11-22-2017

I have a process group as follow: ListFile > FetchFile > mergeContent >convertCSVtoAvro > PutParquet On 1st execution, everything works fine, which all 15 files in the directory are written into parquet. After that, if new file is added in the directory, it will be ingested, but the original parquet file was overwritten. What I want is `append` contents of the new file into the parquet file, but not 'overwrite' it. I would like to know are there any approach/ processor to resolve this issue?

Online	Offline
Last Visited	‎03-26-2018 06:38 AM

Member Since	‎11-22-2017 07:07 AM
Last Visited	‎03-26-2018 06:38 AM
Posts	10

Cloudera Community

Cloudera Community

Re: Does PutParquet processor support writing to L...

Does PutParquet processor support writing to Local...

Re: How to append flowfiles into parquet instead o...

How to append flowfiles into parquet instead of ov...