question Re: Text file to Parquet file conversion using Pig in Archives of Support Questions (Read Only)

Text file to Parquet file conversion using Pig

Aswanth11 — Thu, 25 Feb 2016 15:53:59 GMT

Hi ,

Please help on the text file to parquet file conversion using pig.

Got this syntax.

store A into '/test-warehouse/tinytable' USING parquet.pig.ParquetStorer;

using this am getting error.

whether text file has to be loaded first into alias A. and then run the above store command.

"/test-warehouse/tinytable" - this path is parquet file put path ?

Re: Text file to Parquet file conversion using Pig

nsabharwal — Thu, 25 Feb 2016 16:02:39 GMT

@prakash pal I am assuming that you are using CDH or following this

A --> You are loading text file using parquet.pig.ParquetLoader

now, storing A as paruqet using parquet.pig.ParquetStorer

Answer: You will load text file=A

and then store A as parquet

Re: Text file to Parquet file conversion using Pig

nsabharwal — Thu, 25 Feb 2016 16:04:05 GMT

See this https://community.hortonworks.com/questions/9949/pig-parquetstorer-is-not-working.html

Re: Text file to Parquet file conversion using Pig

Aswanth11 — Thu, 25 Feb 2016 17:11:58 GMT

Thanks Neeraj for reply. It worked after syntactical changes. I loaded using PigStorage and stored using parquet.pig.ParquetStorer. Hope this is correct.

And original file is created as /user/folder/pig_file_par/part-m-00000.parquet.

Re: Text file to Parquet file conversion using Pig

nsabharwal — Fri, 26 Feb 2016 09:09:05 GMT

@prakash pal Perfect! Please close the thread by accepting the answer.