Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Text file to Parquet file conversion using Pig

Solved Go to solution

Text file to Parquet file conversion using Pig

Explorer

Hi ,

Please help on the text file to parquet file conversion using pig.

Got this syntax.

store A into '/test-warehouse/tinytable' USING parquet.pig.ParquetStorer;

using this am getting error.

whether text file has to be loaded first into alias A. and then run the above store command.

"/test-warehouse/tinytable" - this path is parquet file put path ?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Text file to Parquet file conversion using Pig

@prakash pal I am assuming that you are using CDH or following this

A --> You are loading text file using parquet.pig.ParquetLoader

now, storing A as paruqet using parquet.pig.ParquetStorer

Answer: You will load text file=A

and then store A as parquet

4 REPLIES 4

Re: Text file to Parquet file conversion using Pig

@prakash pal I am assuming that you are using CDH or following this

A --> You are loading text file using parquet.pig.ParquetLoader

now, storing A as paruqet using parquet.pig.ParquetStorer

Answer: You will load text file=A

and then store A as parquet

Re: Text file to Parquet file conversion using Pig

Re: Text file to Parquet file conversion using Pig

Explorer

Thanks Neeraj for reply. It worked after syntactical changes. I loaded using PigStorage and stored using parquet.pig.ParquetStorer. Hope this is correct.

And original file is created as /user/folder/pig_file_par/part-m-00000.parquet.

Re: Text file to Parquet file conversion using Pig

@prakash pal Perfect! Please close the thread by accepting the answer.