question Re: Data is not coming in proper order while storing output results from pig into hive table in Archives of Support Questions (Read Only)

Data is not coming in proper order while storing output results from pig into hive table

jrathod — Mon, 30 Oct 2017 21:35:35 GMT

Two csv sheet data is loaded into the pig and applied the union operation on that and below is the command with its output in PIG command-in-pig.png pig-output.png but while storing this output from pig into hive table which I have already created,the data is not coming in correct order and below are the result of that pig output in hive table hive-output.png

Re: Data is not coming in proper order while storing output results from pig into hive table

kgautam — Mon, 30 Oct 2017 23:14:59 GMT

By order do you mean the sequence of row in pig output and hive output .

If yes then that will never match.

Pig writes into file in hdfs using MR / TEZ .

This file is mapped to a hive table.
When you run a query on hive select * from Table , hive spawns a Map only job to read the file. now which mapper completes first and display output is not determenistic.

Re: Data is not coming in proper order while storing output results from pig into hive table

jrathod — Tue, 31 Oct 2017 11:59:49 GMT

Thankyou @kgautam for this information. It will be helpful if you suggest me any other way through which I can get the data in correct row sequence in hive table through PIG.