question Re: I am getting this error to load the pig ouput into hive table how i can solve??? plz help me in Support Questions

I am getting this error to load the pig ouput into hive table how i can solve??? plz help me

Former Member — Thu, 01 Sep 2016 12:10:37 GMT

grunt> aa = foreach avg_rate generate $0, ( case when detik_rating > 0 then 'positive'   when detik_rating < 0 then 'negative'  when detik_rating == 0 then 'neutral' else 'null' end );
2016-09-01 05:11:38,883 [main] WARN  org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_LONG 21 time(s).
grunt> STORE bb INTO 'default.finaluniqlo' using org.apache.hive.hcatalog.pig.HCatStorer();
2016-09-01 05:11:59,047 [main] WARN  org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_LONG 21 time(s).
2016-09-01 05:11:59,339 [main] WARN  org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_LONG 3 time(s).
2016-09-01 05:11:59,390 [main] INFO  hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-09-01 05:11:59,393 [main] INFO  hive.metastore - Connected to metastore.
2016-09-01 05:11:59,608 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1115: Column name for a field is not specified. Please provide the full schema as an argument to HCatStorer.
Details at logfile: /root/pig_1472704532436.log

Re: I am getting this error to load the pig ouput into hive table how i can solve??? plz help me

Former Member — Thu, 01 Sep 2016 12:20:04 GMT

2016-09-01 05:11:59,608 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1115: Column name for a field is not specified. Please provide the full schema as an argument to HCatStorer. Details at logfile: /root/pig_1472704532436.log

grunt> STORE aa INTO 'default.finaluniqlo' using org.apache.hive.hcatalog.pig.HCatStorer(); 2016-09-01 05:15:41,910 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_LONG 21 time(s). 2016-09-01 05:15:42,063 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_LONG 3 time(s). 2016-09-01 05:15:42,177 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083 2016-09-01 05:15:42,181 [main] INFO hive.metastore - Connected to metastore. 2016-09-01 05:15:42,428 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1115: Column name for a field is not specified. Please provide the full schema as an argument to HCatStorer. Details at logfile: /root/pig_1472704532436.log

grunt> describe aa; 2016-09-01 05:16:07,581 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_LONG 3 time(s).

aa: {group: (a: chararray,b: chararray,c: chararray,d: chararray,e: chararray,f: chararray,g: chararray,score: int),chararray}

how to define schemma and how to load into hcatstore

Re: I am getting this error to load the pig ouput into hive table how i can solve??? plz help me

pminovic — Thu, 01 Sep 2016 14:02:36 GMT

Give names to the fields of aa, like for example, ("as var1", "as var2" added):

aa = foreach avg_rate generate $0 as var1, (case when detik_rating > 0 then 'positive' when detik_rating < 0 then 'negative' when detik_rating == 0 then 'neutral' else 'null' end) as var2;

Re: I am getting this error to load the pig ouput into hive table how i can solve??? plz help me

Former Member — Thu, 01 Sep 2016 20:30:21 GMT

thanks u so much

Re: I am getting this error to load the pig ouput into hive table how i can solve??? plz help me

Former Member — Thu, 01 Sep 2016 20:58:36 GMT

grunt> aa = foreach avg_rate generate as var1,( case when starbux_rating > 0 then 'positive' when starbux_rating < 0 then 'negative' when starbux_rating == 0 then 'neutral' else 'null' end ) as var2;

2016-09-01 13:51:33,113 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <line 55, column 34> mismatched input 'var1' expecting LEFT_PAREN Details at logfile: /root/pig_1472733213934.log

grunt> aa = foreach avg_rate generate $0,( case when starbux_rating > 0 then 'positive' when starbux_rating < 0 then 'negative' when starbux_rating == 0 then 'neutral' else 'null' end ) as var1; 2016-09-01 13:52:09,677 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_LONG 27 time(s).

grunt> describe aa; 2016-09-01 13:52:22,751 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_LONG 3 time(s). aa: {group: (a: chararray,b: chararray,c: chararray,d: chararray,e: chararray,f: chararray,g: chararray,h: chararray,i: chararray,j: chararray,k: chararray,l: chararray,m: chararray,n: chararray,o: chararray,p: chararray),var1: chararray}

grunt> c = foreach aa generate FLATTEN(aa);

2016-09-01 13:53:54,997 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: Pig script failed to parse: <line 56, column 32> Invalid scalar projection: aa : A column needs to be projected from a relation for it to be used as a scalar Details at logfile: /root/pig_1472733213934.log

grunt> c = foreach aa generate FLATTEN(var1);

while am flatten group comma also eliminated in front of var2

how to eliminate?????

Re: I am getting this error to load the pig ouput into hive table how i can solve??? plz help me

pminovic — Thu, 01 Sep 2016 22:09:54 GMT

You omitted $0: aa =foreach avg_rate generate $0 as var1,... and then try to store using HCatalog per your original question.

Re: I am getting this error to load the pig ouput into hive table how i can solve??? plz help me

pminovic — Fri, 02 Sep 2016 16:06:02 GMT

Hi @swathi thukkaraju, I see that you are using this solution in another question, so I guess it worked. If so, can you please accept & up-vote my answer to help us manage resolved questions. Thanks!

Re: I am getting this error to load the pig ouput into hive table how i can solve??? plz help me

Former Member — Fri, 02 Sep 2016 19:16:15 GMT

it is getting type cast error scalar