Errors on Lab 3

ttle — Fri, 30 Sep 2016 04:47:26 GMT

Script:

a = LOAD 'geolocation' using org.apache.hive.hcatalog.pig.HCatLoader();
b = filter a by event != 'normal';
c = foreach b generate driverid, event, (int) '1' as occurance;
d = group c by driverid;
e = foreach d generate group as driverid, SUM(c.occurance) as t_occ;
g = LOAD 'drivermileage' using org.apache.hive.hcatalog.pig.HCatLoader();
h = join e by driverid, g by driverid;
final_data = foreach h generate $0 as driverid, $1 as events, $3 as totmiles, (float) $3/$1 as riskfactor;
store final_data into 'riskfactor' using org.apache.hive.hcatalog.pig.HCatStorer();

Error:

 ls: cannot access /hadoop/yarn/local/usercache/maria_dev/appcache/application_1475181427748_0005/container_1475181427748_0005_01_000002/hive.tar.gz/hive/lib/slf4j-api-*.jar: No such file or directory 
 ls: cannot access /hadoop/yarn/local/usercache/maria_dev/appcache/application_1475181427748_0005/container_1475181427748_0005_01_000002/hive.tar.gz/hive/hcatalog/lib/*hbase-storage-handler-*.jar: No such file or directory 
 WARNING: Use "yarn jar" to launch YARN applications. 
 16/09/29 21:30:53 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 
 16/09/29 21:30:53 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 
 16/09/29 21:30:53 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL 
 16/09/29 21:30:53 INFO pig.ExecTypeProvider: Trying ExecType : TEZ 
 16/09/29 21:30:53 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType 
 2016-09-29 21:30:53,748 [main] INFO  org.apache.pig.Main - Apache Pig version 0.16.0.2.5.0.0-1245 (rexported) compiled Aug 26 2016, 02:07:35 
 2016-09-29 21:30:53,748 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1475181427748_0005/container_1475181427748_0005_01_000002/pig_1475184653746.log 
 2016-09-29 21:30:54,859 [main] INFO  org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found 
 2016-09-29 21:30:55,011 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020 
 2016-09-29 21:30:56,136 [main] INFO  org.apache.pig.PigServer - Pig Script ID for the session: PIG-script.pig-720ef8a5-85e0-4929-abc5-514885b28262 
 2016-09-29 21:30:56,680 [main] INFO  org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 
 2016-09-29 21:30:56,852 [main] INFO  org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook 
 2016-09-29 21:30:57,964 [main] WARN  org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist 
 2016-09-29 21:30:58,026 [main] INFO  hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083 
 2016-09-29 21:30:58,101 [main] INFO  hive.metastore - Connected to metastore. 
 2016-09-29 21:30:58,731 [main] WARN  org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist 
 2016-09-29 21:30:58,733 [main] INFO  hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083 
 2016-09-29 21:30:58,734 [main] INFO  hive.metastore - Connected to metastore. 
 2016-09-29 21:30:58,983 [main] WARN  org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_FLOAT 1 time(s). 
 2016-09-29 21:30:59,098 [main] WARN  org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist 
 2016-09-29 21:30:59,143 [main] INFO  hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083 
 2016-09-29 21:30:59,145 [main] INFO  hive.metastore - Connected to metastore. 
 2016-09-29 21:30:59,569 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 0:  
  Output Location Validation Failed for: 'riskfactor More info to follow: 
 Pig 'double' type in column 2(0-based) cannot map to HCat 'BIGINT'type.  Target filed must be of HCat type {DOUBLE} 
 Details at logfile: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1475181427748_0005/container_1475181427748_0005_01_000002/pig_1475184653746.log 
 2016-09-29 21:30:59,589 [main] INFO  org.apache.pig.Main - Pig script completed in 6 seconds and

Re: Errors on Lab 3

pminovic — Fri, 30 Sep 2016 09:20:16 GMT

It seems that totmiles is double, while the 3rd field in table 'riskfactor' is BIGINT. As the erros says "Target field must be of HCat type {DOUBLE}".

Re: Errors on Lab 3

ttle — Fri, 30 Sep 2016 23:49:23 GMT

Thanks. That was the issue.

question Re: Errors on Lab 3 in Archives of Support Questions (Read Only)

Errors on Lab 3

Re: Errors on Lab 3

Re: Errors on Lab 3