https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144524#M107106 <P><A rel="user" href="https://community.cloudera.com/users/14797/frulouis.html" nodeid="14797">@Fru N.</A> you are trying to load /tmp/demo/piglab04/data/drivers.tsv but your file has a csv extension. Change from tsv to csv. It should work. </P> Fri, 09 Dec 2016 03:08:10 GMT mrizvi 2016-12-09T03:08:10Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144514#M107096 <P>I can't run pig on Hortonworks Sandbox in Azure. Keep getting this errors.</P><PRE>Input(s): Failed to read data from "hdfs://sanbox.hortonworks.com:8020/tmp/demo/data/drivers.tsv" </PRE><P>Any ideas what the issue could be? I can't seem to get this to work. </P> Fri, 09 Dec 2016 01:36:39 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144514#M107096 fru_louis 2016-12-09T01:36:39Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144515#M107097 <P><A rel="user" href="https://community.cloudera.com/users/14797/frulouis.html" nodeid="14797">@Fru N.</A> can you check whether the file is present at the specified location, and if yes, what are the permissions for that file?</P> Fri, 09 Dec 2016 01:39:05 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144515#M107097 mrizvi 2016-12-09T01:39:05Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144516#M107098 <P>Yes.. I checked. And file is present. I gave 777 permission.</P><P>SInce it's Azure, not sure if there are additional settings in need to change with my hdfs-site.xml</P> Fri, 09 Dec 2016 01:59:28 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144516#M107098 fru_louis 2016-12-09T01:59:28Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144517#M107099 <P><A rel="user" href="https://community.cloudera.com/users/14797/frulouis.html" nodeid="14797">@Fru N.</A>, you do not have to change anything in hdfs-site.xml. Can you please send entire job log.</P> Fri, 09 Dec 2016 02:02:57 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144517#M107099 mrizvi 2016-12-09T02:02:57Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144518#M107100 <P><A href="https://community.hortonworks.com/questions/70584/pig-on-hortonworks-sandbox-in-azure.html#">@Mushtaq Rizvi</A></P><P>grunt&gt; drivers = LOAD '/tmp/demo/piglab04/data/drivers.tsv' USING PigStorage(','); </P><PRE>grunt&gt; dump drivers; 2016-12-08 18:44:08,284 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 2016-12-08 18:44:08,386 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code. 2016-12-08 18:44:08,451 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstPara llelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, St reamTypeCastInserter]} 2016-12-08 18:44:08,611 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/fnhdp/staging and resources directory is /tmp/temp101 1891030 2016-12-08 18:44:08,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: hdfs://sandbox.hortonworks.com:8020/tmp/demo/piglab04/data/drivers.tsv at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:279) at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateNewSplits(MRInputHelpers.java:412) at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:291) at org.apache.pig.backend.hadoop.executionengine.tez.plan.optimizer.LoaderProcessor.processLoads(LoaderProcessor.java:171) at org.apache.pig.backend.hadoop.executionengine.tez.plan.optimizer.LoaderProcessor.visitTezOp(LoaderProcessor.java:183) at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:249) at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:55) at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:87) at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:46) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher.processLoadAndParallelism(TezLauncher.java:461) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher.launchPig(TezLauncher.java:171) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:304) at org.apache.pig.PigServer.launchPlan(PigServer.java:1431) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1416) at org.apache.pig.PigServer.storeEx(PigServer.java:1075) at org.apache.pig.PigServer.store(PigServer.java:1038) at org.apache.pig.PigServer.openIterator(PigServer.java:951) at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:754) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:376) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66) at org.apache.pig.Main.run(Main.java:565) at org.apache.pig.Main.main(Main.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://sandbox.hortonworks.com:8020/tmp/demo/piglab04/data/drivers.tsv at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323) at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36) at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:265) ... 29 more 2016-12-08 18:44:08,981 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2118: Input path does not exist: hdfs://sandbox.hortonworks.com:8020/tmp/demo/piglab04/data/drivers.tsv </PRE><P>Details at logfile: /home/fnhdp/pig_1481222529518.log </P> Fri, 09 Dec 2016 02:46:31 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144518#M107100 fru_louis 2016-12-09T02:46:31Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144519#M107101 <P><A href="https://community.hortonworks.com/questions/70584/pig-on-hortonworks-sandbox-in-azure.html#">@Mushtaq Rizvi</A> -Note: The file does exists. I have confirmed that. </P><P>Causedby: org.apache.hadoop.mapreduce.lib.input.InvalidInputException:Input path does not exist: hdfs://sandbox.hortonworks.com:8020/tmp/demo/piglab04/data/drivers.tsv </P> Fri, 09 Dec 2016 02:48:08 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144519#M107101 fru_louis 2016-12-09T02:48:08Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144520#M107102 <P>it says that the input path does not exist, can you also show the output of this command:</P><PRE>hdfs dfs -ls /tmp/demo/piglab04/data/</PRE> Fri, 09 Dec 2016 02:50:03 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144520#M107102 mrizvi 2016-12-09T02:50:03Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144521#M107103 <PRE>hadoop fs -ls hdfs://sandbox.hortonworks.com:8020/tmp/demo/piglab04/data Found 3 items -rwxrwxrwx 3 maria_dev hdfs 2043 2016-12-08 16:05 hdfs://sandbox.hortonworks.com:8020/tmp/demo/piglab04/data/drivers.csv -rwxrwxrwx 3 maria_dev hdfs 26205 2016-12-08 16:05 hdfs://sandbox.hortonworks.com:8020/tmp/demo/piglab04/data/timesheet.csv -rwxrwxrwx 3 maria_dev hdfs 2272077 2016-12-08 16:05 hdfs://sandbox.hortonworks.com:8020/tmp/demo/piglab04/data/truck_event_text_partition.csv </PRE> Fri, 09 Dec 2016 02:58:20 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144521#M107103 fru_louis 2016-12-09T02:58:20Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144522#M107104 <PRE>hdfs dfs -ls /tmp/demo/piglab04/data/ Found 3 items -rwxrwxrwx 3 maria_dev hdfs 2043 2016-12-08 16:05 /tmp/demo/piglab04/data/drivers.csv -rwxrwxrwx 3 maria_dev hdfs 26205 2016-12-08 16:05 /tmp/demo/piglab04/data/timesheet.csv -rwxrwxrwx 3 maria_dev hdfs 2272077 2016-12-08 16:05 /tmp/demo/piglab04/data/truck_event_text_partition.csv </PRE> Fri, 09 Dec 2016 03:00:21 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144522#M107104 fru_louis 2016-12-09T03:00:21Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144523#M107105 <P>For reference, i'm using the Microsoft Azure: Hortonworks Sandbox with HDP 2.4</P> Fri, 09 Dec 2016 03:07:38 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144523#M107105 fru_louis 2016-12-09T03:07:38Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144524#M107106 <P><A rel="user" href="https://community.cloudera.com/users/14797/frulouis.html" nodeid="14797">@Fru N.</A> you are trying to load /tmp/demo/piglab04/data/drivers.tsv but your file has a csv extension. Change from tsv to csv. It should work. </P> Fri, 09 Dec 2016 03:08:10 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144524#M107106 mrizvi 2016-12-09T03:08:10Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144525#M107107 <P>From Grunt, I get this...</P><PRE>grunt&gt; fs -ls /tmp/demo/piglab04/data Found 3 items -rwxrwxrwx 3 maria_dev hdfs 2043 2016-12-08 16:05 /tmp/demo/piglab04/data/drivers.csv -rwxrwxrwx 3 maria_dev hdfs 26205 2016-12-08 16:05 /tmp/demo/piglab04/data/timesheet.csv -rwxrwxrwx 3 maria_dev hdfs 2272077 2016-12-08 16:05 /tmp/demo/piglab04/data/truck_event_text_partition.csv </PRE><P>So, not really sure why it happens only when running pig job. Suspecting it's something to do with the Sandbox image.</P> Fri, 09 Dec 2016 03:10:48 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144525#M107107 fru_louis 2016-12-09T03:10:48Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144526#M107108 <P>That it is! Worked like a charm. Thanks <A href="https://community.hortonworks.com/questions/70584/pig-on-hortonworks-sandbox-in-azure.html#">@Mushtaq Rizvi</A></P> Fri, 09 Dec 2016 03:18:34 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144526#M107108 fru_louis 2016-12-09T03:18:34Z https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144527#M107109 <P>Glad to hear that, please accept the best answer to close the thread</P> Fri, 09 Dec 2016 04:20:51 GMT https://community.cloudera.com/t5/Support-Questions/Pig-on-Hortonworks-Sandbox-In-Azure/m-p/144527#M107109 mrizvi 2016-12-09T04:20:51Z