Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

I am facing problem in pig mapreduce, i have inatalled pig -0.16.0 and hadoop-1.2.1 but during starting pig -x mapreduce , following error occurs , please suggest me what is wrong.

avatar
New Contributor

hduser@atl-Latitude-E6410:~$ cd /usr/lib/pig/pig-0.16.0/ hduser@atl-Latitude-E6410:/usr/lib/pig/pig-0.16.0$ bin/pig -x mapreduce 16/07/03 16:07:37 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 16/07/03 16:07:37 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 16/07/03 16:07:37 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType 16/07/03 16:07:37 WARN pig.Main: Cannot write to log file: /usr/lib/pig/pig-0.16.0/pig_1467542257885.log 2016-07-03 16:07:37,890 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0 (r1746530) compiled Jun 01 2016, 23:09:59 2016-07-03 16:07:37,950 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/hduser/.pigbootup not found 2016-07-03 16:07:38,089 [main] ERROR org.apache.pig.Main - ERROR 4010: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath). If you plan to use local mode, please put -x local option in command line 2016-07-03 16:07:38,090 [main] WARN org.apache.pig.Main - There is no log file to write to. 2016-07-03 16:07:38,090 [main] ERROR org.apache.pig.Main - org.apache.pig.backend.executionengine.ExecException: ERROR 4010: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath). If you plan to use local mode, please put -x local option in command line at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.getExecConf(HExecutionEngine.java:158) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:195) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112) at org.apache.pig.impl.PigContext.connect(PigContext.java:305) at org.apache.pig.PigServer.<init>(PigServer.java:231) at org.apache.pig.PigServer.<init>(PigServer.java:219) at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:46) at org.apache.pig.Main.run(Main.java:561) at org.apache.pig.Main.main(Main.java:176)

2016-07-03 16:07:38,175 [main] INFO org.apache.pig.Main - Pig script completed in 614 milliseconds (614 ms)

1 ACCEPTED SOLUTION

avatar
Contributor

The problem here is that you are running Pig in MapReduce mode, which requires the necessary hadoop jars set in your classpath.

You have two options over here-

1. Set the Hadoop Classpath by using the below command :-

Assuming you are seting this from the same folder as you are building your code with Maven and have put all your 3rd paty JARs in target/libs...

Command -> export HADOOP_CLASSPATH=./target/classes:./target/libs/*

2. Run in Pig in Local mode - This mode allows you to run in Local mode, you need access to a single machine; all files are installed and run using your local host and local file system.

Command -> pig -x local

Hope this helps 🙂

View solution in original post

3 REPLIES 3

avatar
Contributor

The problem here is that you are running Pig in MapReduce mode, which requires the necessary hadoop jars set in your classpath.

You have two options over here-

1. Set the Hadoop Classpath by using the below command :-

Assuming you are seting this from the same folder as you are building your code with Maven and have put all your 3rd paty JARs in target/libs...

Command -> export HADOOP_CLASSPATH=./target/classes:./target/libs/*

2. Run in Pig in Local mode - This mode allows you to run in Local mode, you need access to a single machine; all files are installed and run using your local host and local file system.

Command -> pig -x local

Hope this helps 🙂

avatar
New Contributor

thank you Gautam..:) I got it.

avatar
Contributor

@sharda godara - Kindly accept the answer 🙂