Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Pig install and configuration

avatar
Explorer

Hi Team,

When I try to run pig to get to grunt shell, error I am getting is as follows:

16/08/04 02:33:40 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 16/08/04 02:33:40 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 16/08/04 02:33:40 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType 2016-08-04 02:33:40,231 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0 (r1746530) compiled Jun 01 2016, 23:10:49 2016-08-04 02:33:40,232 [main] INFO org.apache.pig.Main - Logging error messages to: /usr/local/hadoop/etc/hadoop/pig_1470278020218.log 2016-08-04 02:33:40,313 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/hadoop/.pigbootup not found 2016-08-04 02:33:40,829 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://172.31.24.167:8020 2016-08-04 02:33:41,072 [main] INFO org.apache.pig.Main - Pig script completed in 1 second and 283 milliseconds (1283 ms) Exception in thread "main" java.lang.IncompatibleClassChangeError: Implementing class at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:803) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at org.apache.pig.tools.pigstats.PigStatsUtil.<clinit>(PigStatsUtil.java:73) at org.apache.pig.Main.run(Main.java:665) at org.apache.pig.Main.main(Main.java:176) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

My bashrc is as follows:

export PIG_HOME=/usr/local/hadoop/pig export PATH=$PATH:$PIG_HOME/bin export PIG_CLASSPATH=/usr/local/hadoop/etc/Hadoop

Environment:

Ubuntu 14.04

pig-0.16.0

jps shows

3404 DataNode 5114 Jps 4569 ResourceManager 4719 NodeManager 3247 NameNode 3634 SecondaryNameNode

Appreciate any pointers I am missing.

Regards

Anil Khiani

1 ACCEPTED SOLUTION

avatar
Master Guru

@Anil Khiani what version of HDP are you using? can you verify you have pig client installed on node you are trying to execute pig grunt shell on?

View solution in original post

5 REPLIES 5

avatar
@Anil Khiani

Can you please check export PIG_CLASSPATH=$HADOOP_HOME/conf/

avatar
Master Guru

@Anil Khiani what version of HDP are you using? can you verify you have pig client installed on node you are trying to execute pig grunt shell on?

avatar
Explorer

Thanks Sunile. It was indeed pig version. I installed Apache Pig version 0.9.2 (r1232772) and it got resolved.

Cheers - Anil

avatar
Master Guru

@Anil Khiani Glad it worked. best of luck

avatar
Explorer

Hi Sunile/SBandaru,

I checked with pig -x local and got the grunt shell to LOAD, DUMP and all pig scripts work fine. I tried running pig today (without -x local) to connect to hdfs using hadoop conf, it throws following exception

ERROR 2999: Unexpected internal error. Failed to create DataStorage

and

Server IPC version 9 cannot communicate with client version 4

Appreciate all your help/support in advance.

Regards

Anil Khiani