Pig install and configuration

[#1]

Hi Team,

When I try to run pig to get to grunt shell, error I am getting is as follows:

16/08/04 02:33:40 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 16/08/04 02:33:40 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 16/08/04 02:33:40 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType 2016-08-04 02:33:40,231 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0 (r1746530) compiled Jun 01 2016, 23:10:49 2016-08-04 02:33:40,232 [main] INFO org.apache.pig.Main - Logging error messages to: /usr/local/hadoop/etc/hadoop/pig_1470278020218.log 2016-08-04 02:33:40,313 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/hadoop/.pigbootup not found 2016-08-04 02:33:40,829 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://172.31.24.167:8020 2016-08-04 02:33:41,072 [main] INFO org.apache.pig.Main - Pig script completed in 1 second and 283 milliseconds (1283 ms) Exception in thread "main" java.lang.IncompatibleClassChangeError: Implementing class at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:803) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at org.apache.pig.tools.pigstats.PigStatsUtil.<clinit>(PigStatsUtil.java:73) at org.apache.pig.Main.run(Main.java:665) at org.apache.pig.Main.main(Main.java:176) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

My bashrc is as follows:

export PIG_HOME=/usr/local/hadoop/pig export PATH=$PATH:$PIG_HOME/bin export PIG_CLASSPATH=/usr/local/hadoop/etc/Hadoop

Environment:

Ubuntu 14.04

pig-0.16.0

jps shows

3404 DataNode 5114 Jps 4569 ResourceManager 4719 NodeManager 3247 NameNode 3634 SecondaryNameNode

Appreciate any pointers I am missing.

Regards

Anil Khiani

[#2]

@Anil Khiani

Can you please check export PIG_CLASSPATH=$HADOOP_HOME/conf/

[#3 · Accepted solution]

@Anil Khiani what version of HDP are you using? can you verify you have pig client installed on node you are trying to execute pig grunt shell on?

[#4]

Thanks Sunile. It was indeed pig version. I installed Apache Pig version 0.9.2 (r1232772) and it got resolved.

Cheers - Anil

[#5]

@Anil Khiani Glad it worked. best of luck

[#6]

Hi Sunile/SBandaru,

I checked with pig -x local and got the grunt shell to LOAD, DUMP and all pig scripts work fine. I tried running pig today (without -x local) to connect to hdfs using hadoop conf, it throws following exception

ERROR 2999: Unexpected internal error. Failed to create DataStorage

and

Server IPC version 9 cannot communicate with client version 4

Appreciate all your help/support in advance.

Regards

Anil Khiani