Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

PIF HBase UDF-java.lang.UnsatisfiedLinkError: Cannot load libcrypto.so / java.io.IOException: Failed

PIF HBase UDF-java.lang.UnsatisfiedLinkError: Cannot load libcrypto.so / java.io.IOException: Failed

Champion Alumni

Hello,

 

I created an UDF for pig for writing into HBse; However, when I run my job , the job finishes successfully, there is nothing written into my HBase table and the logs I have 2 errors:

  java.io.IOException: Failed on local exception: java.io.EOFException; 
  Host Details : local host is: "ip"; destination host is: "ip":59856;
  at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
  at org.apache.hadoop.ipc.Client.call(Client.java:1415)
  at org.apache.hadoop.ipc.Client.call(Client.java:1364)
  at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
  at com.sun.proxy.$Proxy29.getJobReport(Unknown Source)
  at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)
  at sun.reflect.GeneratedMethodAccessor288.invoke(Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:320)
  at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:419)
  at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:553)
  at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:183)
  at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:582)
  at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:580)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
  at org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:580)
  at org.apache.hadoop.mapred.JobClient.getTaskReports(JobClient.java:635)
  at org.apache.hadoop.mapred.JobClient.getMapTaskReports(JobClient.java:629)
  at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:150)
  at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:468)
  at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
  at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
  at org.apache.pig.PigServer.execute(PigServer.java:1297)
  at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
  at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
  at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
  at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
  at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
  at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
  at org.apache.pig.Main.run(Main.java:478)
  at org.apache.pig.PigRunner.run(PigRunner.java:49)
  at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:286)
  at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:226)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
  at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:74)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
  Caused by: java.io.EOFException
  at java.io.DataInputStream.readInt(DataInputStream.java:392)
  at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1055)
  at org.apache.hadoop.ipc.Client$Connection.run(Client.java:950)
  
  
  ailed to load OpenSSL Cipher.
  java.lang.UnsatisfiedLinkError: Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)!
  at org.apache.hadoop.crypto.OpensslCipher.initIDs(Native Method)
  at org.apache.hadoop.crypto.OpensslCipher.<clinit>(OpensslCipher.java:89)
  at org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec.<init>(OpensslAesCtrCryptoCodec.java:50)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
  at org.apache.hadoop.crypto.CryptoCodec.getInstance(CryptoCodec.java:67)
  at org.apache.hadoop.crypto.CryptoCodec.getInstance(CryptoCodec.java:100)
  at org.apache.hadoop.fs.Hdfs.<init>(Hdfs.java:91)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  at org.apache.hadoop.fs.AbstractFileSystem.newInstance(AbstractFileSystem.java:129)
  at org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:157)
  at org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:242)
  at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:334)
  at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:331)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
  at org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331)
  at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:448)
  at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
  at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:137)
  at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:121)
  at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:111)
  at org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
  at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
  at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
  at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
  at org.apache.hadoop.mapred.JobClient.init(JobClient.java:472)
  at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:450)
  at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:158)
  at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
  at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
  at org.apache.pig.PigServer.execute(PigServer.java:1297)
  at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
  at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
  at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
  at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
  at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
  at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
  at org.apache.pig.Main.run(Main.java:478)
  at org.apache.pig.PigRunner.run(PigRunner.java:49)
  at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:286)
  at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:226)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
  at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:74)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
  2016-04-26 10:32:56,288 [main] DEBUG org.apache.hadoop.util.PerformanceAdvisory  - Crypto codec org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec is not available.
  2016-04-26 10:32:56,288 [main] DEBUG org.apache.hadoop.util.PerformanceAdvisory  - Using crypto codec org.apache.hadoop.crypto.JceAesCtrCryptoCodec.
  2016-04-26 10:32:56,290 [main] DEBUG org.apache.hadoop.mapreduce.Cluster  - Picked org.apache.hadoop.mapred.YarnClientProtocolProvider as the ClientProtocolProvider
  2016-04-26 10:32:56,321 [main] INFO  org.apache.pig.tools.pigstats.ScriptState  - Pig script settings are added to the job
  2016-04-26 10:32:56,805 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation  - mapred.job.reduce.markreset.buffer.percent is deprecated.
  
  
  
    java.net.ConnectException: Connection refused
  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
  at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529)
  at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493)
  at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:606)
  at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:700)
  at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:367)
  at org.apache.hadoop.ipc.Client.getConnection(Client.java:1463)
  at org.apache.hadoop.ipc.Client.call(Client.java:1382)
  at org.apache.hadoop.ipc.Client.call(Client.java:1364)
  at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
  at com.sun.proxy.$Proxy20.getApplicationReport(Unknown Source)
  at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getApplicationReport(ApplicationClientProtocolPBClientImpl.java:163)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
  at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
  at com.sun.proxy.$Proxy21.getApplicationReport(Unknown Source)
  at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplicationReport(YarnClientImpl.java:365)
  at org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:294)
  at org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:152)
  at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:319)
  at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:419)
  at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:553)
  at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:183)
  at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:582)
  at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:580)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
  at org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:580)
  at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:598)
  at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.progressOfRunningJob(Launcher.java:274)
  at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.calculateProgress(Launcher.java:257)
  at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:335)
  at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
  at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
  at org.apache.pig.PigServer.execute(PigServer.java:1297)
  at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
  at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
  at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
  at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
  at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
  at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
  at org.apache.pig.Main.run(Main.java:478)
  at org.apache.pig.PigRunner.run(PigRunner.java:49)
  at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:286)
  at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:226)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
  at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:74)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
  2016-04-26 10:33:05,395 [main] DEBUG org.apache.hadoop.ipc.Client  - IPC Client (1962449973) connection to ip/ip:8032 from agherman: closed
  2016-04-26 10:33:05,395 [main] TRACE org.apache.hadoop.ipc.ProtobufRpcEngine  - 1: Exception <- ip:8032: getApplicationReport {java.net.ConnectException: Call From ip/ip to ip:8032 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused}



I cannot manage therefore to find out the source of the problem...

 

Thank you in advance!

GHERMAN Alina
Don't have an account?
Coming from Hortonworks? Activate your account here