Member since
12-10-2015
36
Posts
9
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3915 | 12-09-2017 07:12 PM | |
5057 | 02-02-2016 08:34 AM |
02-02-2016
08:23 PM
How do I ensure this?
... View more
02-02-2016
07:41 PM
the mchine is a2 I still have problems of crashing. And i just restart the machine once a day i
... View more
02-02-2016
08:34 AM
1 Kudo
Hi, I managed to solve it There are 2 things need to be done for this to work: First add few jars in the libjars and second add hbase lib to hadoop class path and then it should work. Here are the libjars needed hbase-client.jar,hbase-server.jar,hbase-protocol.jar,hbase-common.jar,htrace-core-2.00.jar pay attention to htrace-core-2.00.jar which is Cloudera jar and I am using hortonworks sandbox 2.2.4.2-2 and need to add the hbase lib to hadoop-env.sh otherwise you wont be able to run the main thread Thank you for your help hope it helps!!
... View more
02-01-2016
02:36 PM
Do you mean these files? hbase-client.jar hadoop-mapreduce-client-core.jar
... View more
02-01-2016
02:27 PM
It is almost the same exception from the same package: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/protobuf/generated/ClientProtos
... View more
02-01-2016
02:25 PM
OK, I tried the examples and I get the same exception I think it means that the problem is with the deployment.. I don't have a pom I just create a jar and put it with relevant jars and then run it with hadoop jar
... View more
02-01-2016
01:51 PM
Here is my code for the driver for mapreduce : public int run(String[] args) throws Exception {
// TODO Auto-generated method stub
if (args.length!=1){
System.out.println("usage: [input]");
System.exit(-1);
}
Job job = Job.getInstance(getConf());
job.setJarByClass(HBaseDriver.class);
FileInputFormat.addInputPath(job
, new Path(args[0]));
job.getConfiguration().set(TableOutputFormat.OUTPUT_TABLE, "students");
job.setInputFormatClass(StudentInputFormat.class);
job.setMapperClass(HBaseStudentsMapper.class);
job.setOutputFormatClass(TableOutputFormat.class);
job.setNumReduceTasks(0);
returnjob.waitForCompletion(true)? 0 :1;
}
Looks like the system fails on this line : job.setOutputFormatClass(TableOutputFormat.class); the message is: Exception in thread "main" java.lang.NoClassDefFoundError: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/protobuf/generated/MasterProtos$MasterService$BlockingInterface The interesting thing is that if I comment this line and do some Hbase stuff in the mapper it works. More over I have he hbase-protocol library and I supply it with my libjars command here is how I run the map reduce: hadoop jar mo.jar HBaseDriver -libjars hbase-client.jar,hbase-common.jar,hbase-protocol.jar,htrace-core-2.00.jar,hbase-server.jar input
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache HBase
12-10-2015
09:05 PM
Logs from master pasted part of it but these are the exceptions I have
... View more
12-10-2015
09:03 PM
2015-12-10 16:55:39,949 ERROR [Thread-74] util.PolicyRefresher: PolicyRefresher(serviceName=sandbox_hbase): failed to refresh policies. Will continue to use last known version of policies (4) com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) at com.sun.jersey.api.client.Client.handle(Client.java:648) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503) at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:71) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175) at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154) Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.<init>(HttpClient.java:211) at sun.net.www.http.HttpClient.New(HttpClient.java:308) at sun.net.www.http.HttpClient.New(HttpClient.java:326) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:997) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:933) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:851) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1301) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:240) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147) ... 8 more 2015-12-10 16:55:44,547 INFO [timeline] httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection refused 2015-12-10 16:55:44,547 INFO [timeline] httpclient.HttpMethodDirector: Retrying request 2015-12-10 16:55:44,548 INFO [timeline] httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection refused 2015-12-10 16:55:44,548 INFO [timeline] httpclient.HttpMethodDirector: Retrying request 2015-12-10 16:55:44,548 INFO [timeline] httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection refused 2015-12-10 16:55:44,548 INFO [timeline] httpclient.HttpMethodDirector: Retrying request 2015-12-10 16:55:44,548 WARN [timeline] timeline.HadoopTimelineMetricsSink: Unable to send metrics to collector by address:http://sandbox.hortonworks.com:6188/ws/v1/timeline/metrics
... View more
- « Previous
-
- 1
- 2
- Next »