03-23-2017 10:59 PM
Previously we were using habse-testing-util 1.2.3 and phoenix 4.6, phoenix 4.6 was compiled with 1.2.0 version of hbase. All the unit tests were running fine. Hbase mini cluster is being started with the following code:
// Start HbaseMiniCluster for Unit testing val hbaseTestUtil = new HBaseTestingUtility() // Setting a fixed port for Zookeeper since Hbasetestingutility assigns random port everytime hbaseTestUtil.getConfiguration.set("test.hbase.zoo
keeper.property.clientPort", "2181") // Start the cluster hbaseTestUtil.startMiniCluster
We recently shifted to cloudera based installation of version CDH-5.10.0. We built phoenix with 1.2.0-cdh5.10.0 version of hbase and 2.6.0-cdh5.10.0 version of hadoop and we are using manual rpm based cloudera installation. All our jobs running fine on this cloudera based installation and newly buit phoenix (jobs write data to hbase using phoenix). Issue here is running unit tests. We changed the version of hbase-testing-utility dependency to 1.2.0-cdh5.10.0. Units tests compiled successfully with this version but getting following exception at runtime:
log4j:WARN No appenders could be found for logger (org.apache.hadoop.hbase.HBaseCommonTestingUtility
). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#nocon fig for more info. Formatting using clusterid: testClusterID *** RUN ABORTED *** java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Put.setWriteToWAL(Z )V at org.apache.phoenix.schema.PTableImpl$PRowImpl.newM utations(PTableImpl.java:700) at org.apache.phoenix.schema.PTableImpl$PRowImpl.<ini t>(PTableImpl.java:693) at org.apache.phoenix.schema.PTableImpl.newRow(PTable Impl.java:618) at org.apache.phoenix.schema.PTableImpl.newRow(PTable Impl.java:634) at org.apache.phoenix.execute.MutationState.addRowMut ations(MutationState.java:252) at org.apache.phoenix.execute.MutationState.access$20 0(MutationState.java:91) at org.apache.phoenix.execute.MutationState$2.init(Mu tationState.java:325) at org.apache.phoenix.execute.MutationState$2.<init>( MutationState.java:322) at org.apache.phoenix.execute.MutationState.toMutatio ns(MutationState.java:320) at org.apache.phoenix.execute.MutationState.toMutatio ns(MutationState.java:310) ... [INFO] -------------------------------------------------- ----------------------
So I have couple of questions here:
1. Is this the write way of starting hbase mini cluster and unit tests as given in above code blocks for CDH-5.10.0?
2. I checked org.apache.hadoop.hbase.client.Put class in hbase-client-1.2.0-cdh5.10.0 jar and it doesn't have method setWriteToWAL(), so how phoenix was compiled successfully with 1.2.0-cdh5.10.0 version of hbase?
03-24-2017 06:59 AM
Apache Phoenix is tied in to specific versions of HBase. Due to maintaining compatibility across CDH5 releases, version 1.2.0-cdh5.10.0 has some api incompatibilities with Apache HBase 1.2.0. Unfortunately, some of these API differences are ones that matter to Phoenix. Generally, this means some small code changes are needed to get Phoenix to build against CDH jars.
Note that Phoenix on CDH is not something Cloudera supports. Still, you can take a look at the Cloudera Labs git repo for what kind of changes are needed. We never made a Phoenix 4.6 build, IIRC. But we did make a Phoenix 4.7