We are using the Hortonworks Apache Spark - Apache HBase Connector (SHC) in quite a few projects and we are planning to migrate to Scala 2.12 and Spark 3.x but from what I can see the only available versions the connector are using Scala 2.11 (and Spark 2.4.x). I's also a bit puzzled about the release process, source code, etc. The shc connector seems to have a public repository: https://github.com/hortonworks-spark/shc There are few branches of it (all with no updates since 2018) and quite a few forks. The master version is based on Scala 2.11, and should support Hbase 2.x and Spark 2.4.x (as long as is compiled with 2.11 but note that Scala 2.11 is deprecated since Spark 2.4.1 and won't be supported in Spark 3). From the git history I can see that the repo was under active development till 2018, then, in the last two years, there are only two commits (last one at the beginning of 2020). On the other hand the list of issues keep growing with some user wandering whether the repo is still actively maintained or not (ie. https://github.com/hortonworks-spark/shc/issues/328) and some other asking clarification about a possible update to Scala 2.12 (ie. https://github.com/hortonworks-spark/shc/issues/310 and https://github.com/hortonworks-spark/shc/issues/341) What puzzle me is also that there are artifacts of shc being published on maven repository quite regularly (see https://mvnrepository.com/artifact/com.hortonworks.shc/shc-core). They are still compiled with Scala 2.11 but they are released quite regularly (once a month I would say with the latest from October 2020). Unfortunately is not clear where the source code they are based upon sits (is it still an open source project?). Besides, those artifacts in maven have a Maven group id "com.hortonworks.shc" whereas the one in the public git repository has a group id "com.hortonworks". So, to summarize, can anyone please help with the following questions: 1) clarify whether the public git repository is still maintained? 2) tell me where I can find the source code for the artifacts available on Maven central repository? 3) info about a possible release a version of the connector based on Scala 2.12 and Spark 3? Thanks in advance.
... View more