question Re: Managing Hive Schema Changes in Support Questions

Managing Hive Schema Changes

Ging — Fri, 18 Mar 2022 12:45:01 GMT

We are planning to automate our hive schema changes as part of our CI/CD deployment. Currently we are using Gitlab to manage our repo and execute deployment pipeline. One of the tools we're considering is Liquibase. This blog detailed out the way to implement this https://docs.liquibase.com/install/tutorials/impala-hive.html. It explains that in order to use Liquibase with Hive, it needs an impala/hive extensions. This approach seems promising. However I'm concerned that the extension is outdated and may impose security risk in our system since the last update of it was 5 years ago (https://github.com/eselyavka/liquibase-impala).

I was wondering if anyone here have the same use case. What approach/options did you choose to implement the solution? I appreciate any comments or suggestion from this groups.

Thank you!

Re: Managing Hive Schema Changes

smruti — Thu, 24 Mar 2022 19:16:36 GMT

@Ging I don't think there is much in the liquibase extension that could impose security risks. But, it's better to check with Liquibase. About Hive and Impala JDBC drivers, you could download the latest from Cloudera website, and not 2.6.4/2.6.2 as mentioned in the Liquibase blog. Very soon we are going to release newer versions that address the recent log4j vulnerabilities.

Re: Managing Hive Schema Changes

VidyaSargur — Mon, 28 Mar 2022 05:55:41 GMT

@Ging, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.

Re: Managing Hive Schema Changes

Ging — Sun, 03 Apr 2022 12:11:18 GMT

Thank you. We contacted Liquibase. Unfortunately, they don't have a built-in extension of Hive/Impala extension. Though they promise to look into this requirement in more detail. Hopefully, they come-up with a solution soon.