Compatibility of HBase with HDFS - Erasure Coding (EC)

2 replies
2910 views
Translate All

Some hdfs operations (hsync, append, ... ) are not supported with the erasure coding (ec) policy.

Does hbase work well when the data folders of hbase are set to the erasure coding policy?

Is Hbase compatible with HDFS-EC instead of HDFS-replica ?

rki_Answer

Gold ⭐️

HI @bgkim, Hbase won't work well with data folders of hbase set to EC policy. EC did not implement hflush which is required to not lose data. Additionally, the Data locality is also killed with EC which makes it slower.

https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/cdppvc-data-migration-opdb/topics/cdppvc-data-migration-hbase-unsupported-interfaces-features.html

--
Was your question answered? Please take some time to click on “Accept as Solution” below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

rki_Answer

Gold ⭐️

HI @bgkim, Hbase won't work well with data folders of hbase set to EC policy. EC did not implement hflush which is required to not lose data. Additionally, the Data locality is also killed with EC which makes it slower.

https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/cdppvc-data-migration-opdb/topics/cdppvc-data-migration-hbase-unsupported-interfaces-features.html

--
Was your question answered? Please take some time to click on “Accept as Solution” below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Translate

nitishgoyal13

Silver ⭐️⭐️

Is anyone aware if there are any plans for hflush implementation in EC in future versions?

Translate

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.