Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Compatibility of HBase with HDFS - Erasure Coding (EC)

avatar
New Contributor

Some hdfs operations (hsync, append, ... ) are not supported with the erasure coding (ec) policy. 

Does hbase work well when the data folders of hbase are set to the erasure coding policy?

Is Hbase compatible with HDFS-EC instead of HDFS-replica ?

 

 

 

1 ACCEPTED SOLUTION

avatar
Super Collaborator

HI @bgkim, Hbase won't work well with data folders of hbase set to EC policy. EC did not implement hflush which is required to not lose data. Additionally, the Data locality is also killed with EC which makes it slower.

 

https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/cdppvc-data-migration-opdb/topics/cdppvc-...

 

--
Was your question answered? Please take some time to click on “Accept as Solution” below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

View solution in original post

2 REPLIES 2

avatar
Super Collaborator

HI @bgkim, Hbase won't work well with data folders of hbase set to EC policy. EC did not implement hflush which is required to not lose data. Additionally, the Data locality is also killed with EC which makes it slower.

 

https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/cdppvc-data-migration-opdb/topics/cdppvc-...

 

--
Was your question answered? Please take some time to click on “Accept as Solution” below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
New Contributor

Is anyone aware if there are any plans for hflush implementation in EC in future versions?