Created on 07-03-2018 02:54 AM - edited 09-16-2022 06:24 AM
Hello ~
I'm looking for the solution of below symtoms.
Anyone who knows the solution ?
Thanks .
Environment : CDH 5.14 (HBase 1.2)
-- create test table create 'jkkim','f' -- put an sample data put 'jkkim','row2','f:name','jonggyun Kim2' -- put another sample data with TTL put 'jkkim','row4','f:name','jonggyun Kim',{TTL => 10000} -- scan after 15 seconds scan 'jkkim' -- row4 does not shown result (expected) ROW COLUMN+CELL row2 column=f:name, timestamp=1530609778641, value=jonggyun Kim2 1 row(s) in 0.0120 seconds --put sample data again with TTL put 'jkkim','row4','f:name','jonggyun Kim',{TTL => 10000} -- Run manually flush command flush 'jkkim' --scan after 15 seconds scan 'jkkim' -- row4 is shown result (not expected) ROW COLUMN+CELL row2 column=f:name, timestamp=1530609778641, value=jonggyun Kim2 row4 column=f:name, timestamp=1530609975940, value=jonggyun Kim
Created 07-03-2018 07:11 PM
Cell TTLs are a HFile V3 feature (as far as persistence goes).
CDH5 HBase uses HFile V2 by default for backward compatibility reasons with older CDH5 HBase versions. To persist features properly into HFiles, you must manually enable the HFile V3 feature.
You are likely missing the following configuration property:
<property> <name>hfile.format.version</name> <value>3</value> </property>
This must be manually added into both the CM fields noted below:
Note that enabling this will not let you rollback your new HFiles of V3 back to V2 in future. That said, we do have a lot of users on V3 running without any issues.
Created 07-03-2018 07:11 PM
Cell TTLs are a HFile V3 feature (as far as persistence goes).
CDH5 HBase uses HFile V2 by default for backward compatibility reasons with older CDH5 HBase versions. To persist features properly into HFiles, you must manually enable the HFile V3 feature.
You are likely missing the following configuration property:
<property> <name>hfile.format.version</name> <value>3</value> </property>
This must be manually added into both the CM fields noted below:
Note that enabling this will not let you rollback your new HFiles of V3 back to V2 in future. That said, we do have a lot of users on V3 running without any issues.
Created 07-03-2018 07:43 PM
Dear Harsh J
Thank you very much.
Very helpful.
One more questions about changing hfile format.
After change hfile format version, is it rewrite all of hfile to new version format?
Or Just be applied only newly created hfile?
Thanks.
Created on 07-03-2018 08:15 PM - edited 07-03-2018 08:29 PM
Immediately after the config change and restart, the existing HFiles will
stay as is (on V2), only newly flushed HFiles will be V3. But when the
table/region undergoes a major compaction, all HFiles will be rewritten to
V3. You can force a rewrite with the HBase Shell 'major_compact' command to
have it immediately rewrite all files.
Created 07-03-2018 08:23 PM
Got it!
Thank you for the explanation.