Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Cloudera Employee

Hortonworks had released HDP 2.6 in Q2 2017. Isilon engineering team worked closely with Hortonworks engineering team to validate HDP 2.6 with OneFS.

One of the features of HDP 2.6 was LLAP (Long Live And Process). LLAP enables as fast as sub-second SQL analytics on Hadoop by intelligently caching data in memory with persistent servers that instantly process SQL queries.

However, please note that LLAP is officially NOT SUPPORTED with OneFS. Customers are advised NOT to use LLAP (even though it may seem to work after config changes) because it could potentially result in data inconsistency issue

Additional Details

  • By default, LLAP uses “inode paths” to access data from file system. OneFS currently does not support referencing files by inode. So queries coming through LLAP will fail on OneFS.
  • However, it is possible to configure Hive LLAP to NOT use inode path by setting hive.llap.io.use.fileid.path = false

With this configuration change, the queries will be executed and this will make LLAP technically “usable” with Isilon but will also cause potential data inconsistency issue. This is because with this capability turned off, LLAP will try to access data by filename (and not by inode path) in the filesystem, which means that the LLAP cache can potentially has an outdated file with the same filename but not knowing that there is a newer one to fetch.

Recommendation:

LLAP support with Isilon is currently planned for future release of OneFS. For more details on timing of release, contact your Isilon/Hortonworks representative

997 Views
Comments
New Contributor

@Vivek Somani : Can you please let me know if Hive LLAP is supported by OneFS in new release? If yes please share the version .

Thanks in advance for your help .

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎09-29-2017 12:38 AM
Updated by:
 
Contributors
Top Kudoed Authors