Community Articles

Find and share helpful community-sourced technical articles.
Celebrating as our community reaches 100,000 members! Thank you!

HDP 2.2 and Isilon OneFS are now officially certified by Hortonworks and EMC Isilon and ready for Hadoop deployment.

Hortonworks and EMC Isilon have a close engineering relation that started in September of 2014, to ensure that Hortonworks Data Platform (HDP) is integrated with the Isilon OneFS filesystem. The HDP test suite employed to certify the Isilon OneFS filesystem has over 10,000 individual tests. The gives assurance to customers that both EMC and Hortonworks have done the due diligence to ensure that all Hadoop workloads work on this integrated platform, with security and operational ease.


HDP with Isilon reference architecture

Isilon OneFS has implemented the HDFS API as an over the wire protocol consistent with its multi-protocol support for NFS, SMB and others. This is different from implementations of Hadoop Compatible File Systems (HCFS) in that OneFS mimics the HDFS behavior for the subset of features that it supports. The deployment model for HDP with Isilon is shown in the figure above, where HDP is installed on a compute cluster where the nodes can be on-premise or in the cloud. Installation of HDP over Isilon is supported via a custom Ambari Agent that is included with OneFS.

Details of HDP Certification with Isilon


Certification of HDP with Isilon is an ongoing commitment from EMC and Hortonworks. It started with with HDP 2.1 and Isilon OneFS in Q2 of 2015. The Isilon engineering team recently wrapped up HDP 2.2 certification with Isilon OneFS and is currently in the process of certifying the HDP 2.3 with Isilon OneFS 8.0 with an expected completion date of Q1 2016.


The coverage of components as part of the HDP certification effort is depicted above. Components in the HDP 2.2 column are part of the HDP 2.2 certification and new components that will be added in HDP 2.3 are included in the HDP 2.3 column, including Spark on YARN, Tez as the default execution engine for Hive and Pig, Kafka for streaming data ingest and Knox for perimeter security.

If you have currently deployed HDP 2.2 with Isilon and are considering upgrading to HDP 2.3, we have validated that HDP 2.3 is compatible with HDP while detailed certification testing is in progress. Details of the the upgrade process can be found in a recent blog post shared by Isilon engineering team.


Vinod, this is a great FAQ article. Is this the "latest" certification? Will this be limited to HDP 2.2 and HDP 2.3? How do we maintain this info in this post so it stays current over the years as multiple certifications are done over many versions?


Thanks David. The commitment from EMC and HWX is ongoing certification. I am not sure if AnswerHub allows for versioning, so folks can look at historical posts. However I will update this article going forward.