HDP 2.2 and Isilon OneFS 7.2.0.4 are now officially certified by Hortonworks and EMC Isilon and ready for Hadoop deployment. Hortonworks and EMC Isilon have a close engineering relation that started in September of 2014, to ensure that Hortonworks Data Platform (HDP) is integrated with the Isilon OneFS filesystem. The HDP test suite employed to certify the Isilon OneFS filesystem has over 10,000 individual tests. The gives assurance to customers that both EMC and Hortonworks have done the due diligence to ensure that all Hadoop workloads work on this integrated platform, with security and operational ease. HDP with Isilon reference architecture Isilon OneFS has implemented the HDFS API as an over the wire protocol consistent with its multi-protocol support for NFS, SMB and others. This is different from implementations of Hadoop Compatible File Systems (HCFS) in that OneFS mimics the HDFS behavior for the subset of features that it supports. The deployment model for HDP with Isilon is shown in the figure above, where HDP is installed on a compute cluster where the nodes can be on-premise or in the cloud. Installation of HDP over Isilon is supported via a custom Ambari Agent that is included with OneFS. Details of HDP Certification with Isilon Certification of HDP with Isilon is an ongoing commitment from EMC and Hortonworks. It started with with HDP 2.1 and Isilon OneFS 7.2.0.1 in Q2 of 2015. The Isilon engineering team recently wrapped up HDP 2.2 certification with Isilon OneFS 7.2.0.4 and is currently in the process of certifying the HDP 2.3 with Isilon OneFS 8.0 with an expected completion date of Q1 2016. The coverage of components as part of the HDP certification effort is depicted above. Components in the HDP 2.2 column are part of the HDP 2.2 certification and new components that will be added in HDP 2.3 are included in the HDP 2.3 column, including Spark on YARN, Tez as the default execution engine for Hive and Pig, Kafka for streaming data ingest and Knox for perimeter security. If you have currently deployed HDP 2.2 with Isilon and are considering upgrading to HDP 2.3, we have validated that HDP 2.3 is compatible with HDP 7.2.1.1 while detailed certification testing is in progress. Details of the the upgrade process can be found in a recent blog post shared by Isilon engineering team. ... View more

EMC Blog Post containing instructions when upgrading HDP 2.2 to HDP 2.3 with Isilon We have been working with the Isilon engineering team to document the upgrade process from HDP 2.2. to HDP 2.3 when the underlying storage layer is OneFS instead of HDFS. The instructions in the blog post are to be used in conjunction with Ambari instructions on upgrading HDP. Important to note that Isilon is working on making enhancements to OneFS to support rolling upgrade. This will be made available in an upcoming release of OneFS. ... View more