Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!
As enterprises continue to run critical analytics workloads on premises, developers and platform teams increasingly need deployments that go beyond service installation and focus on real workload validation. This blog shares the experience of deploying Cloudera on premises on Nutanix AHV using a step-by-step deployment guide and validating the environment with Project Axon, an end-to-end on-prem analytics use case that exercises ingestion, processing, analytics, and AI workflows across Cloudera services.
Building on this foundation, the deployment was further validated by integrating Nutanix Objects as an S3-compatible object store and Nutanix Files as enterprise NFS storage for AI workloads. These additional validations demonstrate how Cloudera services can seamlessly leverage native Nutanix storage while supporting production-ready analytics, data engineering, SQL, and AI use cases.
Across both validations, the goal was the same: ensure developers can build, run, and operate production-style pipelines without friction from the underlying platform.
Why does this matter?
Nutanix AHV delivers a modern hyperconverged foundation with the following benefits:
- Simplified Operations – A single, unified platform for compute, storage, and networking
- Elastic Scaling – Independent scaling of Cloudera master and worker nodes based on workload demands
- Cost Efficiency – No additional hypervisor licensing overhead
- Enterprise Security – Integrated networking, isolation, and encryption capabilities
- Automation – API-driven VM lifecycle management through Prism Central
These capabilities make Nutanix AHV an ideal substrate for Cloudera’s hybrid data platform.
Technical Stack
Cloudera and Nutanix components:
Component | Key Version | Role |
RedHat Enterprise Linux (RHEL) | 9.6 | Operating System |
Cloudera Manager | 7.13.2.0 | Centralized cluster management |
Cloudera Runtime | 7.3.2.0 | Cloudera's core data runtime |
Cloudera Data Services (ECS) | 1.5.5 SP2 | Platform for Data Services |
Prism Central | Version pc.7.5 | Nutanix’s centralized management platform |
AOS (Acropolis OS) Version | 7.5.1 | Software-defined storage layer |
AHV version | 11.0.1 | Nutanix hypervisor for virtual machines |
| Nutanix Objects | 5.3.0.1 | S3-compatible object storage |
| Nutanix Files | 5.3 | Enterprise NFS storage for AI workloads |
You can also refer to the Nutanix Compatibility and Interoperability Matrix for the latest Cloudera support details and validated configurations here.
End-to-End Functional Validation Using Project Axon
Using Project Axon, the deployment was validated across the full analytics lifecycle, covering secure ingestion, distributed processing, SQL analytics, and visualization on Cloudera running on Nutanix AHV. Specifically, the Bank Branch Performance Analytics use case from Project Axon was executed to validate real-world behavior across services:
- Data Ingestion: Apache NiFi ingested data from a dummy Python-based generator across multiple datasets and persisted it into HDFS/Ozone, validating secure and stable ingestion.
- Data Processing & Engineering: Spark 3.5.4–based Virtual Clusters were provisioned using Cloudera Data Engineering. Hadoop authentication was configured using keytabs, and Spark jobs were executed for data transformation and summarization, with Airflow pipelines orchestrating the workflows.
- Analytics & SQL: As part of this field validation, Hive and Impala Virtual Warehouses were enabled using Cloudera Data Warehouse, with interactive queries executed via Hue on data ingested by NiFi and processed by Cloudera Data Engineering Spark jobs.
- End-to-End Outcome: Analytics results were visualized using Cloudera Data Visualization by creating dashboards such as best-performing branches, revenue contribution per branch, and call center records analysis, confirming correct data flow across all layers.
Cloudera AI Deployment Validation
In addition to the Project Axon validation, Cloudera AI was deployed and validated on the Nutanix-backed Kubernetes environment.
- The AI Workbench was created successfully
- Hadoop authentication was configured
- Session lifecycle operations were validated using sample project templates
- Cloudera Agent Studio was deployed
Nutanix Storage Integration Validation
As part of the extended validation effort, Cloudera services were integrated with Nutanix's native storage offerings to validate production-ready storage workflows across analytics, data engineering, AI, and SQL services.
Nutanix Objects Integration
Nutanix Objects was validated as an S3-compatible object store across multiple Cloudera services using the native S3A connector.
The following scenarios were successfully validated:
- Cloudera Base (HDFS, Hive and Spark): External tables were created on Nutanix Objects, with end-to-end read, write, and Spark interoperability successfully validated.
- Cloudera Data Engineering (CDE): Application logs generated by Spark and Airflow workloads were successfully offloaded to Nutanix Objects, confirming centralized log storage.
- Cloudera Data Warehouse (CDW): Hive Virtual Warehouse tables were created with managed table locations residing directly on Nutanix Objects, validating SQL analytics against S3-backed storage.
These validations demonstrate that Nutanix Objects can serve as a unified object storage platform across multiple Cloudera services without requiring application changes.
Nutanix Files Integration
Nutanix Files was validated as the shared enterprise NFS storage backend for Cloudera AI.
An external NFS share hosted on Nutanix Files was configured during Cloudera AI Workbench provisioning. Workbench creation, session lifecycle operations, and project storage were successfully validated using the mounted NFS share, confirming support for AI development workloads backed by enterprise file storage.
Known Limitation: Ranger Authorization Service (RAZ) with Nutanix Objects
As part of the validation, Ranger Authorization Service (RAZ) integration with Nutanix Objects was also evaluated. Cloudera RAZ requires AWS-like Security Token Service (STS) AssumeRole support to enable dynamic credential exchange through Knox IDBroker, which is currently not supported by Nutanix Objects.
As a result, RAZ integration could not be validated as part of this deployment. This is a current architectural limitation rather than a deployment or configuration issue, and this is a recommended capability for the Nutanix Objects roadmap.
For detailed testing methodology, configurations, and validation results, see the [Cloudera on Nutanix Joint Technical Whitepaper].
Enterprise Security
A comprehensive security framework was implemented to meet enterprise compliance and governance requirements:
- Identity & Access: Integrated Active Directory as the centralized identity provider, DNS server, leveraging LDAP and Kerberos for secure authentication and principal management.
- Data Protection: Enabled AutoTLS across all Cloudera services to secure data in transit, and enforced fine-grained authorization using Ranger policies.
- Gateways : Enabled Knox for secure SSO-based access and configured Atlas for metadata and lineage management.
Additional Deployment Considerations
Based on field discussions and deployment experience, the following practices are recommended for production deployments on Nutanix AHV:
- For storage attached to HDFS DataNode VMs, using Nutanix Volume Groups (VGs) is recommended, as they behave similarly to VMware independent disks and remain preserved even if a VM is accidentally deleted, unlike standard vDisks.
- When leveraging Nutanix AOS replication, a commonly recommended approach is configuring HDFS replication factor (RF) as 2 together with AOS RF2 and enabling EC-X for storage efficiency and resiliency.
- VM affinity and anti-affinity policies, workload isolation, and spare capacity planning should be considered to support high availability and planned maintenance operations for physical nodes on Nutanix.
For additional deployment guidelines and VM placement best practices, refer to the guide here.
Conclusion
This deployment validates Cloudera on premises on Nutanix AHV for running end-to-end analytics, AI, and data engineering workloads with enterprise-grade security, scalability, and operational simplicity.
As part of this validation, the Project Axon use case successfully demonstrated end-to-end data ingestion, processing, analytics, visualization, and AI workflows across Cloudera services. The validation was further extended to include Nutanix Objects as an S3-compatible object store across Cloudera Base, Cloudera Data Engineering, and Cloudera Data Warehouse, as well as Nutanix Files as enterprise NFS storage for Cloudera AI Workbench, demonstrating seamless integration with Nutanix storage services.
Together, these validations provide customers and partners with a production-ready reference architecture for deploying Cloudera on Nutanix AHV while leveraging the broader Nutanix ecosystem.
