We are thrilled to share the latest release of Cloudera Data Engineering 1.23 on public cloud. With major enhancements for developer experience and product stability, this release enables you to benefit from increased productivity and collaboration while ensuring enterprise-grade security.  Discover New Benefits & Features Boosted Development Productivity: The External IDE Connectivity (Tech Preview) powered by Apache Spark Connect enables practitioners to run large-scale Spark workloads remotely on Cloudera Data Engineering. That means you can access Spark from your favorite IDE and collaborate more efficiently with your team.   Optimized, Cost-Effective Workflows: Support for Apache Airflow 2.9 and Python 3.11 expands access to the latest libraries and frameworks, equipping you with tools to meet your dynamic needs for building scalable data pipelines. Apache Iceberg 1.5, together with the support of Apache Spark 3.5, delivers better overall performance and cost management. For example, in use cases such as Change Data Capture (CDC), improvements in row-level deletes with Merge-on-Read enable more efficient query processing. This results in faster query execution and reduced resource consumption, leading to potential cost savings. Improved Administrative Capabilities: We strengthened stability with In-Place Upgrade and Backup & Restore to ensure a reliable and smooth upgrade experience.  Ready to Learn More? View Product Release Notes, Compatibility and Runtime Components for Cloudera Data Engineering Start your 5-day trial for Cloudera Data Engineering on AWS  Download the new datasheet for Cloudera Data Engineering ... View more

We are thrilled to announce that Cloudera Data Engineering now supports AWS Graviton3 instances. This marks a significant milestone in our commitment to delivering high-performance, cost-effective solutions for your data engineering needs.  AWS Graviton is a family of 64-bit ARM-based CPUs designed to support improved compute performance, memory bandwidth, and machine learning capabilities with less energy consumption –all essential criteria for modern data engineering workloads. This blog walks you through how you can take advantage of these powerful processors to deliver stronger results.  15%+ Cost Savings and Performance Enhancements One of the key benefits of switching to AWS Graviton3 instances is reducing costs, while achieving comparable performance. Cloudera customers may reduce their costs by 15-20% compared to alternative instance types. Get Started with Graviton in Cloudera Data Engineering Create AWS Graviton instances for Cloudera Data Engineering in six steps: Log in to the Cloudera Data Engineering Console: Ensure you have the necessary permissions to create and manage services. Navigate to the Administration Page: Go to the Administration page, where you can create and manage your Cloudera Data Engineering services. Create a New Cloudera Data Engineering Service: Click on "Create New Service."  Choose Graviton Instance Types: In the "Workload Type" drop-down menu, select one of the supported Graviton instance types. Here is the list of supported Graviton instance types: r7gd r7g m7gd m7g  Configure Service Details: Set up the details of your new Cloudera Data Engineering service according to your requirements, including network settings, storage configurations, and any additional options. Review and Creation: Review your service configuration and click on "Create" to start your new Cloudera Data Engineering service using Graviton3 instances. You can check here for the compatibility matrix of Graviton in Cloudera Data Engineering. Learn more about how Cloudera Data Engineering can help you orchestrate and secure data pipelines. Explore our community page for the latest discussions and documentation for detailed best practices.  The Cloudera Data Engineering team looks forward to your feedback and is excited to see how you take advantage of this new capability. If you need assistance, our support team is here to help, as always. ... View more

We are excited to announce a series of new updates on Cloudera Data Engineering for the public cloud. Explore how you can benefit from greater performance, cost-efficiency, and versatility with new features for modern data engineering workloads. Discover New Benefits  15%+ Cost Savings with AWS Graviton: We now support AWS ARM-based processors (Graviton). Switching to Graviton instances can reduce infrastructure costs by 15-20% while achieving comparable performance.  Upgraded Performance: Support for Apache Spark 3.5 and Apache Iceberg 1.4 boosts performance and provides access to more practitioner-focused capabilities, including Spark Connect and enhanced PySpark and SQL capabilities. This dual upgrade also enables multi-table commits and enhances consistency in your data operations. You can find our compatibility matrix here. Productivity Boost with Flexible Practitioner Tooling: Take advantage of iterative development with Interactive Sessions. Develop, test, and refine your data workflows in a highly dynamic and flexible environment. Ready to Learn More? Start your 5-day Trial for Cloudera Data Engineering on Public Cloud View full release notes Join our Data Engineering Community ... View more