HDP 2.6 Documentation Updates for Data Science Components

The Spark Component Guide and Command Line Installation Guide were updated to reflect new Spark features. Here are links to several of the latest features:

• Support for Spark 2, documented in several topics including:
  • Installing Spark Using Ambari
  • Installing and Configuring Apache Spark 2 (manual installation)
  • Running Spark
  • Configuring Spark2 for Wire Encryption
  • Automating Spark Jobs with Oozie Spark Action
  • Using Livy with Spark Versions 1 and 2
• Livy API information, in Submitting Spark Applications Through Livy
• Enabling Spark SQL user impersonation for the Spark Thrift Server (doAs support), in Configuring the Spark Thrift Server

The Zeppelin Component Guide was updated with additional details and examples for configuring Zeppelin with LDAP/AD and Kerberos security; see Configuring Zeppelin Security. In addition, the documentation for interpreters and user impersonation was extended. Portions of this information that apply to HDP 2.5 were also added to the Security chapter in the HDP 2.5 Zeppelin Component Guide.

In the messaging area, the Kafka Component Guide has additional information in Configuring Kafka for a Production Environment.