HDP 2.5 Documentation Updates for Streaming Components Storm and Kafka

[#1]

Our stream analytics documentation received substantial updates for HDP 2.5.0.

The Apache Storm Component Guide was reorganized to focus on administration and development workflow: installing and configuring Storm, developing Storm applications, moving data into and out of Storm, and managing topologies.

New content includes the following topics:

• Installing Storm using Ambari.
• Configuring Storm for a production environment.
• Implementing windowing computations on data streams.
• Implementing state management in core Storm.
• Configuring and using the HDFS spout to ingest data from HDFS.
• Monitoring and debugging Apache Storm topologies through the use of dynamic log levels, topology event logging, distributed log search, and dynamic worker profiling.

The Storm application development chapter differentiates more clearly between core Storm and Trident, and the Kafka spout subsection describes performance settings and tradeoffs.

The Storm Ambari view is documented in the Apache Ambari Views Guide, and the Apache Ambari Apache Storm Kerberos Configuration Guide (formerly Configuring Storm for Kerberos Over Ambari) has moved to the Security Guide.

The Apache Kafka Component Guide was enhanced with the following content:

• Installing Kafka using Ambari.
• Configuring Kafka for a production environment.
• Sample code for a basic Kafka producer and consumer, with and without SSL enabled on the cluster.

The Apache Ambari Apache Kafka Kerberos Configuration Guide (formerly Configuring Kafka for Kerberos Over Ambari) has moved to the Security Guide.

If you have comments or suggestions, corrections or updates regarding our documentation, let us know on HCC. Help us continue to improve our documentation!