how to find kafka and spark packages versions?

[#1]

I am new to spark and Kafka. I am following the tutorial at http://henning.kropponline.de/2016/12/25/simple-spark-streaming-kafka-example-in-a-zeppelin-notebook...

In the artilces, kafka and spark packages are

org.apache.spark:spark-streaming-kafka_2.10:1.6.2

org.apache.kafka:kafka_2.10:0.8.2.2

org.apache.kafka:kafka-clients:0.8.2.2

I installed hdp 2.6. What are the version of those packages should I use?

how could I find it out?

[#2]

@Andy Liang

1. If you have access to Ambari, you can see the components version details on Stacks and Version Page

On Ambari screen, top right, Hit the Admin tab > Stacks and Version

Or directly using http://ambarihost:port/#/main/admin/stack/services

Replace ambarihost:port with details from your cluster.

If you dont have access to ambari:

2. Try out the hdp-select command. This will list all packages installed.

hdp-select status

You can grep for the package you are looking for:

hdp-select status | grep kafka

hdp-select status | grep spark

3. Alternatively, you can also use the standard yum commands to get the similar result.

However, using yum will list multiple versions of packages if they exist on your system.

yum list installed | grep spark

yum list installed | grep kafka

[#3]

@Andy Liang

Release notes page for each version of HDP includes versions packaged for each project. Following is for HDP 2.6.1. Kafka version is 0.10.1.2 and Apache Spark is 2.1.1 as well as Apache Spark 1.6.3.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_release-notes/content/comp_versions.html

to find version of Spark from command line, type spark-submit --version.

[#4]

@Andy Liang If you are using ambari, then you can refer to 'Admin > Stack and Version' page (<ambari_url>/#/main/admin/stack/services) to know the version for respective service.

Attaching the screenshot.stack-version.jpg