Thanks, we'll try with GraphiteSink first. ... View more

Thanks for the code! We'll definitely keep this in mind, but for our task at hand we don't have access to Spark source, so we'll go for GraphiteSink or JmxSink. ... View more

Hi @drussell and @Andrew Grande, thank you for your responses. If Camus is dead, is Goblin the best way to move data from Kafka to HDFS? Camus has some nice features like topic discovery, partitioning, load balancing etc. I hope Goblin offers them too. ... View more

HBase ports changed in HDP-2.3.4, details here. And region servers has logical names which include RS port. An example RS name: "sandbox.hortonworks.com,16020,1460965964168". Before, the RS port was 60020, and now it's 16020. So, if you have 5 machines running RS, before and after the upgrade there are 10 RSs to be taken care of, and in your case HBase master may still think that RS names with 60020 are still being used. Restarting HBase (or just restarting HBase master) is supposed to remove them, and solve your issues. Before and after the restart you can check Regions servers in your Hbase Web UI (HBase --> Quick Links). ... View more

For Spark on HDP, please check HDP Spark Guide. For general Spark programming check Spark Programming Guide with all examples given in Scala, Java, and Python. If you want to program in Java you need a development environment like Eclipse or IntelliJ. ... View more