Thank you so much. ... View more

yes, you are right HDP is license free. however the cluster install needs master and slave infrastructure, so all the HDFS and data name, name node, yarn alone with some mandatory like Hive, pig, tez need installed and maintained. All of these cost a lots on AWS/EC2. in other hand, without HDP cluster, the monitoring, upgrade, version compatiblility and security on kafka, spark, zookeeper may bring issues in long run. admin have to deal with it. I am looking for use case of zookeeper/kafka cluster and spark to compare to see if it is worth to do so. How do you think? ... View more

Thank you very much Wang for confirm the zk and kf and work alone. yes, with HDP, kafka and zookeeper is better administrated and monitored. I did setup setup kafka cluster and mongodb with HDP, it seemed very easy steps. However, I tried to save some money for our company, so I came up with this question. thank you for confirm it. ... View more

yes, thank you, all working. I missed ';' at end of update statement. thanks again. ... View more

1. make sure your ntpd, it need runing on all hosts. ntpd status and ntpd start 2. check your selinux file if SELINUX=disabled 3. your /etc/hosts file include all host privte ip and public ip. i.e. ip-..... ec2... 10.23.24.234 4. disable the iptables, service iptables stop. or systemctl stop firewall 5. reboot machine and make sure ntpd status is active. this worked for me all the time. you need to clikc on the 'check here for warnings' to see exact info. ... View more

Hi, I came back to distributed mode again. After ssh-keygen -t rsa and cat id_rsa.pub >> authorized_keys, this may cause the 'connection refused' error in my case. It was able to create the connector, and create testCollection in mongodb, however, no data in the testCollection. can you give a tip on why I could not pass the data? also, netstat -tnulp|grep 8083 output nothing. so it means the connector is not created, right? Thank you so much. Robin ... View more

Thank you very much. I am trying to connect-standalone, and replace localhost to public IP, I still got the same error: 1) ./bin/connect-standalone.sh /tmp/connect-standalone .properties [2017-07-04 16:20:19,573] INFO Usage: ConnectStandalone worker.properties connec tor1.properties [connector2.properties ...] (org.apache.kafka.connect.cli.Connec tStandalone:61) 2) curl -X POST -H "Content-Type: application/json" --data @/tmp/mongo_connector_configs.json http://ec2-54-111-22-333.us-west-1.compute.amazonaws.com:8083/connectors curl: (7) Failed connect to ec2-54-153-75-126.us-west-1.compute.amazonaws.com:8083; Connection refused Can you help? thanks, Robin ... View more