first check, whether these datanodes are reachable from ambari-server using ssh protocol and their hostnames. And also try to do vice-versa then telnet from datanode to ambari server using ambari-server hostname on port 8440. If everything looks good. Then kill the current ambari-agent daemon and restart this service. Please make sure there is no hung stale instance of Ambani-Agent is running. If it does not work then stop Ambari server. Stop postgresql DB server Now Start Ambari-Server and it will start postgreSQL server itself. Let me know if it does not fix the issue. ... View more

A HDFS block corresponds to 1 file in local file system on a datanode. So regardless of data-size, all the data will be broken into 128 MB data-files stored into local file system by default. The last chunk 84 MB will be also written to a new data-file. So you will find following block files (data-files) in your datanode local file system: Here is an example: /haadoop/data/dfs/data/ ├── current │ ├── BP-1079595417-192.168.2.45-1412613236271 │ │ ├── current │ │ │ ├── VERSION │ │ │ ├── finalized │ │ │ │ └── subdir0 │ │ │ │ └── subdir1 │ │ │ │ ├── blk_1073741825 (128 MB) │ │ │ │ ├── blk_1073741826 (128 MB) │ │ │ │ ├── blk_1073741827 (84 MB) Look for 'dfs.datanode.data.dir' property in HDFS configuration. It tells where these files (which represent HDFS blocks) are located on a datanode local file system. ... View more

It is a dockerized multi-node environment. Firewall and NTP issues cause warnings during the node registration, which can be safely ignored. Because all nodes are virtualized inside the VM itself using Docker so, you do not need to address Firewall issue. Same for NTP, all nodes are getting time from the host OS, so they will be always in sync. They should not cause a fatal error. Please provide log for the registration-failure issue. So that we could pin-point the actual error. The issue could be related to AWS->security group. Did you open required ports in AWS->security group so that the VM could download HDP and Ambari repositories from web during the node-registration and HDP installation process. Can you open all the ports for the time-being and try again. ... View more

I guess you got the wrong setup guide. Here is the right setup guide: https://s3-us-west-1.amazonaws.com/setup-guides/HDPOperationsMigration-ManageAmbari-2.1-Downloadable+VM+Setup+Guide_Rev1.0.pdf BTW, it is a pretty old course based on HDP2.1. It is a subset of main HDP admin course. I would recommend you, to try latest HDP Admin course instead. Thanks, Manish ... View more