06-17-2019 01:55 PM
We have a CDH 5.13.0 cluster that had to be moved from Azure VMs to Google Cloud. Image of each VM was captured in Azure, and then VMs recreated in GCP with all nodes now, obviously, getting new names.
To resolve nodes name change an entry for each host was added to hosts file (to map from old name to new node names).
Cluster seems to be running, HDFS is up, but I'm running into multiple problems like HDFS nodes having high Block Count, Hive queries not working (I don't see any jobs starting when I submit a query, and then I'd get a random "The operation doesn't have handle attached." or a timeout).
First quesiton, is this scenario of moving to new hosts, and just remapping old nost names to new via hosts file entries even supported?
Second question, how to troubleshoot the cluster now? Rebalancing doesn't seem to resolve the Block Count issue. Cluster is running, but I feel something is broken -- as I can't run any workflows. Where do I start?