Yes, we had 2.3.4 installed in this node. However we have taken the following steps and its working fine now.
1) Deleted 2.3.4 and current folders under /usr/hdp
2) Restarted the ambari-agent
3) Took care of the issues found in pre run checks (like deleting old rpms, folders and users, etc.)
4) Added the new host. However it required python-argparse rpm
5) Added Data Nodes, Node Managers and other services and they are working fine now.