Support Questions

Find answers, ask questions, and share your expertise

yet another errno 1111 - cant connect to data node

this morning one of my datanodes was not sending heartbeats and the whole server seemed to have been locked up. now when it came back i did the usual start up of things through ambari no issues everything came back green. however 2 alerts popped up. one was the Datanode process and the other was the Datanode webui giving me an errno 1111 connect refused can not connect to connection refused

in the past i could connect and see the datanode was working by putting <ip-address>:1022 in my webbrowser and it would take me to a screen showing its running. however its not allowing that anymore. ambari says its started but HDFS is showing a dead node and the webui isnt working. i checked the logs and i cant for the life of me see any type of error. and there is no error in the start up process as well.

so any help on troubleshooting and or fixing this issue would be great.

thanks in advance.


Hi @steve coyle, have you upgraded your Linux kernel recently? If so it is likely this issue:

You can either rollback the kernel version, or upgrade to a newer kernel that fixes that this regression (assuming your vendor has one), or apply the workaround mentioned in the description of that Jira.