Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Run remote job Spring Boot YARN

Highlighted

Run remote job Spring Boot YARN

Explorer

Hi,

Trying to run this simple Spring Boot Yarn app. Launching app from Windows for it to be deployed on Hortonworks Sandbox HDP 2.5.

I just simply run original Spring Boot YARN app. Jars are about to deploy to sandbox, but after that I'm getting error:

org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /app/gs-yarn-basic/gs-yarn-basic-container-0.1.0.jar could only

be replicated to 0 nodes instead of minReplication (=1).

There are 1 datanode(s) running and 1 node(s) are excluded in this operation.

All services are working, datanode is up and there is enough memory.

I know, the problem is in network configuration of sandbox - by default it's IP is 172.17.0.2, my network is 192.168.0.x.

When I set in my app

fsUri: hdfs://localhost:9000

according to this article, client tries to talk to datanode, but the datanode is running on 172.17.0.2, so from my host machine it cannot copy files.

That's why i tried this instructions and my sanbox now has IP 192.168.0.x.

But when I run my app - it says that it cannot connect to Resource Manager

Client: Retrying connect to server: sandbox.hortonworks.com/192.168.0.x:8032. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

What else configuration should I change?

1 REPLY 1
Highlighted

Re: Run remote job Spring Boot YARN

New Contributor

In the application.yaml file you need to add the resource manager port, as in the HDP cluster it's 8050 instead of 8032.

spring:
    hadoop:
        fsUri: hdfs://grvd-hd-c-1:8020
        resourceManagerHost: grvd-hd-m-1
        resourceManagerPort: 8050

Don't have an account?
Coming from Hortonworks? Activate your account here