Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cloudera ApiResource host not found

Cloudera ApiResource host not found

Explorer

I run Cloudera  cm_api on AWS instances. The line below works if I use AWS instance private dns names. But, it fails when I use AWS public dns names, i.e. when CM_HOST = ip-10-11-33-253.ec2.internal, It works.  When CM_HOST = ec2-34-238-174-255.compute-1.amazonaws.com, it fails. But, my application needs to use AWS public dns name. How to solve the problem?

 

API = ApiResource(CM_HOST, version=5, username=ADMIN_USER, password=ADMIN_PASS)

 

cm_api.api_client.ApiException: Host 'ec2-34-238-174-255.compute-1.amazonaws.com' not found. (error 404)

 

9 REPLIES 9

Re: Cloudera ApiResource host not found

Super Guru

@PeterLuo,

 

This is more a DNS issue than an API issue it appears.

 

The error you are getting is a 404 which is pretty strange as that means the server to which you connected said the file attempted to be loadded is not found.

 

check to see what IP address ec2-34-238-174-255.compute-1.amazonaws.com is resolving to and try using curl or a web browser to find out where that goes.

 

If all you want is for ec2-34-238-174-255.compute-1.amazonaws.com to resovle to the same IP address as  ip-10-11-33-253.ec2.internal, then you could just map the  ip-10-11-33-253.ec2.internal ip address to the ec2-34-238-174-255.compute-1.amazonaws.com hostname on your client host.

 

 

Re: Cloudera ApiResource host not found

Explorer

I have 1 master nodes and 4 agents.  Here are the contents of /etc/hosts on these five nodes:

 

cat /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

10.11.32.160 ip-10-11-32-160.ec2.internal ec2-18-212-108-181.compute-1.amazonaws.com

10.11.32.29  ip-10-11-32-29.ec2.internal ec2-54-161-27-121.compute-1.amazonaws.com

10.11.33.83  ip-10-11-33-83.ec2.internal ec2-52-203-192-145.compute-1.amazonaws.com

10.11.32.239 ip-10-11-32-239.ec2.internal ec2-54-175-16-126.compute-1.amazonaws.com

10.11.32.127 ip-10-11-32-127.ec2.internal ec2-34-207-216-32.compute-1.amazonaws.com

 

I can ping ec2-18-212-108-181.compute-1.amazonaws.com.

 

ping ec2-18-212-108-181.compute-1.amazonaws.com

PING ip-10-11-32-160.ec2.internal (10.11.32.160) 56(84) bytes of data.

64 bytes from ip-10-11-32-160.ec2.internal (10.11.32.160): icmp_seq=1 ttl=64 time=0.457 ms

64 bytes from ip-10-11-32-160.ec2.internal (10.11.32.160): icmp_seq=2 ttl=64 time=0.324 ms

 

I can also ssh ec2-18-212-108-181.compute-1.amazonaws.com. But, this API call still fails, 

API = ApiResource(CM_HOST, version=19, username=ADMIN_USER, password=ADMIN_PASS)

 

cm_api.api_client.ApiException: Host 'ec2-18-212-108-181.compute-1.amazonaws.com' not found. (error 404)

 

It works if I change the CM_HOST to use AWS internal address, ip-10-11-32-160.ec2.internal. It looks like somehow cm_api does not accept external dns.

 

 

 

Re: Cloudera ApiResource host not found

Super Guru

@PeterLuo,

 

Wait a minute... what is the actual code you are executing and what is an example of success/failure.

 

I ask because the only reason I can think of that you would get a 404 back from Cloudera Manager on an API call is that your API is using a verison newer than Cloudera Manager supports or you are making an API call that uses an endpoint that is not supported by the API version you specified.

 

Run this command to find out the latest version of the API supported by your CM:

 

curl -X GET -u "admin":"admin" -k http://ec2-18-212-108-181.compute-1.amazonaws.com:7180/api/version; echo ""

 

(change the user/pass as necessary)

The result should be the hightest version of the API that CM supports.

Add that version into your api ApiResource()

 

Also, you can enable API debug logging in CM by going to Administration --> Settings --> Advanced and checking the box next to Enable Debugging of API

 

This will allow you to see CM debugging in the /var/log/cloudera-scm-server/cloudera-scm-server.log file so you can try to figure out what is going on.

 

Re: Cloudera ApiResource host not found

Explorer

The output is v19.

 

I use cm_api to create Cloudera cluster from scratch. There is no way to turn on debuging via GUI. Any command line I can use to trun on debuging?

 

P.S. I use cm_api to install CDH5.14.0.

Re: Cloudera ApiResource host not found

Super Guru

@PeterLuo

 

curl -u admin:admin -H "Content-Type: application/json" -X PUT -d '{ "items" : [ {"name" : "ENABLE_API_DEBUG","value" : "true"} ] }' http://cm_host:7180/api/v19/m/config

Re: Cloudera ApiResource host not found

Super Guru

@PeterLuo

 

Sorry, not sure what happened to that command line:

 

curl -u admin:lizard -H "Content-Type: application/json" -X PUT -d '{ "items" : [ {"name" : "ENABLE_API_DEBUG","value" : "true"} ] }' http://cm_host:7180/api/v19/cm/config

Re: Cloudera ApiResource host not found

Explorer

I meant how to use cm_api to turn on the debug mode. The cm_host does not exist till I use cm_api to create it. The problem is that the creation of Cloudera cluster failed when I use cm_api scripts to create it. The curl command you gave to me is to enable the debug mode after cluster is created. My problem is the creation of cluster failed when I use cm_api scripts to create cluster with AWS public domain name. 

Re: Cloudera ApiResource host not found

Super Guru

@PeterLuo,

 

I think we may need some clarification of what you are doing.

You cannot use the API without Cloudera Manager being installed and running.

You are doing something that is resulting in a 404 HTTP error.

If you are not using TLS, you can then use tcpdump and wireshark to capture and decode packets to show exactly what API call is being made to CM.

 

You need to find out why there is a 404 returned.  I'm just trying to give some options:

 

- turn on API debugging in CM

- use tcpdump/wireshark

- maybe add some debugging to your python code,

Re: Cloudera ApiResource host not found

Explorer

I made changes in /etc/hosts so that I do not have to use public dns names. Using private dns name works.