parcel download from repo via proxy failing

[#1]

Hi

I've configured a proxy server for the parcel download from a remote repo

I've restarted the cloudera-scm-server and cloudera-scm-agent processes, but still getting the same error

note: I masked the actual hostname by xxx.net

if I use curl from the same host, using the same http proxy, i can download the manifest file

So I guess the issue must be somewhere in CDM

any suggestion welcome

2017-06-07 11:01:06,913 ERROR ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: (19 skipped) Unable to retrieve remote parcel repository manifest
java.util.concurrent.ExecutionException: java.net.ConnectException: http://xxx.net/apps/cloudera/kafka/manifest.json
    at com.ning.http.client.providers.netty.NettyResponseFuture.abort(NettyResponseFuture.java:297)
    at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:104)
    at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:399)
    at org.jboss.netty.channel.DefaultChannelFuture.addListener(DefaultChannelFuture.java:145)
    at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.doConnect(NettyAsyncHttpProvider.java:1041)
    at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.execute(NettyAsyncHttpProvider.java:858)
    at com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:512)
    at com.ning.http.client.AsyncHttpClient$BoundRequestBuilder.execute(AsyncHttpClient.java:234)
    at com.cloudera.parcel.components.ParcelDownloaderImpl.getRepositoryInfoFuture(ParcelDownloaderImpl.java:534)
    at com.cloudera.parcel.components.ParcelDownloaderImpl.getRepositoryInfo(ParcelDownloaderImpl.java:492)
    at com.cloudera.parcel.components.ParcelDownloaderImpl.syncRemoteRepos(ParcelDownloaderImpl.java:344)
    at com.cloudera.parcel.components.ParcelDownloaderImpl$1.run(ParcelDownloaderImpl.java:416)
    at com.cloudera.parcel.components.ParcelDownloaderImpl$1.run(ParcelDownloaderImpl.java:411)
    at com.cloudera.cmf.persist.ReadWriteDatabaseTaskCallable.call(ReadWriteDatabaseTaskCallable.java:36)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: http://xxx.net/apps/cloudera/kafka/manifest.json
    at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:100)
    ... 16 more
Caused by: java.nio.channels.UnresolvedAddressException
    at sun.nio.ch.Net.checkAddress(Net.java:101)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)

[#2]

Hi jeroenr!

You're right, this is most likely an issue with cloudera manager's configuration. The parcels are downloaded by CM server, so the status of cloudera-scm-agent shouldn't metter.

Please help me better understand your situation:

Have you set up proxy via Administration > Settings > Network in Cloudera Manager?
Is xxx.net your proxy's domain name, or the hostname of a pracel repository mirror?

Seems like CM can't resolve the domain xxx.net causing this failure. Could you check if you can resolve xxx.net to an IP from the same host? (dig xxx.net)

[#3]

xxx.net is the repo server

I've configured the proxy server and the proxy port via Administration > Settings > Network in CM

proxy server: http://abcd.net

port: 8080

then checking for parcels

the error message shows connectException: http://xxx.net/apps/cloudera/kafka/manifest.json

setting the same proxy on the cmdline on the same host:

export http_proxy=http://abcd.net:8080

curl http://xxx.net/apps/cloudera/kafka/manifest.json

the manual test runs fine

[#4]

also tested the repo using the actual ip address instead of xxx.net

the error message shows connectException: http://1.2.3.4/apps/cloudera/kafka/manifest.json

got the same error

the same test using curl was running fine again

I've been searching other cases and found some references to issues when there's an Oracle db used for CDM

I've got the same setup with an Oracle db, any specific tables I could check for any issues?

[#5 · Accepted solution]

I would double-check network and CM configuration before considering a database issue.

It seems that cloudera manager is ignoreing your proxy configuration. I tried to reproduce your error, and I found that specifing http:// before the proxy servers domain name could cause this. Could you confirm that you have it set to abcd.net and not http://abcd.net?

If that doesn't help, please specify your CM, OS and database version, so I can try that configuration.

[#6]

thanks, it was the http:// in the proxy server config