Reply
Highlighted
Explorer
Posts: 12
Registered: ‎06-16-2016
Accepted Solution

parcel download from repo via proxy failing

Hi

 

I've configured a proxy server for the parcel download from a remote repo

I've restarted the cloudera-scm-server and cloudera-scm-agent processes, but still getting the same error

 

note: I masked the actual hostname by xxx.net

if I use curl from the same host, using the same http proxy, i can download the manifest file

So I guess the issue must be somewhere in CDM

 

any suggestion welcome

 

2017-06-07 11:01:06,913 ERROR ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: (19 skipped) Unable to retrieve remote parcel repository manifest
java.util.concurrent.ExecutionException: java.net.ConnectException: http://xxx.net/apps/cloudera/kafka/manifest.json
    at com.ning.http.client.providers.netty.NettyResponseFuture.abort(NettyResponseFuture.java:297)
    at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:104)
    at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:399)
    at org.jboss.netty.channel.DefaultChannelFuture.addListener(DefaultChannelFuture.java:145)
    at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.doConnect(NettyAsyncHttpProvider.java:1041)
    at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.execute(NettyAsyncHttpProvider.java:858)
    at com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:512)
    at com.ning.http.client.AsyncHttpClient$BoundRequestBuilder.execute(AsyncHttpClient.java:234)
    at com.cloudera.parcel.components.ParcelDownloaderImpl.getRepositoryInfoFuture(ParcelDownloaderImpl.java:534)
    at com.cloudera.parcel.components.ParcelDownloaderImpl.getRepositoryInfo(ParcelDownloaderImpl.java:492)
    at com.cloudera.parcel.components.ParcelDownloaderImpl.syncRemoteRepos(ParcelDownloaderImpl.java:344)
    at com.cloudera.parcel.components.ParcelDownloaderImpl$1.run(ParcelDownloaderImpl.java:416)
    at com.cloudera.parcel.components.ParcelDownloaderImpl$1.run(ParcelDownloaderImpl.java:411)
    at com.cloudera.cmf.persist.ReadWriteDatabaseTaskCallable.call(ReadWriteDatabaseTaskCallable.java:36)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: http://xxx.net/apps/cloudera/kafka/manifest.json
    at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:100)
    ... 16 more
Caused by: java.nio.channels.UnresolvedAddressException
    at sun.nio.ch.Net.checkAddress(Net.java:101)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)

Cloudera Employee
Posts: 4
Registered: ‎05-29-2017

Re: parcel download from repo via proxy failing

Hi jeroenr!

 

You're right, this is most likely an issue with cloudera manager's configuration. The parcels are downloaded by CM server, so the status of cloudera-scm-agent shouldn't metter.

 

Please help me better understand your situation:

Have you set up proxy via Administration > Settings > Network in Cloudera Manager?
Is xxx.net your proxy's domain name, or the hostname of a pracel repository mirror?

Seems like CM can't resolve the domain xxx.net causing this failure. Could you check if you can resolve xxx.net to an IP from the same host? (dig xxx.net)

Explorer
Posts: 12
Registered: ‎06-16-2016

Re: parcel download from repo via proxy failing

xxx.net is the repo server

 

I've configured the proxy server and the proxy port via Administration > Settings > Network in CM

proxy server: http://abcd.net

port: 8080

 

then checking for parcels

 

the error message shows connectException: http://xxx.net/apps/cloudera/kafka/manifest.json

 

setting the same proxy on the cmdline on the same host:

export http_proxy=http://abcd.net:8080

curl http://xxx.net/apps/cloudera/kafka/manifest.json

 

the manual test runs fine

Explorer
Posts: 12
Registered: ‎06-16-2016

Re: parcel download from repo via proxy failing

also tested the repo using the actual ip address instead of xxx.net

the error message shows connectException: http://1.2.3.4/apps/cloudera/kafka/manifest.json

got the same error

 

the same test using curl was running fine again

 

I've been searching other cases and found some references to issues when there's an Oracle db used for CDM

I've got the same setup with an Oracle db, any specific tables I could check for any issues?

Cloudera Employee
Posts: 4
Registered: ‎05-29-2017

Re: parcel download from repo via proxy failing

I would double-check network and CM configuration before considering a database issue.

 

It seems that cloudera manager is ignoreing your proxy configuration. I tried to reproduce your error, and I found that specifing http:// before the proxy servers domain name could cause this. Could you confirm that you have it set to abcd.net and not http://abcd.net?

 

If that doesn't help, please specify your CM, OS and database version, so I can try that configuration.

Explorer
Posts: 12
Registered: ‎06-16-2016

Re: parcel download from repo via proxy failing

thanks, it was the http:// in the proxy server config

Announcements