Docker Cloudera image Hue login issues

[#1]

Hi,

I have just successfully run a Docker container image of Cloudera.  I can connect to the instance using Safari on the Mac.  However, I get an error message that says it can't connect:

"Could not connect to quickstart.cloudera:21050 (code THRIFTTRANSPORT): TTransportException('Could not connect to quickstart.cloudera:21050',)"

Also, when I do a check configuration it comes up with a bunch of errors:

hadoop.hdfs_clusters.default.webhdfs_url	Current value: http://localhost:50070/webhdfs/v1 Failed to access filesystem root
OOZIE_EMAIL_SERVER	Email notifications is disabled for Workflows and Jobs as SMTP server is localhost.
Hive	Failed to access Hive warehouse: /user/hive/warehouse
HBase Browser	Failed to authenticate to HBase Thrift Server, check authentication configurations.
Impala	No available Impalad to send queries to.
Hadoop Security: Sentry Service	Failed to connect to Sentry API (version 1).
Hadoop Security: Sentry Service	Failed to connect to Sentry API (version 2).

Anyone have a direction they can point me on how to resolve all of these issues.  I thought the Docker image would work out of the box.

Thanks!

[#2]

@pollard,

Are any of your services running?  Maybe provide a screen shot of Cloudera Manager's home page and also the hosts page.

"Could not connect to quickstart.cloudera:21050" means that Hue cannot connect to the impala daemon so make sure that it is running and listening on port 21050

[#3]

According to the output logs:

Using CATALINA_BASE:   /var/lib/solr/tomcat-deployment

Using CATALINA_HOME:   /usr/lib/solr/../bigtop-tomcat

Using CATALINA_TMPDIR: /var/lib/solr/

Using JRE_HOME:        /usr/java/jdk1.7.0_67-cloudera

Using CLASSPATH:       /usr/lib/solr/../bigtop-tomcat/bin/bootstrap.jar

Using CATALINA_PID:    /var/run/solr/solr.pid

Started Impala Catalog Server (catalogd) :                 [  OK  ]

Started Impala Server (impalad):                           [  OK  ]

Yes, impalad is running.  But, according to "ps aux" it is not:

ps aux | grep -i impala

impala    2693  0.1  0.0 357552   612 ?        Sl   Sep14   6:24 /usr/lib/impala/sbin/statestored -log_dir=/var/log/impala -state_store_port=24000

impala    3077  0.3  3.0 1034932 62260 ?       Sl   Sep14  13:46 /usr/lib/impala/sbin/catalogd -log_dir=/var/log/impala

There's no "impalad" process running.  I assumed the log was correct and did not check.

The "catalogd.quickstart.cloudera.impala.log.ERROR.20180914-185251.3077" log contains:

--------------------------------------------------------------------------

First error:

--------------------------------------------------------------------------

Log file created at: 2018/09/14 18:52:51

Running on machine: quickstart.cloudera

Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

E0914 18:52:51.962208  3077 logging.cc:126] stderr will be logged to this file.

W0914 18:53:34.936743  3448 Client.java:886] Failed to connect to server: quickstart.cloudera/172.17.0.2:8020: try once and fail.

Java exception follows:

java.net.ConnectException: Connection refused

        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)

        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)

        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)

        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)

        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:648)

        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:744)

        at org.apache.hadoop.ipc.Client$Connection.access$3000(Client.java:396)

        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1557)

        at org.apache.hadoop.ipc.Client.call(Client.java:1480)

        at org.apache.hadoop.ipc.Client.call(Client.java:1441)

        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)

        at com.sun.proxy.$Proxy17.listCachePools(Unknown Source)

        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.listCachePools(ClientNamenodeProtocolTranslatorPB.java:1276)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:260)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)

        at com.sun.proxy.$Proxy18.listCachePools(Unknown Source)

        at org.apache.hadoop.hdfs.protocol.CachePoolIterator.makeRequest(CachePoolIterator.java:55)

        at org.apache.hadoop.hdfs.protocol.CachePoolIterator.makeRequest(CachePoolIterator.java:33)

        at org.apache.hadoop.fs.BatchedRemoteIterator.makeRequest(BatchedRemoteIterator.java:77)

        at org.apache.hadoop.fs.BatchedRemoteIterator.makeRequestIfNeeded(BatchedRemoteIterator.java:85)

        at org.apache.hadoop.fs.BatchedRemoteIterator.hasNext(BatchedRemoteIterator.java:99)

        at org.apache.impala.catalog.CatalogServiceCatalog$CachePoolReader.run(CatalogServiceCatalog.java:243)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

        at java.lang.Thread.run(Thread.java:745)

--------------------------------------------------------------------------

Last error:

--------------------------------------------------------------------------

E0917 13:07:50.805730  3448 CatalogServiceCatalog.java:248] Error loading cache pools:

Java exception follows:

java.net.ConnectException: Call From quickstart.cloudera/172.17.0.2 to quickstart.cloudera:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

        at sun.reflect.GeneratedConstructorAccessor5.newInstance(Unknown Source)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

        at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:791)

        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:731)

        at org.apache.hadoop.ipc.Client.call(Client.java:1508)

        at org.apache.hadoop.ipc.Client.call(Client.java:1441)

        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)

        at com.sun.proxy.$Proxy17.listCachePools(Unknown Source)

        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.listCachePools(ClientNamenodeProtocolTranslatorPB.java:1276)

        at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:260)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)

        at com.sun.proxy.$Proxy18.listCachePools(Unknown Source)

        at org.apache.hadoop.hdfs.protocol.CachePoolIterator.makeRequest(CachePoolIterator.java:55)

        at org.apache.hadoop.hdfs.protocol.CachePoolIterator.makeRequest(CachePoolIterator.java:33)

        at org.apache.hadoop.fs.BatchedRemoteIterator.makeRequest(BatchedRemoteIterator.java:77)

        at org.apache.hadoop.fs.BatchedRemoteIterator.makeRequestIfNeeded(BatchedRemoteIterator.java:85)

        at org.apache.hadoop.fs.BatchedRemoteIterator.hasNext(BatchedRemoteIterator.java:99)

        at org.apache.impala.catalog.CatalogServiceCatalog$CachePoolReader.run(CatalogServiceCatalog.java:243)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

        at java.lang.Thread.run(Thread.java:745)

Caused by: java.net.ConnectException: Connection refused

        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)

        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)

        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)

        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)

        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:648)

        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:744)

        at org.apache.hadoop.ipc.Client$Connection.access$3000(Client.java:396)

        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1557)

        at org.apache.hadoop.ipc.Client.call(Client.java:1480)

--------------------------------------------------------------------------

Also just found this in "impalad.quickstart.cloudera.impala.log.ERROR.20180914-185252.3129":

--------------------------------------------------------------------------

Log file created at: 2018/09/14 18:52:52

Running on machine: quickstart.cloudera

Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

E0914 18:52:52.005654  3129 logging.cc:126] stderr will be logged to this file.

E0914 18:53:35.009699  3129 impala-server.cc:282] Could not read the root directory at hdfs://quickstart.cloudera:8020. Error was:

Call From quickstart.cloudera/172.17.0.2 to quickstart.cloudera:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

E0914 18:53:35.010474  3129 impala-server.cc:285] Aborting Impala Server startup due to improper configuration. Impalad exiting.

--------------------------------------------------------------------------

NOTE: This is the docker image without any modifications at all to the image.  I would assume it was tested and worked when it was created???

Any ideas?

Thanks!

[#4]

@pollard,

Would would assume that this would work out of the box, but it sounds like there are some problems starting certain services do to connectivity issues.  I'm having trouble figuring out how Cloudera Manager would show services in good health if services cannot connect to HDFS.

Can you share a screen shot of your Cloudera Manager Home screen?

Is HDFS running and in good health.  Can you run simple HDFS commands?

If you restart the cluster, does it seem to work fine?

Hue is a higher level service, so I sould start by taking a look at HDFS and make sure it is accessible.

I haven't played with the Docker version, so I hope those who have may have some feedback here, too.

[#5]

This is what I get when I try to restart:

----------------------------------------------------------------------------------------------------

service cloudera-scm-agent restart

By default, the Cloudera QuickStart VM run Cloudera's Distribution including

Apache Hadoop (CDH) under Linux's service and configuration management. If you

wish to migrate to Cloudera Manager, you must run one of the following

commands.

To use Cloudera Express (free), run:

    sudo /home/cloudera/cloudera-manager --express

This requires at least 8 GB of RAM and at least 2 virtual CPUs.

To begin a 60-day trial of Cloudera Enterprise with advanced management

features, run:

    sudo /home/cloudera/cloudera-manager --enterprise

This requires at least 10 GB or RAM and at least 2 virtual CPUs.

Be aware that after rebooting, it may take several minutes before Cloudera

Manager has started all of the services it manages and is ready to accept

connections from clients.

----------------------------------------------------------------------------------------------------

Now trying: 

/home/cloudera/cloudera-manager --express --force

Because I tried the "--express" option alone and it said I needed at least 8GB.  I believe the Docker container image allocates 4GB.  That may be the whole problem???

The restart did not work as it closed out most of the processes that run when the container is run and started up with others and not near as many.

This is the command I run to get the container going...

docker run -dit --hostname=quickstart.cloudera --name "cloudera" --privileged=true -p 8888:8888 88ed37152d45 /usr/bin/docker-quickstart

[#6]

@pollard,

Thanks for inspiring me to use docker on my Mac for the first time :-).

I ran the following to start things up:

docker run --hostname=quickstart.cloudera --privileged=true -t -i -p 8888:8888 -p 80:80 -p 7180:7180 6d2b3c820712 /usr/bin/docker-quickstart

Oddly, the first time I started without using "-p" options and found that I got connection refused messages when trying to use hdfs dfs -ls / in the container shell.

The second time I ran it with the command above, I could not run "hdfs" commands without error and get my file listings back.  Also, I can run Hive and Impala queries from Hue.  I didn't enable Cloudera Manager.

Honestly, all I did was

- docker run

- docker stop

- docker run...

I'm suspicious of the network in your container since it it seems (like in my first try) that process in the container cannot connect to "quickstart.cloudera"

[#7]

Thanks for your response.

Glad I could nudge you out of the dark and into the light.  🙂  And glad you're using a Mac!  :-). I worked for Apple for a couple of years.

I believe this is important but the Docker image I'm running is: cloudera-quickstart-vm-5.13.0-0-beta-docker

I didn't bother mapping port 80 since I have Apache running on my Mac and it's using port 80. My stop it and try to see if there's something to it.  Also, I mapped other ports but don't believe I mapped to 7180.  

Also, how did you run docker stop and then docker run without removing the container?  I know if I don't get rid of the container, docker run will error out and say it already exists when it tries to create the container.

-------------------------------------------------------------

So, I stopped Apache, used the port mappings you used and I seem to get a little more running.  I can now see the "impalad" process but still get the error with port 21050 when I log in and cannot get to Hive through the interface.  Interestingly, I see a listener port of "25010" which leads me to believe there's a typo in a configuration file???

I cannot execute the hdfs command you used successfully so I believe I've located the 2 hdfs processes by looking at a list of processes returned by filtering for "hdfs".  One process, which I believe is the primary hdfs node has a process listing of:

hdfs       276  4.3  3.7 1546804 77192 ?       Sl   13:42   0:50 /usr/java/jdk1.7.0_67-cloudera/bin/java -Dproc_datanode -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop-hdfs -Dhadoop.log.file=hadoop-hdfs-datanode-quickstart.cloudera.log -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,RFA -Djava.library.path=/usr/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -server -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.datanode.DataNode

What I believe is the secondary node has a listing of:

hdfs       610  2.5  3.6 1516348 74976 ?       Sl   13:43   0:28 /usr/java/jdk1.7.0_67-cloudera/bin/java -Dproc_secondarynamenode -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop-hdfs -Dhadoop.log.file=hadoop-hdfs-secondarynamenode-quickstart.cloudera.log -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,RFA -Djava.library.path=/usr/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode

Mapping the process IDs with the ports from netstat -tulpne I get:

tcp        0      0 0.0.0.0:50020               0.0.0.0:*                   LISTEN      497        3274353    276/java            

tcp        0      0 0.0.0.0:50090               0.0.0.0:*                   LISTEN      497        3274497    610/java            

These ports are very different than what seems like is needed and maybe why I cannot connect using the Hue interface???

I am currently searching the whole container to see if I can find where they are configured...

Also, given that the error I get back when trying to access the hdfs path comes back with this error:

18/09/18 13:57:59 WARN ipc.Client: Failed to connect to server: quickstart.cloudera/172.17.0.2:8020: try once and fail.

java.net.ConnectException: Connection refused

I assume that somewhere there is a file that says to try to connect to port 8020 for the "hdfs dfs" command.  There is no port listening on port 8020.  This is why I'm looking for the configuration file that maps the 2 hdfs processes to 50020 and 50090.  Am I thinking correctly?

BTW, I'm using the following command from root (/) to find the configuration for the above port:

find . -type f -exec grep -il '50090' {} \;

I would assume there is a configuration file with that port set???  I have seen applications use defaults set inside the code if it doesn't find a configuration parameter in a config file.

I changed my host /etc/hosts file to resolve quickstart.cloudera to localhost.  I can now use quickstart.cloudera:8888 to load the Hue GUI.

I get this error in the log: catalogd.quickstart.cloudera.impala.log.ERROR.20180918-132552.3090

Running on machine: quickstart.cloudera

Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

E0918 13:25:52.987486  3090 logging.cc:126] stderr will be logged to this file.

E0918 13:26:56.120744  3483 CatalogServiceCatalog.java:248] Error loading cache pools: 

Java exception follows:

java.net.ConnectException: Call From quickstart.cloudera/172.17.0.2 to quickstart.cloudera:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:791)

at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:731)

at org.apache.hadoop.ipc.Client.call(Client.java:1508)

at org.apache.hadoop.ipc.Client.call(Client.java:1441)

at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)

at com.sun.proxy.$Proxy17.listCachePools(Unknown Source)

at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.listCachePools(ClientNamenodeProtocolTranslatorPB.java:1276)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:260)

at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)

at com.sun.proxy.$Proxy18.listCachePools(Unknown Source)

at org.apache.hadoop.hdfs.protocol.CachePoolIterator.makeRequest(CachePoolIterator.java:55)

at org.apache.hadoop.hdfs.protocol.CachePoolIterator.makeRequest(CachePoolIterator.java:33)

at org.apache.hadoop.fs.BatchedRemoteIterator.makeRequest(BatchedRemoteIterator.java:77)

at org.apache.hadoop.fs.BatchedRemoteIterator.makeRequestIfNeeded(BatchedRemoteIterator.java:85)

at org.apache.hadoop.fs.BatchedRemoteIterator.hasNext(BatchedRemoteIterator.java:99)

at org.apache.impala.catalog.CatalogServiceCatalog$CachePoolReader.run(CatalogServiceCatalog.java:243)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.net.ConnectException: Connection refused

at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)

at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)

at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)

at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)

at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:648)

at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:744)

at org.apache.hadoop.ipc.Client$Connection.access$3000(Client.java:396)

at org.apache.hadoop.ipc.Client.getConnection(Client.java:1557)

at org.apache.hadoop.ipc.Client.call(Client.java:1480)

... 24 more

You're way more experienced in this than I, so I would appreciate you letting me know what you think about my trouble-shooting approach and does it seem like I'm headed in the right direction?

Also, I don't believe I've told you but when I do a "docker run" the first time and check the process list, there's a huge number of processes that come back compared to after I run "docker start"???

[#8 · Accepted solution]

@pollard,

Cool, I worked with Apple for a bit in 2005ish too. 🙂

To business, looks like it was Docker memory default to 2GB that was the cause.  I forgot that I noticed my container was sluggish so I bumped my Docker memory to 4GB.  I just prooved it by reverting to 2GB and then to 4GB as explained below:

Here is my session:

(1)

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
<none>              <none>              6d2b3c820712        19 hours ago        7GB

(2)

docker run --hostname=quickstart.cloudera --privileged=true -t -i -p 8888:8888 -p 80:80 -p 7180:7180 6d2b3c820712 /usr/bin/docker-quickstart

(3)

(in container shell)

# sudo -u hdfs hdfs dfs -ls /
18/09/18 16:57:09 WARN ipc.Client: Failed to connect to server: quickstart.cloudera/172.17.0.2:8020: try once and fail.
java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)

(4)

I log into Hue as cloudera:cloudera

Check Config shows all the failures you mentioned in your first post

(5)

start over

$ exit (in the terminal launched)

* basically, kill the running container

(6)

In Docker UI configuration, Advanced:

Set Memory to 4.0GiB

Click the Apply & Restart button

(7)

Run:

$ docker run --hostname=quickstart.cloudera --privileged=true -t -i -p 8888:8888 -p 80:80 -p 7180:7180 6d2b3c820712 /usr/bin/docker-quickstart

(8)

now everything works:

# sudo -u hdfs hdfs dfs -ls /
Found 6 items
drwxrwxrwx   - hdfs  supergroup          0 2017-10-23 23:46 /benchmarks
drwxr-xr-x   - hbase supergroup          0 2018-09-18 17:06 /hbase
drwxr-xr-x   - solr  solr                0 2017-10-23 23:48 /solr
drwxrwxrwt   - hdfs  supergroup          0 2018-09-18 17:06 /tmp
drwxr-xr-x   - hdfs  supergroup          0 2017-10-23 23:48 /user
drwxr-xr-x   - hdfs  supergroup          0 2017-10-23 23:48 /var

(9)

Hue is fine... can run queries

NOTE:  the 80 and 7180 ports aren't really necessary for exposure to host OS for this test case.

I hope this solves the issue for you too!

If so, it is pretty lame that lower memory would somehow let servers start and listen but then cause issues for networking to those listening ports.  I guess that is a welcome to docker for me 🙂

[#9]

2005 was close to the time I left.  You may have heard of a web based school administration product called Powerschool that Apple bought and sold later on.  I was one of the main software engineers on the Powerschool product.

U da man!

That was it all along...  Really weird things were happening though.  Logs saying impalad was running ok and it was no where to be found.  Oh well.

Thanks for the hard work.  Tell your manager I said to give you a raise.  🙂

[#10]

LOL... thanks man.

I was in the .Mac group (what became iCloud eventually) so I'm sorry to say I didn't know too much about Powerschool.

Looks like a pretty cool idea that I will check out more later.

I'm just very glad that memory was the only limitation.  Thanks for sticking with it trying to get it to work; I think most would have given up, so that says a lot 😉

Feel free to share anything else crazy that happens.