How to connect to Postgres in HDF running in VirtualBox on Windows?

[#1]

While awaiting setup of a new development environment, we were trying to test some concepts in an HDF sandbox running in VirtualBox (version 5.2.16r123759(Qt5.6.2)) on Windows 7 (Windows 7 Enterprise Service pack 1 with 32 GB RAM and 64-bit Operating System). At a high level, our test was to transfer JSON records from Kafka to PostGres.

I found HDF 3.1 already has Postgres 9.2.23 installed. I did the following:

1) Port 5432 was added to VirtualBox Port Forwarding Rules, as shown in the attachment.

2) Windows Hosts file was updated, as shown in the attachment.

3) Postgres /var/lib/pgsql/data/pg_hba.conf was modified according to https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.1/bk_installing-hdf/content/postgres-remote-c...

4) Postgresql service was successfully stopped and restarted

5) From the shell-in-a-box console, psql was used to create a mytest login and database, as shown in the attachment.

6) From Windows, psql aborts with error "psql: server closed the connection unexpectedly...", as shown in the attachment.

I am able to connect into the sandbox using WinSCP on port 2202 to transfer files with File Protocal: SCP.

Kindly let me know any suggestions to allow Windows psql to connect to Postgres in HDF running in VirtualBox,

Harold

[#2]

@Harold Brown

In addition to the above steps you will also need to add a port inside the Docker container to expose the port 5432 something as described in the following article (Although the following article if for HDP the same concept should be applicable for HDF sandbox as well with regard to adding a port).

https://community.hortonworks.com/articles/65914/how-to-add-ports-to-the-hdp-25-virtualbox-sandbox.h...

[#3]

Thank you very much for your explanation and guidance, Jay! Unfortunately, sandbox.service is not listed in HDF (just sandbox-ambari-services.service and sandbox-splash-web-server.service) nor does the /root/start_scripts/start_sandbox.sh file exist as mentioned in the article. Any suggestions on where to find documentation for HDF 3.1 (Linux sandbox-hdf.hortonworks.com 4.17.2-1.el7.elrepo.x86_64 #1 SMP Sat Jun 16 11:18:11 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux)?

[#4]

If the end goal is to transfer Kafka to Postgres, you have access to NiFi.

Otherwise, exposing the internal Postgres server that is used for Hive, Ambari, Oozie, and other services is probably not a good idea.

It would be recommended to run a standalone Postgres server to minimize the blast-radius of failure and maintain service uptime.