Can we install or integrate Application servers like tomcat, websphere with hadoop cluster? Is there is any possibility?

[#1]

Hi,

I am just wondering if we can install/integrate the web application servers on hadoop cluster?

Thanks,

Rahul

[#2 · Accepted solution]

@Rahul Reddy Kamuru

Yes. You can install and integrate. However, you can also have your web and application servers residing on their dedicated infrastructure and accessing various services from the Hadoop ecosystem via JDBC, ODBC or REST API. From the Hadoop lingo point of view, they would reside on the EDGE nodes and act as Hadoop clients. Installing on the Hadoop cluster while is possible, it will make operations on the Hadoop cluster to impact also those web and application servers, making everything more complicated also on upgrades etc. Separation of concerns generates a clean architecture. However, it is not unusual to have Tomcat installed on the cluster to allow development of various services which will either access data stored in HDFS, Hive, HBase etc. or even submit jobs to Spark. Those will be mainly "glue" services that will allow building some data pipelines or enable data services for BI tools residing outside of the cluster, e.g. Tableau, MicroStrategy, ZoomData etc.

If this response helped, please vote and accept it as a best answer.

[#3]

Thanks Stanca for providing the information. Can you provide any supporting material or sources how can we implement this?

Thanks,

Rahul

[#4]

@Rahul Reddy Kamuru

I answered the broad question with a broad answer :). I assume that you have a ton of specific questions. One would ask what are you trying to achieve, but that would reveal the ton of questions.

I assume that you already know to install Websphere and Tomcat and that is not in scope for your question.

I assume that you already know how to write a web services and that is not in scope of your question.

If you wish to setup a JDBC connection to your service of choice in Hadoop ecosystem, check that specific service documentation. It would require a specific JDBC driver and the connection string is pretty much standard with mnor specifics for each service.

If you wish to learn more about to write applications that will access various services, aside from reading the developer guides for services in the Hadoop ecosystem, I recommend this link with tutorials: http://hortonworks.com/tutorials/

I'd like to re-iterate that is not a good idea to install those web servers in the hadoop cluster for reasons that I already presented in my original response. You'd rather install them on separate servers which will access remotely services in the hadoop cluster.

+++

If you feel that I answered the original question and my response helped, please don't forget to accept it as the best answer. Thanks.