Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDP Component working in deep

avatar
Rising Star

Hi Community Team,

I'm trying to dig deeper in HDP component how HDP component works?

How one compent call to other other component?

How can i achieve it?

Could you please help me to find the hadoop in deep?

Thanks,

Vinay K

1 ACCEPTED SOLUTION

avatar
Super Collaborator

What component are you asking about? What are you trying to achieve?

They typically call each other over combinations of separate protocols.

- HDFS and YARN interact via RPC/IPC.

- Ambari Server and Agents are over HTTP & REST. Ambari also needs JDBC connections to the backing database.

- Hive, Hbase, and Spark can use Thrift Server. The Hive metastore uses JDBC.

- Kafka has its own TCP protocol.

I would suggest starting on a specific component for the use case(s) you want. Hadoop itself is only comprised of HDFS & YARN + MapReduce

View solution in original post

1 REPLY 1

avatar
Super Collaborator

What component are you asking about? What are you trying to achieve?

They typically call each other over combinations of separate protocols.

- HDFS and YARN interact via RPC/IPC.

- Ambari Server and Agents are over HTTP & REST. Ambari also needs JDBC connections to the backing database.

- Hive, Hbase, and Spark can use Thrift Server. The Hive metastore uses JDBC.

- Kafka has its own TCP protocol.

I would suggest starting on a specific component for the use case(s) you want. Hadoop itself is only comprised of HDFS & YARN + MapReduce