Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

HDP Component working in deep

Expert Contributor

Hi Community Team,

I'm trying to dig deeper in HDP component how HDP component works?

How one compent call to other other component?

How can i achieve it?

Could you please help me to find the hadoop in deep?

Thanks,

Vinay K

1 ACCEPTED SOLUTION

Super Collaborator

What component are you asking about? What are you trying to achieve?

They typically call each other over combinations of separate protocols.

- HDFS and YARN interact via RPC/IPC.

- Ambari Server and Agents are over HTTP & REST. Ambari also needs JDBC connections to the backing database.

- Hive, Hbase, and Spark can use Thrift Server. The Hive metastore uses JDBC.

- Kafka has its own TCP protocol.

I would suggest starting on a specific component for the use case(s) you want. Hadoop itself is only comprised of HDFS & YARN + MapReduce

View solution in original post

1 REPLY 1

Super Collaborator

What component are you asking about? What are you trying to achieve?

They typically call each other over combinations of separate protocols.

- HDFS and YARN interact via RPC/IPC.

- Ambari Server and Agents are over HTTP & REST. Ambari also needs JDBC connections to the backing database.

- Hive, Hbase, and Spark can use Thrift Server. The Hive metastore uses JDBC.

- Kafka has its own TCP protocol.

I would suggest starting on a specific component for the use case(s) you want. Hadoop itself is only comprised of HDFS & YARN + MapReduce