@Ye Jun I have uploaded a sample with the "pom.xml" for version check in the github repo: https://github.com/jaysensharma/MiddlewareMagicDemos/tree/master/HDP_Ambari/Hive/HiveJavaClient ... View more

@Ye Jun Try using a different version of "hive-jdbc" driver jar for example i am using "http://repo.hortonworks.com/content/repositories/releases/org/apache/hive/hive-jdbc/1.2.1000.2.5.3.0-37/" ... View more

@Ye Jun I see that some issues were reported with the "hive-jdbc" driver version jar. So can you please clarify which exact version of this jar are you using. As i am able to run my test successfully at my end (for example i am using "http://repo.hortonworks.com/content/repositories/releases/org/apache/hive/hive-jdbc/1.2.1000.2.5.3.0-37/" ) ... View more

@Aditya Sharma I have developed a very simple HBase Client code completely using maven so it would be easier for you to test and run. Please refer to the demo: https://github.com/jaysensharma/MiddlewareMagicDemos/tree/master/HDP_Ambari/HBase_Client Few changes you will need to make in the code. config.set("hbase.zookeeper.quorum", "erie3.example.com,erie1.example.com,erie4.example.com,erie2.example.com"); config.set("hbase.zookeeper.property.clientPort", "2181"); config.set("zookeeper.znode.parent", "/hbase-unsecure"); config.addResource(new Path("/PATH/TO/HBase_Client/src/main/resources/hbase-site.xml")); config.addResource(new Path("/PATH/TO/HBase_Client/src/main/resources/core-site.xml")); config.addResource(new Path("/PATH/TO/HBase_Client/src/main/resources/hdfs-site.xml")); . Please use your own "hbase-site.xml","core-site.xml" and "hdfs-site.xml" files. . Then run it as following: How to build and run? # cd /PATH/TO/HBase_Client # mvn clean install exec:java ... View more

@Sanaz Janbakhsh For HDF i responded to another thread just now. https://community.hortonworks.com/questions/90400/minimum-hardware-and-clustering-requirements-for-h.html#answer-90414 ... View more

@Sanaz Janbakhsh Regarding your other queries. like Regarding VM vs Physical server: VM based pros: 1. 'Easier' managing nodes. Some IT infrastructure teams insist on VMs even if you want to map 1 physical node to 1 virtual node because all their other infrastructure is based on VMs. 2. Taking advantage of NUMA and memory locality. There are some articles on this from virtual infrastructure providers that you can take a look at. VM based disadvantages: (example may vary based on your usage and cluster) 1. Overhead. As an example, if you are running 4VMs per physical node, you are running 4 OS, 4 Datanode services, 4 Nodemanagers, 4 ambari-agents, 4 metrics collectors and 4 of any other worker services instead of one. These services will have overhead compared to running 1 of each. 2. Data Locality and redundancy. Now, there is support to know physical nodes, so no two replicas go into same physical node but that is extra configuration. You might run into virtual disk performance problems if they are not configured properly. Given a choice, I prefer using Physical servers. However, its not always your choice. In those cases, make sure you try to get following. 1. Explicit virtual disk to physical disk mapping. Say you have 2 VMs per physical node and each physical node has 16 data drives. Make sure to split 8 drives to one VM and 8 more to another VM. This way, physical disks are not shared between VMs. 2. Don't go for more than 2 VMs per physical node. This is so you minimize overhead from the services running. . For a very basic cluster setup you can have simple two-node, non-secure, unicast cluster comprised of three instances of NiFi: The NCM, Node 1, Node 2 Please see: https://docs.hortonworks.com/HDPDocuments/HDF2/HDF-2.0.2/bk_administration/content/clustering.html ... View more

@Sanaz Janbakhsh Some of the S/W and H/W requirements can be found in the following link: https://docs.hortonworks.com/HDPDocuments/HDF2/HDF-2.0.2/bk_ambari-installation/content/system-requirements.html ... View more

@Eun-seop Yu If you have 3 node cluster you need three VMs (Computers/Host machines) Every host should have a unique Hostname and the output of the following command on every host should return the correct FQDN # hostname -f - Every host should have an "/etc/hosts" file with same content in it so that it can resolve each other. Example: 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 10.10.10.101 master 10.10.10.102 slave1 10.10.10.103 slave1 Same entry should be present in master/slave1/slave2 hosts "etc/hosts" file. Do not delete the first two line (127.0.0.1 and ::1) of entry from the /etc/hosts file as that is default. - Then you need to setup Password less ssh between from Master host to the slave hosts. - Then install Ambari and configure Cluster. Please see Hortonworks Video on the same: HDP 2.4 Multinode Hadoop Installation using Ambari: https://www.youtube.com/watch?v=zutXwUxmaT4 And https://www.youtube.com/watch?v=bC6Tpfdyy5k ... View more