Member since
04-27-2016
9
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1930 | 06-15-2016 07:23 AM |
06-15-2016
09:10 AM
Hi All, We have six physical machines. Which way of cluster setup is good? Having those physical Machines as it is (or) Create multiple VMs on top of those machines and create a big cluster? Those machines are highly available machines with more than 450 GB of RAM. Please Suggest me!
... View more
Labels:
06-15-2016
07:46 AM
The Optimized Row Columnar (ORC) file format provides a highly efficient way to store Hive data. It just like a File to store group of rows called stripes, along with auxiliary information in a file footer. It just a storage format, nothing to do with ORC/Spark.
... View more
06-15-2016
07:23 AM
I found it on RedHat documentation, There is no one right answer to this question.
It depends on your needs and requirements. 😞 /dev/: This is the name of the
directory in which all device files reside. Because partitions reside on hard
disks, and hard disks are devices, the files representing all possible
partitions reside in /dev/ Red Hat Enterprise Linux uses a
naming scheme that is file-based, with file names in the form of /dev/xxyN. Where, xx: The first two letters of the
partition name indicate the type of device on which the partition resides,
usually sd. y: This letter indicates which
device the partition is on. For example, /dev/sda for the first hard disk,
/dev/sdb for the second, and so on. N: The final number denotes the
partition. The first four (primary or extended) partitions are numbered 1
through 4. Logical partitions start at 5. So, for example, /dev/sda3 is the
third primary or extended partition on the first hard disk, and /dev/sdb6 is
the second logical partition on the second hard disk. In Red Hat Enterprise Linux each
partition is used to form part of the storage necessary to support a single set
of files and directories. Mounting a partition makes its storage available
starting at the specified directory (known as a mount point). For example, if partition /dev/sda5 is mounted on /usr/, that would
mean that all files and directories under /usr/ physically reside on /dev/sda5.
So the file /usr/share/doc/FAQ/txt/Linux-FAQ would be stored on /dev/sda5,
while the file /etc/gdm/custom.conf would not. It is also possible that one or
more directories below /usr/ would be mount points for other partitions. For
instance, a partition (say, /dev/sda7) could be mounted on /usr/local/, meaning
that /usr/local/man/whatis would then reside on /dev/sda7 rather than
/dev/sda5. Generally speaking, the
disk spacing for /dev partition depends on number and size of the partitions
(both primary and logical)to be used by operating system. However, there is no
one right answer to this question. It depends on your needs and requirements. Conclusion: Red Hat recommends that, unless you have a reason for
doing otherwise, you should at least create the following partitions: swap,
/boot/, and / (root).
... View more
06-14-2016
08:48 AM
Hi @Mats Johansson, Thanks for your reply! Yes, /dev directory contains the special device files for all the devices that's it. However, do we need to explicitly mention any sizing numbers for the partitions /dev/sda1, /dev/sda2 ..etc. What kind of information will store under those /dev partitions w.r.t HDP configuration setup. Thanks!
... View more
06-14-2016
07:18 AM
Very thankful for this write-up.These are the things to be noted before setting-up the cluster, It solve most of the admin related hurdles. One question, We have 2* 80 GB 5G SATA SSD RAID-1 Hard-drive for Master node, in this blog-post, it suggests 32 GB for /dev partition. Please suggest the recommended partition by considering the above hard-drive limits
... View more
06-14-2016
06:51 AM
1 Kudo
Hi All, We have 2* 80 GB 6G SATA SSD RAID-1 Hard-drive for Master node, According to Hortonworks recommendation in one of the blog-post, it suggests 32 GB for /dev partition. Our client is not interested in providing 32 GB for /dev partition, since the maximum disk capacity is 2*80 GB. Full Master configuration details: 2x E5-2670V2 2.5GHz-25MB 10C CPU 448GB PC3-14900L RAM 2x 80GB 6G SATA SSD RAID 1 I would like to ask why we need 32 GB for /dev partition for Hadoop master nodes, also please suggest the recommended partition by considering the above hard-drive limits. Thanks !
... View more
Labels:
- Labels:
-
Apache Hadoop
06-14-2016
06:39 AM
Hi All, We need to setup a 6 node Hadoop cluster with one additional Edge node (where Hadoop client components need to be installed like, Hue, SQOOP). We are planning to make it as a kerberized cluster, additionally we are integrating each cluster node(All 6 nodes) with Microsoft AD using SSSD pattern. How do we enable Kerberos on Edge node, since it will not co-exist with the cluster, and not available under Ambari UI scope. Please suggest the way to solve this.
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop