Member since
10-04-2016
243
Posts
280
Kudos Received
43
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
909 | 01-16-2018 03:38 PM | |
5292 | 11-13-2017 05:45 PM | |
2306 | 11-13-2017 12:30 AM | |
1227 | 10-27-2017 03:58 AM | |
27397 | 10-19-2017 03:17 AM |
08-31-2017
01:31 AM
2 Kudos
When I am logged in as a non hdfs user, I am able to set storage policies but not able to get the storage policy for a path I just set. I am able to get storage policies only when I login as hdfs user. Is this expected behavior or is their a solution for this ? I am using HDP 2.6.1
... View more
Labels:
- Labels:
-
Apache Hadoop
08-31-2017
01:26 AM
2 Kudos
@Sagar Shimpi As we moved into 2017, we can now remove storage policy for a certain path! hdfs storagepolicies -unsetStoragePolicy -path <path>
... View more
08-30-2017
07:07 PM
5 Kudos
@Viswa
Here are the 2 major aspects on which they differ:
1. Connection:
The Hive CLI, which connects directly to HDFS and the Hive Metastore, and can be used only on a host with access to those services.
Beeline, which connects to HiveServer2 and requires access to only one .jar file: hive-jdbc-<version>-standalone.jar .
2. Authentication Hive CLI uses only Storage Based Authentication Beeline uses SQL standard-based authorization or Ranger-based authorization. Thus greater security. It is better to use Beeline for the above reasons than Hive CLI (I believe it will soon be deprecated). Read here for greater understanding on beeline : https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_data-access/content/beeline-vs-hive-cli.html
... View more
08-30-2017
03:02 PM
4 Kudos
@Freemon Johnson Login to the host which has Ambari. This will list all components in all versions of HDP that are there in your system. yum list installed | grep @HDP- You can filter using a specific version, ex for HDP 2.6.1.0 yum list installed | grep @HDP-2.6.1.0
... View more
08-28-2017
09:13 PM
This is one of the really useful things out here!
... View more
08-24-2017
10:06 PM
2 Kudos
For redundancy, dfs.namenode.name.dir has paths to two different disks. Thus there are FSImage files on two different disks so that on failure of one disk, the fs image is available from another disk and the Failover Controller is not triggered just because of one disk failure. Typically, the HDP upgrade will create a backup of FS Images from the path mentioned in dfs.namenode.dir. In this case it will create a backup of the redundant copy as well, which will be expensive in terms of storage and time. Is there a way, where we can guide ambari upgrade process to choose one of the redundant files instead of all of them ?
... View more
Labels:
08-24-2017
08:38 PM
6 Kudos
You can use the collect_set UDAF function in hive. select name, collect_set(city) from people group by name; collect_Set will remove all duplicates. If you need the duplicates then you must use "collect_list". Read here for official documentation and details.
... View more
08-23-2017
04:57 PM
@Jay SenSharma - Thank you for prompt response. I see this only works for Beeline version 2.3.0 and latest. I am currently using 1.2.1 and thus this will have no effect. However, I appreciate your help. I will try for a workaround for version 1.2.1, if nothing else works I will upgrade to 2.3.0 and accept your answer.
... View more
08-23-2017
04:21 PM
1 Kudo
When using HiveCLI, if we SET hive.cli.print.current.db = true; Then the command prompt for Hive CLI displays the current database name until the end of the session as shown below: hive mydb > I tried using the same property in Beeline but had no effect. Is there a different way of achieving this in Beeline ?
... View more
Labels:
- Labels:
-
Apache Hive