Cloudera Community
egarelnabi
user rank
Guru

Re: Upgrade from hdp2.4 to 2.5 is failing as hive ...

by Guru in Support Questions
‎05-10-2017 02:56 PM
‎05-10-2017 02:56 PM
Can you please share the error log ... View more

Re: Spark DataFrame - difference between sort and ...

by Guru in Support Questions
‎05-10-2017 02:25 PM
‎05-10-2017 02:25 PM
@Dinesh Chitlangia OrderBy is just an alias for the Sort function and should give the same result. The below is from the Spark documentation: /** * Returns a new Dataset sorted by the given expressions. * This is an alias of the `sort` function. * * @group typedrel * @since 2.0.0 */ @scala.annotation.varargs def orderBy(sortCol: String, sortCols: String*): Dataset[T] = sort(sortCol, sortCols : _*) Both will order across partitions. To get an understanding of how Spark does a sort take a look at the explanation in the link below: http://stackoverflow.com/questions/32887595/how-does-spark-achieve-sort-order If you would like to sort within a partition then you can use repartitionAndSortWithinPartitions. https://spark.apache.org/docs/1.6.0/api/java/org/apache/spark/rdd/OrderedRDDFunctions.html#repartitionAndSortWithinPartitions(org.apache.spark.Partitioner) ... View more

Re: What are the ways of accessing Hive from remot...

by Guru in Support Questions
‎05-02-2017 04:10 AM
2 Kudos
‎05-02-2017 04:10 AM
2 Kudos
@Tech Gig 1) For access from a remote server, your only option is JDBC. See link below for details. https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBC 2) Essentially, Hive Thrift Server is the same as HiveServer2. HiveServer2 uses the Thrift RPC protocol, hence the naming. The Thrift Client is is your Beeline or any client connecting to HiveServer2 through the Thrift RPC protocol. For an explanation of the evolution of Hive from HiveServer1 to HiveServer2 (Thrift) take a look at the below link: http://toodey.com/2015/07/25/what-is-beeline-in-latest-hiveserver2-difference-between-hiveserver1-and-hiveserver2/ For info about HiveServer2 and Thrift RPC: https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2#SettingUpHiveServer2-HiveServer2 For info on Beeline and other clients: https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients ... View more

Re: how to backup hive tables ?

by Guru in Support Questions
‎03-22-2017 03:02 PM
1 Kudo
‎03-22-2017 03:02 PM
1 Kudo
@Nikhil Pawar 1) Stop Hive on the target cluster 2) Distcp all the necessary files on HDFS to the secondary cluster. 3) Take a SQL dump of your Hive Metastore (which is in MySQL or Postgres). 4) Restore the SQL dump on your target cluster. 5) Use the Hive Metatool "-updateLocation" command on the target cluster to change the Metastore URIs https://cwiki.apache.org/confluence/display/Hive/Hive+MetaTool 6) Start Hive on the target cluster To make the process easier, assuming this is a one-time thing, I suggest that you copy the entire Metastore rather than trying to pick and choose certain tables. While being selective is possible, it will add a bit more complexity to your process. ... View more

Re: How to make Apache Kylin work with HDP 2.5.3

by Guru in Support Questions
‎12-22-2016 05:36 PM
1 Kudo
‎12-22-2016 05:36 PM
1 Kudo
Take a look at the response/instructions here: https://community.hortonworks.com/questions/1293/how-to-make-kylin-work-with-hdp-23.html It references HDP 2.3, but still applies for HDP 2.5. It contains links to the "installation guide" and "quick-start" instructions. It will also give you an idea of some of the "undocumented" changes that may need to be made. note: You will probably not need to download the Binary for HBase 1.1 since it should already be included in the newer versions of Kylin. ... View more

Re: Add tag in Atlas on Sandbox HDP 2.5

by Guru in Support Questions
‎12-22-2016 02:27 PM
‎12-22-2016 02:27 PM
Actually you mean: API "Terms" = Business Term API "Traits" = Tag ... View more

Re: Can we authenticate users in Nifi using Ldap? ...

by Guru in Support Questions
‎12-22-2016 03:02 AM
‎12-22-2016 03:02 AM
@sprakash Take a look at the response here: https://community.hortonworks.com/questions/53495/how-to-integrate-nifi-and-ldap-and-how-to-add-diff.html ...and yes, you need to configure it to run over HTTPS ... View more

Re: Atlas: How to pull the Tag attribute value?

by Guru in Support Questions
‎12-21-2016 05:39 AM
‎12-21-2016 05:39 AM
The easiest way would be to do a full text search not DSL. Just search for the attribute value itself. You can do this through the UI or through the API as follows: GET http://<atlas-server- host:port>/api/atlas/discovery/search/fulltext?query={query_string} ... View more

Re: Atlas: How to pull the Tag attribute value?

by Guru in Support Questions
‎12-21-2016 03:46 AM
‎12-21-2016 03:46 AM
What you provided above is the response for: GET http://<atlas-server-host:port>/api/atlas/types/{typeName} Please post the response for: GET http://<atlas-server-host:port>/api/atlas/entities/{entity_guid} It should have the "ID123456" under "values" right after the "typeName":"UserID" If you don't know the entity_guid, navigate to the entity you want ("A" in this case) in the Atlas UI. The alphanumeric string at the end of the URL is the entity_guid. (http://<atlas-server-host:port>/index.html#!/detailPage/{entity_guid}) ... View more

Re: Atlas: How to pull the Tag attribute value?

by Guru in Support Questions
‎12-21-2016 12:59 AM
‎12-21-2016 12:59 AM
@H G If you want to get the Tag/Trait info then use: GET http://<atlas-server-host:port>/api/atlas/types/{trait_name} If you want to get the Tags/Traits associated with an Entity, with details, use: GET http://<atlas-server-host:port>/api/atlas/entities/{guid} ... View more
Contact Me
Online
Offline
Last Visited
‎08-14-2019 09:54 AM
Community Statistics
Member Since ‎10-06-2015 09:21 PM
Last Visited ‎08-14-2019 09:54 AM
Posts 273
Kudos received 202