Member since
08-24-2018
9
Posts
0
Kudos Received
0
Solutions
06-29-2019
03:13 PM
Hello, We have a topic with 3 partitions where 1.5gb data getting pushed per minute but the whole data getting pushed to one partition(0) and other 2 partitions(1,2) not getting any data. I believe Kafka by default take cares of distributing data between brokers and partitions. Please help out with your thoughts. Thanks, Arshad
... View more
Labels:
04-18-2019
03:07 PM
Hi All, We have druid on our hdp cluster hdfs as deep storage and set of nodes being used for that druid cluster. Is it possible to have another separate druid cluster with another set of nodes in same hdp cluster ? If you have different nodes for brokers/middle/history/coordination/overlord nodes for new druid cluster in same hdp cluster is it possible to have 2 druid clusters isolated in same hdp and ambari. My question comes with zookeeper leader election and how middle manager nodes will route the segments will point to particular hdfs deep storage. Please help. Thanks, Arshad
... View more
Labels:
04-16-2019
05:42 PM
Hi, I have created a simple workflow which imports data from Oracle to hive using sqoop and it is running fine. As part of testing multi tenancy/new namespace, I have put the workflow.xml, coordinator.xml in new namespace and changed namenode to pointing to new namepsace tried running the workflow using oozie. Workflow fails with error JA017 could not look up launched Hadoop job I'd. Could any one please advice what could be wrong. Thanks in advance. Thanks, Arshad
... View more
Labels:
04-09-2019
02:44 PM
We have added a new namespace to our existing one namespace cluster. We have services like hive/spark/hbase/Kafka/ which are using default namespace for storage and processing. We would like to have different hive metastore for our new namespace. Is it possible to have different hive metastore for different namespaces ? I see that we can give new namespace as a location when you are creating hive tables but does any one implemented federation/ multi tenant for hbase? I think federation is not applicable for Kafka as it stores the streams in the host where Kafka brokers are running not in hdfs. I really appreciate if some one can guide me with the above questions. Thanks, Arshad
... View more
Labels:
04-09-2019
12:35 AM
We have added a new namespace to our existing one namespace cluster. We have services like hive/spark/hbase/Kafka/ which are using default namespace for storage and processing. We would like to have different hive metastore for our new namespace. Is it possible to have different hive metastore for different namespaces ? I see that we can give new namespace as a location when you are creating hive tables but does any one implemented federation/ multi tenant for hbase? I think federation is not applicable for Kafka as it stores the streams in the host where Kafka brokers are running not in hdfs. I really appreciate if some one can guide me with the above questions. Thanks, Arshad Thanks, Atehad
... View more
Labels:
04-05-2019
03:32 PM
I think you need to enable hive support when you create a spark session with the value of hive metastore Uris from hive-site.xml where your hive metastore is saved. Then you should be able to see existing tables of hive from spark .
... View more
04-04-2019
06:32 AM
Thanks @Jay for your suggestions. can we have a different ambari UI for each namespace or Is it like same ambari for the all the namespaces we create ? Is it possible to have different configurations at service level unlike host level. when hive is shared for multi tenants we have same metastore location that is one location for multiple namespaces. When we would like to have a different metastore for hive for new namespace , couldn't see any option for that. Same case with hbase region servers and kafka brokers storage. Is it really possible to federate services like hive, hbase and kafka ? I have not tried with Spark yet. Any suggestions will be great. Thanks, Arshad
... View more
04-04-2019
03:46 AM
Hi, We have recently added a new namespace to existing hdp cluster using ambari. Since we have only one ambari we will have same configurations and properties for both the tenants or namespaces . Can we have different configurations and properties set at service level like hive for different namespaces ? Can we have different ambari UI for each namespace so we can have the different configurations and properties ?
... View more
- Tags:
- Federation
03-28-2019
01:35 PM
I am trying to setup multi tenancy cluster for HDP and HDF. I dont find any documentation from hortonworks. Can any one share any documentation/links which i can refer ?
... View more