Member since
04-03-2019
962
Posts
1743
Kudos Received
146
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
11678 | 03-08-2019 06:33 PM | |
4980 | 02-15-2019 08:47 PM | |
4221 | 09-26-2018 06:02 PM | |
10701 | 09-07-2018 10:33 PM | |
5740 | 04-25-2018 01:55 AM |
07-04-2016
09:40 PM
3 Kudos
@Sumit Chauhan It looks like metadata.broker.list is deprecated and we need to use bootstrap.servers instead. I found below github code snippet where it says metadata.broker.list is deprecated https://github.com/apache/samza/blob/master/samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala val bootsrapServers = {
if(properties.containsKey("metadata.broker.list"))
warn("Kafka producer configuration contains 'metadata.broker.list'. This configuration is deprecated . Samza has been upgraded " +
"to use Kafka's new producer API. Please update your configurations based on the documentation at http://kafka.apache.org/documentation.html#newproducerconfigs")
Option(properties.get("bootstrap.servers"))
.getOrElse(throw new SamzaException("No bootstrap servers defined in config for %s." format systemName))
.asInstanceOf[String] bootstrap.servers http://kafka.apache.org/documentation.html metadata.broker.list http://kafka.apache.org/081/configuration.html Hope this information helps!
... View more
07-04-2016
09:30 PM
2 Kudos
@Dale Bradman I believe you have working kerberos configuration, by 'hue' user, you are getting list of files from '/user/' on hdfs which is expected as other users can list the files under /user/ from hdfs. When you login to hue UI, it is trying to create directory under /user/ on hdfs which only 'hdfs' user can do. Can you please create /user/<user-name> directory on HDFS by 'hdfs' user and then try to login via HUE UI again?
... View more
06-30-2016
09:09 PM
@hoda moradi Can you please check oozie launcher logs? It should give you more hint. Please check below link: https://community.hortonworks.com/articles/9148/troubleshooting-an-oozie-flow.html
... View more
06-29-2016
10:32 PM
6 Kudos
@hoda moradi Please add/modify below properties in your core-site.xml hadoop.proxyuser.hive.groups=*
hadoop.proxyuser.hive.hosts=* Restart required services via Ambari UI and retry your query, it should work.
... View more
06-29-2016
05:05 PM
4 Kudos
@james.jones - I don't think so as you are modifying authorization property, it should not consider sql grants once you set authorization to Ranger. Please also have a look at https://community.hortonworks.com/questions/28136/will-ranger-hive-plugin-override-existing-hive-sec.html
... View more
06-27-2016
11:34 AM
@Manikandan Durairaj Can you please confirm if you have your user directory(/user/mani) setup in your dtap filesystem? if its not there then please go ahead and create one.
... View more
06-27-2016
11:07 AM
3 Kudos
@Sagar Shimpi Oozie has feature to purge older jobs from database. by default it's 30 days. Actions related to long running coordinators do not purged until co-ordinator completes, ( example - if you have coordinator running for 6 months, then all the related workflows will be there in database for 6 months ) Will the purge scripts remove the running workflows and coordinators? Ans - No it will not. Below are the respective properties and their default values. Useful link - http://stackoverflow.com/questions/31946225/oozie-not-cleaning-up-old-jobs-from-oozie-database oozie.service.PurgeService.older.than 30 Completed workflow jobs older than this value, in days, will be purged by the PurgeService. oozie.service.PurgeService.coord.older.than 7 Completed coordinator jobs older than this value, in days, will be purged by the PurgeService. oozie.service.PurgeService.bundle.older.than 7 Completed bundle jobs older than this value, in days, will be purged by the PurgeService. oozie.service.PurgeService.purge.old.coord.action false Whether to purge completed workflows and their corresponding coordinator actions of long running coordinator jobs if the completed workflow jobs are older than the value specified in oozie.service.PurgeService.older.than. oozie.service.PurgeService.purge.limit 100 Completed Actions purge - limit each purge to this value oozie.service.PurgeService.purge.interval 3600 Interval at which the purge service will run, in seconds.
... View more
06-26-2016
06:38 PM
@Jobin George Thanks! I have corrected the table name. That's correct! Only one config made by 'kuldeep' will be deleted.
... View more
06-25-2016
09:24 AM
@Mohana Murali Gurunathan - Please remove localhost and add hostname of your namenode in the configuration for fs.defaultFS. current value - localhost:8020 recommended value - <hostname-of-namenode>:8020
... View more
06-25-2016
09:10 AM
5 Kudos
@Jobin George Good question! 🙂 Please find answers below: Note - Please take backup of your ambari database before proceeding with suggested steps . 1) Can we delete a service configuration version in Ambari? Below are the tables responsible for holding service configuration version information widget_layout
clusterconfigmapping
widget_layout_user_widget
serviceconfigmapping
serviceconfig Suppose I have modified my hdfs configuration by 'kuldeep' user and I want to remove it completely from ambari UI. Please execute below queries in ambari DB ambari=> select * from serviceconfig where user_name like '%kuldeep%';
service_config_id | cluster_id | service_name | version | create_timestamp | stack_id | user_name | group_id | note
-------------------+------------+--------------+---------+------------------+----------+-----------+----------+---------
407 | 2 | HDFS | 15 | 1466844482492 | 1 | kuldeep | | kuldeep
(1 row)
ambari=> delete from serviceconfigmapping where service_config_id=407;
DELETE 11
ambari=> delete from serviceconfig where user_name like '%kuldeep%';
DELETE 1
ambari=> select * from clusterconfigmapping where user_name like '%kuldeep%';
cluster_id | type_name | version_tag | create_timestamp | selected | user_name
------------+------------+----------------------+------------------+----------+-----------
2 | hadoop-env | version1466844481864 | 1466844482490 | 1 | kuldeep
(1 row)
ambari=> delete from clusterconfigmapping where user_name like '%kuldeep%';
DELETE 1
ambari=> select * from widget_layout where layout_name like '%kuldeep%';
id | layout_name | section_name | scope | user_name | display_name | cluster_id
-----+------------------------+--------------+-------+-----------+-------------------------+------------
152 | kuldeep_hdfs_dashboard | HDFS_SUMMARY | USER | kuldeep | Standard HDFS Dashboard | 2
(1 row)
ambari=> delete from widget_layout_user_widget where widget_layout_id='152';
DELETE 10
ambari=> delete from widget_layout where layout_name like '%kuldeep%';
DELETE 1 . 2) How many versions of configs will Ambari save in history? I don't think there is any limit, as this version information gets stored in the Ambari DB. . 3) Does Ambari have any views/method to alert for Job/application Failures? No. We don't have any views for YARN Job browser except for TEZ applications. Hope this information helps.
... View more