Member since
10-06-2016
24
Posts
12
Kudos Received
0
Solutions
12-26-2016
10:08 PM
Another detail, when I restart Ambari agent it becomes green and everything works as expected. After some time, the problem occurs again
... View more
12-26-2016
10:00 PM
@Sagar Shimpi Thanks for the answer. The agent and server have the same version as
it's a new cluster. I am not fan uninstalling/reinstalling without understanding what's the reason (RCA).
... View more
12-26-2016
09:59 PM
@Constantin Stanca
Thanks for the answer. The agent and server have the same version as it's a new cluster. I just checked it. There are no errors in /var/log/messages neither.
... View more
12-26-2016
04:32 PM
1 Kudo
Hi, Ambari agent in one of my HDF nodes keeps disconnecting. When I issue "amabri-agent status" I see that it's running, however Ambari gets not data from this node. When I "tail -f /var/log/ambari-agent/ambari-agent.out" I see these errors Controller.py:415 - Unable to reconnect to https://<server>:8441/agent/v1/heartbeat/<server>; (attempts=59918, details=local variable 'data' referenced before assignment) Any idea on how to correct this? Thanks
... View more
Labels:
12-14-2016
08:13 PM
@Alex Gauthier
Do you have more information on how to use it with Spark Streaming ? I have 15 executors
... View more
12-14-2016
07:09 PM
2 Kudos
Hi How can I monitor a spark streaming job ? how can I be noticed if there's a problem ? What happens if the job crash ? any fault tolerance capabilities available here ?
... View more
Labels:
12-14-2016
07:07 PM
1 Kudo
Hi, When I start a Spark Streaming job, all resources are allocated even if the job is in standby and don't use them. I guess this is the normal behaviour but can someone validate this ? I've heard of dynamic resource allocation. Can this help here ? Thanks
... View more
Labels:
12-12-2016
08:55 AM
2 Kudos
I have data in HBase in cluster A, and I need the same data in Hive in another cluster B. What's the best approach to achieve this ? (performance, file format, robustness, ..)
... View more
Labels:
12-12-2016
08:46 AM
1 Kudo
Hi, I have a NiFi flow that consumes lot of memory. How can I diagnosis the flow to isolate the problem and find the processor that
... View more
Labels:
11-24-2016
12:16 AM
Does hive.merge.tezfiles applies when using parquet data in Hive?
... View more
Labels:
11-23-2016
09:13 PM
I would write a job for doing this if it was HDFS only. Since I am using Hive, I am wondering what happens when my job are deleting the original files and users are querying Hive ?!!
... View more
11-23-2016
07:59 PM
2 Kudos
Hi, I Have a Spark Streaming job reading data from Kafka and writing them to Hive or HDFS. I am getting lot of small files. What the different approaches to deal with it ? I am thinking of a periodic job that create a new table T2 from table T1, delete T1, then copy data from T2 to T1. However, data will be unavailable for a short period. Are their other internal Hive or Spark tools to deal with this issue ? I've seen hive.merge.sparkfiles, HiveStreaming but I am not sure how these can help. Thanks
... View more
Labels:
10-25-2016
10:07 PM
2 Kudos
Hi Is it possible to copy HDFS snapshots to another cluster and use them (via distCP for instance)? What does this add compared to copy data directly (not snapshots) via distCP for DR? Thanks
... View more
Labels:
10-06-2016
10:34 PM
Thanks for clarifying @Abdelkrim Hadjidj
Indeed, this is different from what I understood from Chethana response. Can someone else confirm where users/group mapping is done ? Ranger or Hadoop service ?
... View more
10-06-2016
09:58 PM
@slachterman So if I have user x in LDAP and I use usersync, I won't be able to login with user x Credentials ?
... View more
10-06-2016
08:37 PM
Thanks @slachterman So by default we use memberof,ismemberof to get the user group. Can we set it to other value such as uniquemember ?
... View more
10-06-2016
07:59 PM
Hi, The documentation speaks about two configuration for Ranger with LDAP 1- Usersync : http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_security/content/ranger_user_sync_settings.html 2- Authentication : http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_security/content/configure_ranger_authentication.html What's the difference between these two steps? what the role of each configuration? is there a use case where we use different LDAPs for these two steps? Thanks
... View more
Labels:
10-06-2016
07:55 PM
Hi, I've been working on Ranger all day and I got confused. First, we have internal (created with the UI) and external (ranger-sync) users. What's the role of internal users in settings where we have LDAP or AD ? Ranger stores users in two tables (x_portal_users and x_user). From my understanding, when I sync users they got added to both tables (ie: users can login to portal, and can be used for creating policies). Right ? why separate the two tables then ? Thank you for your help
... View more
Labels:
10-06-2016
03:41 PM
Thanks @Chethana Krishnakumar. So, I understand that user/group information is already in Ranger after the sync, it will used to authorize/deny access for each request. So it's important to have fresh data in Ranger. How synchronization is kept between Ranger and LDAP? manual ? scheduled ? realtime ?
... View more
10-06-2016
01:26 PM
@Sagar Shimpi Thanks
... View more
10-06-2016
11:07 AM
Hi @Sagar Shimpi Thank for your quick reply. Do I need to delete users manually? Let's say user x has been deleted from LDAP. Can Ranger detects this automatically and delete user x from its DB?
... View more
10-06-2016
11:04 AM
1 Kudo
Hello What criteria Ranger usses to look up for user in LDAP? Which attribute (memberof, uniquemember) ?
... View more
Labels:
10-06-2016
10:58 AM
Hi, I want to use Ranger-usersync to import users to ranger with a file (export from LDAP) : https://cwiki.apache.org/confluence/display/RANGER/File+Source+User+Group+Sync+process I can see that insert and updates work. However, this does support delete user? The same question if I used usersync with LDAP? I already saw this article but I can not see if the bug has been corrected : https://community.hortonworks.com/articles/49127/does-ranger-user-group-sync-support-removing-users.html Thanks
... View more
Labels:
10-06-2016
10:49 AM
Hi, When a user try to
read a file from HDFS in a Kerberised cluster, he contacts NN and presents its
token. I am trying to understand how the user's groups are checked against
Ranger policies. Does Kerberos token
contains the groups the user belongs to? Or does ranger looks
to the user/group mapping? If ranger does this, is it internal or through LDAP?
... View more
Labels: