Hi, Ambari agent in one of my HDF nodes keeps disconnecting. When I issue "amabri-agent status" I see that it's running, however Ambari gets not data from this node. When I "tail -f /var/log/ambari-agent/ambari-agent.out" I see these errors Controller.py:415 - Unable to reconnect to https://<server>:8441/agent/v1/heartbeat/<server>; (attempts=59918, details=local variable 'data' referenced before assignment) Any idea on how to correct this? Thanks ... View more

Hi How can I monitor a spark streaming job ? how can I be noticed if there's a problem ? What happens if the job crash ? any fault tolerance capabilities available here ? ... View more

Hi, When I start a Spark Streaming job, all resources are allocated even if the job is in standby and don't use them. I guess this is the normal behaviour but can someone validate this ? I've heard of dynamic resource allocation. Can this help here ? Thanks ... View more

I have data in HBase in cluster A, and I need the same data in Hive in another cluster B. What's the best approach to achieve this ? (performance, file format, robustness, ..) ... View more

Does hive.merge.tezfiles applies when using parquet data in Hive? ... View more

Hi, I Have a Spark Streaming job reading data from Kafka and writing them to Hive or HDFS. I am getting lot of small files. What the different approaches to deal with it ? I am thinking of a periodic job that create a new table T2 from table T1, delete T1, then copy data from T2 to T1. However, data will be unavailable for a short period. Are their other internal Hive or Spark tools to deal with this issue ? I've seen hive.merge.sparkfiles, HiveStreaming but I am not sure how these can help. Thanks ... View more

Hi, The documentation speaks about two configuration for Ranger with LDAP 1- Usersync : http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_security/content/ranger_user_sync_settings.html 2- Authentication : http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_security/content/configure_ranger_authentication.html What's the difference between these two steps? what the role of each configuration? is there a use case where we use different LDAPs for these two steps? Thanks ... View more

Hi, I've been working on Ranger all day and I got confused. First, we have internal (created with the UI) and external (ranger-sync) users. What's the role of internal users in settings where we have LDAP or AD ? Ranger stores users in two tables (x_portal_users and x_user). From my understanding, when I sync users they got added to both tables (ie: users can login to portal, and can be used for creating policies). Right ? why separate the two tables then ? Thank you for your help ... View more