I need clarification about Metastore URI configuration of Hive data server. The problem is that in the official cloudera documentation you can see that it is a requirement to block external applications accessing Hive Metastore in order to secure cluster with Sentry:   https://www.cloudera.com/documentation/enterprise/latest/topics/sg_sentry_service_config.html#concept_z5b_42s_p4__section_lvc_4g4_rp   Block the external applications from accessing the Hive metastore: In the Cloudera Manager Admin Console, select the Hive service. On the Hive service page, click the Configuration tab. In the search well on the right half of the Configuration page, search for Hive Metastore Access Control and Proxy User Groups Override to locate the hadoop.proxyuser.hive.groups parameter and click the plus sign. Enter hive into the text box and click the plus sign again. Enter hue into the text box. Enter sentry into the text box. Click Save Changes.   ODI is also an external application so its direct access requirement to Metastore contradicts with this requirement.   Why are we setting Metastore URI in Hive data server? What is it used for? If it is a requirement, how can we explain this contradiction?   Another problem is that we are configuring Metastore HA by default in our BDA. However how can we configure ODI Metadata Uri to support metadata HA configuration? Is there any way to write a Uri supporting connecting more than one Metastore? If not, how can we explain this situation?   ... View more

We're using cloudera with anaconda parcel on bda production cluster . I tried to execute pyspark code that imports pyarrow package , then i faced with error below . Traceback (most recent call last):  File "/opt/cloudera/parcels/SPARK2-2.3.0.cloudera2-1.cdh5.13.3.p0.316101/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 140, in require_minimum_pyarrow_version  File "/opt/cloudera/parcels/Anaconda-3.6.5_2/lib/python3.6/site-packages/pyarrow/__init__.py", line 47, in <module>    from pyarrow.lib import cpu_count, set_cpu_count ImportError: libboost_system.so.1.66.0: cannot open shared object file: No such file or directory During handling of the above exception, another exception occurred: Traceback (most recent call last):  File "lbf_livArea_scr_2.py", line 51, in <module>    @pandas_udf(schema, PandasUDFType.GROUPED_MAP)  File "/opt/cloudera/parcels/SPARK2-2.3.0.cloudera2-1.cdh5.13.3.p0.316101/lib/spark2/python/lib/pyspark.zip/pyspark/sql/udf.py", line 45, in _create_udf  File "/opt/cloudera/parcels/SPARK2-2.3.0.cloudera2-1.cdh5.13.3.p0.316101/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 143, in require_minimum_pyarrow_version ImportError: PyArrow >= 0.8.0 must be installed; however, it was not found. Also output of conda list is below . [ihsany@gbbdap02 ~]$ dzdo /opt/cloudera/parcels/Anaconda/bin/conda list |grep arrow arrow-cpp                 0.9.0            py36h1ae9da6_7    <unknown> pyarrow                   0.9.0                    py36_1    <unknown> [ihsany@gbbdap02 ~]$ dzdo /opt/cloudera/parcels/Anaconda/bin/conda list |grep boost libboost                  1.65.1               habcd387_4    <unknown> ... View more

can we deploy the HttpFS role on more than one node? is it best practice?   we had a problem with one of the namenodes (on node02) in our cluster and it was down for almost 20 hours. once node02  was up, the active namenode ,on node01, started to alert that its not syncronized after restart of the namenode on node02 the alert stopped now active namenode is on node01 standby namenode is on node02 we can see that new logs are created on opt/hadoop/dfs/nn/current  on the actine namenode on node01 but no new files . on the standby namenode no node02 - is it OK ?? ... View more