Impalad COORDINATOR_ONLY with HDFS Short-Circuit Read

Question

Hi,

I'm trying to deploy a few COORDINATOR_ONLY impalad on edge nodes in my cluster (CDH 5.16.1). HDFS short-circuit read are enabled on the cluster and works fine.When starting the COORDINATOR_ONLY nodes, I'm getting following error:

Invalid short-circuit reads configuration: - Impala cannot read or execute the parent directory of dfs.domain.socket.path Aborting Impala Server startup due to improper configuration. Impalad exiting.

Is there a way to run COORDINATOR_ONLY impalad with hdfs short circuit reads enabled.

Thanks in advance!

EricL · Accepted Answer

Thanks Lars for pointing it out.So solution is to disable HDFS shortcircuit read for coordinator only impala daemons: a) create a new role group and add all coordinator only impala daemon hosts to this groupb) Go to "CM -> Cluster -> Impala services -> Configuration";c) Add the following property into "Impala Daemon HDFS Advanced Configuration Snippet (Safety Valve)" for the new role group that you just created: dfs.client.read.shortcircuit falsed) Save the changes and restart the Impala Daemon instance that are affected.Hope above can help.CheersEric

Gomathinayagam · Answer

Can you check if the short-circuit read is configured as per the Documentation HERE ?

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.