question Re: Rack topology script not working ERROR: script /etc/hadoop/conf/topology.sh returned 0 values wh in Archives of Support Questions (Read Only)

Rack topology script not working ERROR: script /etc/hadoop/conf/topology.sh returned 0 values when 1

sid2707 — Mon, 17 Sep 2018 13:06:20 GMT

Hi,

I have written my own topology script and made required configuration in cloudera manager>hdfs>configurations>net.topology.script.file.name property. But the rack topology is not updated and could see ERROR in namenode log as "script /etc/hadoop/conf/topology.sh returned 0 values when 1 were expected.". Please help to resolve the issue.

topology.sh

#!/bin/bash

nodearg=$1 #get the first argument

while [ $# -gt 0 ]; do

for line in `cat topology.data`; do #read line from topology.data file

node=$(echo $line|awk -F ',' '{print $1}') #parse the data and get the hostname to compare

result=""

if [ $nodearg = $node ]; then #compare the hostname in the file with the argument

result=$(echo $line|awk -F ',' '{print $2}') #parse the file again to recive the rack details for the host

break;

else

result="/default/rack-0"

done

shift

echo $result

done

topology.data

hdp-1.hdp.com,/default/rack-1

hdp-2.hdp.com,/default/rack-2

hdp-3.hdp.com,/default/rack-3

19.1.0.13,/default/rack-1

19.1.0.14,/default/rack-2

19.1.0.15,/default/rack-3

Output:

$ ./topology.sh hdp-1. hdp.com

/default/rack-1

$ ./topology.sh 19.1.0.14

/default/rack-2

Thanks and regards

Sidharth

Re: Rack topology script not working ERROR: script /etc/hadoop/conf/topology.sh returned 0 values wh

Tomas79 — Mon, 17 Sep 2018 14:50:37 GMT

The first explanation could be that the script is called without argument maybe?

Re: Rack topology script not working ERROR: script /etc/hadoop/conf/topology.sh returned 0 values wh

sid2707 — Mon, 17 Sep 2018 16:23:46 GMT

So, in that case it will satisfy the first if condition. Do you know how
hadoop invoke topology script? I mean the parameters it passes along with
script file.

Re: Rack topology script not working ERROR: script /etc/hadoop/conf/topology.sh returned 0 values wh

bgooley — Mon, 17 Sep 2018 16:38:42 GMT

@sid2707,

Sounds like you may need to add some debugging to your shell script to identify what values are being evaluated and why no results are being returned.

Re: Rack topology script not working ERROR: script /etc/hadoop/conf/topology.sh returned 0 values wh

bgooley — Mon, 17 Sep 2018 16:40:01 GMT

@sid2707,

This page has a pretty good summary of how this works:

https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-common/RackAwareness.html

Re: Rack topology script not working ERROR: script /etc/hadoop/conf/topology.sh returned 0 values wh

sid2707 — Tue, 18 Sep 2018 06:33:29 GMT

Hi,

Thanks for your response and help.

Everytime i make changes in configs it re-deploy the configurations which was deleting my topology script. So i pushed my script to /mnt/topology/ directory and also tweak the script a bit

It look like below now

topology.sh

#!/bin/bash

while [ $# -gt 0 ]; do

nodearg=$1 #get the first argument

for line in `cat /mnt/topology/topology.data`; do #read line from topology.data file

node=$(echo $line|awk -F ',' '{print $1}') #parse the data and get the hostname to compare

result=""

if [ $nodearg = $node ]; then #compare the hostname in the file with the argument

result=$(echo $line|awk -F ',' '{print $2}') #parse the file again to recive the rack details for the host

break;

else

result="/default/rack-0"

done

shift

echo $result

done

Re: Rack topology script not working ERROR: script /etc/hadoop/conf/topology.sh returned 0 values wh

bgooley — Tue, 18 Sep 2018 18:08:46 GMT

@sid2707,

Very cool and thanks for sharing your solution!