Bad Credentials on Accumulo Tracer

[#1]

Every part of Accumulo starts, except for the tracer. I've attached he logfiles for stdin & stdout below. I'm running Ubuntu 14.04. I don't know why I'm receiving bad credentials, I've tried adding a new "tracer" user with the correct password on my master & I tried changing the password through the CLI to match the password input on Ambari for Accumulo Root Password.

Stdin

 

 File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'cat /var/lib/ambari-agent/tmp/pass | ACCUMULO_CONF_DIR=/usr/hdp/current/accumulo-tracer/conf/server /usr/hdp/current/accumulo-client/bin/accumulo shell -u root -f /var/lib/ambari-agent/tmp/cmds' returned 1. Password: ****************
2017-01-16 14:35:17,831 [trace.DistributedTrace] INFO : SpanReceiver org.apache.accumulo.tracer.ZooTraceClient was loaded successfully.
2017-01-16 14:35:17,901 [shell.Shell] ERROR: org.apache.accumulo.core.client.AccumuloSecurityException: Error BAD_CREDENTIALS for user root - Username or Password is Invalid

Stdout

2017-01-16 14:35:14,749 - File['/usr/hdp/current/hadoop-client/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2017-01-16 14:35:14,749 - File['/usr/hdp/current/hadoop-client/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2017-01-16 14:35:14,754 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop'}
2017-01-16 14:35:14,759 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2017-01-16 14:35:15,026 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.5.3.0-37
2017-01-16 14:35:15,029 - Checking if need to create versioned conf dir /etc/hadoop/2.5.3.0-37/0
2017-01-16 14:35:15,031 - call[('ambari-python-wrap', u'/usr/bin/conf-select', 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.3.0-37', '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
2017-01-16 14:35:15,058 - call returned (1, '/etc/hadoop/2.5.3.0-37/0 exist already', '')
2017-01-16 14:35:15,059 - checked_call[('ambari-python-wrap', u'/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.3.0-37', '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False}
2017-01-16 14:35:15,085 - checked_call returned (0, '')
2017-01-16 14:35:15,086 - Ensuring that hadoop has the correct symlink structure
2017-01-16 14:35:15,086 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2017-01-16 14:35:15,088 - Directory['/usr/hdp/current/accumulo-tracer/conf'] {'owner': 'accumulo', 'group': 'hadoop', 'create_parents': True, 'mode': 0755}
2017-01-16 14:35:15,089 - Directory['/usr/hdp/current/accumulo-tracer/conf/server'] {'owner': 'accumulo', 'group': 'hadoop', 'create_parents': True, 'mode': 0700}
2017-01-16 14:35:15,090 - XmlConfig['accumulo-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/current/accumulo-tracer/conf/server', 'mode': 0600, 'configuration_attributes': {}, 'owner': 'accumulo', 'configurations': ...}
2017-01-16 14:35:15,102 - Generating config: /usr/hdp/current/accumulo-tracer/conf/server/accumulo-site.xml
2017-01-16 14:35:15,102 - File['/usr/hdp/current/accumulo-tracer/conf/server/accumulo-site.xml'] {'owner': 'accumulo', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0600, 'encoding': 'UTF-8'}
2017-01-16 14:35:15,119 - Directory['/var/run/accumulo'] {'owner': 'accumulo', 'group': 'hadoop', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2017-01-16 14:35:15,120 - Directory['/var/log/accumulo'] {'owner': 'accumulo', 'group': 'hadoop', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2017-01-16 14:35:15,126 - File['/usr/hdp/current/accumulo-tracer/conf/server/accumulo-env.sh'] {'content': InlineTemplate(...), 'owner': 'accumulo', 'group': 'hadoop', 'mode': 0644}
2017-01-16 14:35:15,126 - Writing File['/usr/hdp/current/accumulo-tracer/conf/server/accumulo-env.sh'] because contents don't match
2017-01-16 14:35:15,127 - PropertiesFile['/usr/hdp/current/accumulo-tracer/conf/server/client.conf'] {'owner': 'accumulo', 'group': 'hadoop', 'properties': {'instance.zookeeper.host': u'hadoop1.hadoop123.com:2181', 'instance.name': u'hdp-accumulo-instance', 'instance.zookeeper.timeout': u'30s'}}
2017-01-16 14:35:15,132 - Generating properties file: /usr/hdp/current/accumulo-tracer/conf/server/client.conf
2017-01-16 14:35:15,133 - File['/usr/hdp/current/accumulo-tracer/conf/server/client.conf'] {'owner': 'accumulo', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None}
2017-01-16 14:35:15,136 - Writing File['/usr/hdp/current/accumulo-tracer/conf/server/client.conf'] because contents don't match
2017-01-16 14:35:15,136 - File['/usr/hdp/current/accumulo-tracer/conf/server/log4j.properties'] {'content': ..., 'owner': 'accumulo', 'group': 'hadoop', 'mode': 0644}
2017-01-16 14:35:15,137 - TemplateConfig['/usr/hdp/current/accumulo-tracer/conf/server/auditLog.xml'] {'owner': 'accumulo', 'template_tag': None, 'group': 'hadoop'}
2017-01-16 14:35:15,140 - File['/usr/hdp/current/accumulo-tracer/conf/server/auditLog.xml'] {'content': Template('auditLog.xml.j2'), 'owner': 'accumulo', 'group': 'hadoop', 'mode': None}
2017-01-16 14:35:15,141 - TemplateConfig['/usr/hdp/current/accumulo-tracer/conf/server/generic_logger.xml'] {'owner': 'accumulo', 'template_tag': None, 'group': 'hadoop'}
2017-01-16 14:35:15,144 - File['/usr/hdp/current/accumulo-tracer/conf/server/generic_logger.xml'] {'content': Template('generic_logger.xml.j2'), 'owner': 'accumulo', 'group': 'hadoop', 'mode': None}
2017-01-16 14:35:15,144 - TemplateConfig['/usr/hdp/current/accumulo-tracer/conf/server/monitor_logger.xml'] {'owner': 'accumulo', 'template_tag': None, 'group': 'hadoop'}
2017-01-16 14:35:15,147 - File['/usr/hdp/current/accumulo-tracer/conf/server/monitor_logger.xml'] {'content': Template('monitor_logger.xml.j2'), 'owner': 'accumulo', 'group': 'hadoop', 'mode': None}
2017-01-16 14:35:15,147 - File['/usr/hdp/current/accumulo-tracer/conf/server/accumulo-metrics.xml'] {'content': StaticFile('accumulo-metrics.xml'), 'owner': 'accumulo', 'group': 'hadoop', 'mode': 0644}
2017-01-16 14:35:15,148 - TemplateConfig['/usr/hdp/current/accumulo-tracer/conf/server/tracers'] {'owner': 'accumulo', 'template_tag': None, 'group': 'hadoop'}
2017-01-16 14:35:15,150 - File['/usr/hdp/current/accumulo-tracer/conf/server/tracers'] {'content': Template('tracers.j2'), 'owner': 'accumulo', 'group': 'hadoop', 'mode': None}
2017-01-16 14:35:15,150 - TemplateConfig['/usr/hdp/current/accumulo-tracer/conf/server/gc'] {'owner': 'accumulo', 'template_tag': None, 'group': 'hadoop'}
2017-01-16 14:35:15,152 - File['/usr/hdp/current/accumulo-tracer/conf/server/gc'] {'content': Template('gc.j2'), 'owner': 'accumulo', 'group': 'hadoop', 'mode': None}
2017-01-16 14:35:15,153 - TemplateConfig['/usr/hdp/current/accumulo-tracer/conf/server/monitor'] {'owner': 'accumulo', 'template_tag': None, 'group': 'hadoop'}
2017-01-16 14:35:15,154 - File['/usr/hdp/current/accumulo-tracer/conf/server/monitor'] {'content': Template('monitor.j2'), 'owner': 'accumulo', 'group': 'hadoop', 'mode': None}
2017-01-16 14:35:15,155 - TemplateConfig['/usr/hdp/current/accumulo-tracer/conf/server/slaves'] {'owner': 'accumulo', 'template_tag': None, 'group': 'hadoop'}
2017-01-16 14:35:15,156 - File['/usr/hdp/current/accumulo-tracer/conf/server/slaves'] {'content': Template('slaves.j2'), 'owner': 'accumulo', 'group': 'hadoop', 'mode': None}
2017-01-16 14:35:15,157 - TemplateConfig['/usr/hdp/current/accumulo-tracer/conf/server/masters'] {'owner': 'accumulo', 'template_tag': None, 'group': 'hadoop'}
2017-01-16 14:35:15,159 - File['/usr/hdp/current/accumulo-tracer/conf/server/masters'] {'content': Template('masters.j2'), 'owner': 'accumulo', 'group': 'hadoop', 'mode': None}
2017-01-16 14:35:15,159 - TemplateConfig['/usr/hdp/current/accumulo-tracer/conf/server/hadoop-metrics2-accumulo.properties'] {'owner': 'accumulo', 'template_tag': None, 'group': 'hadoop'}
2017-01-16 14:35:15,166 - File['/usr/hdp/current/accumulo-tracer/conf/server/hadoop-metrics2-accumulo.properties'] {'content': Template('hadoop-metrics2-accumulo.properties.j2'), 'owner': 'accumulo', 'group': 'hadoop', 'mode': None}
2017-01-16 14:35:15,167 - File['/var/lib/ambari-agent/tmp/cmds'] {'content': InlineTemplate(...), 'owner': 'accumulo', 'group': 'hadoop', 'mode': 0600}
2017-01-16 14:35:15,167 - Writing File['/var/lib/ambari-agent/tmp/cmds'] because it doesn't exist
2017-01-16 14:35:15,167 - Changing owner for /var/lib/ambari-agent/tmp/cmds from 0 to accumulo
2017-01-16 14:35:15,167 - Changing group for /var/lib/ambari-agent/tmp/cmds from 0 to hadoop
2017-01-16 14:35:15,168 - Changing permission for /var/lib/ambari-agent/tmp/cmds from 644 to 600
2017-01-16 14:35:15,168 - File['/var/lib/ambari-agent/tmp/pass0'] {'content': InlineTemplate(...), 'owner': 'accumulo', 'group': 'hadoop', 'mode': 0600}
2017-01-16 14:35:15,169 - Writing File['/var/lib/ambari-agent/tmp/pass0'] because it doesn't exist
2017-01-16 14:35:15,169 - Changing owner for /var/lib/ambari-agent/tmp/pass0 from 0 to accumulo
2017-01-16 14:35:15,169 - Changing group for /var/lib/ambari-agent/tmp/pass0 from 0 to hadoop
2017-01-16 14:35:15,169 - Changing permission for /var/lib/ambari-agent/tmp/pass0 from 644 to 600
2017-01-16 14:35:15,170 - File['/var/lib/ambari-agent/tmp/pass'] {'content': InlineTemplate(...), 'owner': 'accumulo', 'group': 'hadoop', 'mode': 0600}
2017-01-16 14:35:15,170 - Writing File['/var/lib/ambari-agent/tmp/pass'] because it doesn't exist
2017-01-16 14:35:15,170 - Changing owner for /var/lib/ambari-agent/tmp/pass from 0 to accumulo
2017-01-16 14:35:15,170 - Changing group for /var/lib/ambari-agent/tmp/pass from 0 to hadoop
2017-01-16 14:35:15,171 - Changing permission for /var/lib/ambari-agent/tmp/pass from 644 to 600
2017-01-16 14:35:15,171 - Execute['cat /var/lib/ambari-agent/tmp/pass | ACCUMULO_CONF_DIR=/usr/hdp/current/accumulo-tracer/conf/server /usr/hdp/current/accumulo-client/bin/accumulo shell -u root -f /var/lib/ambari-agent/tmp/cmds'] {'not_if': 'ambari-sudo.sh su accumulo -l -s /bin/bash -c \'cat /var/lib/ambari-agent/tmp/pass0 | ACCUMULO_CONF_DIR=/usr/hdp/current/accumulo-tracer/conf/server /usr/hdp/current/accumulo-client/bin/accumulo shell -u root -e "userpermissions -u ambari-qa"\'', 'user': 'accumulo'}

Command failed after 1 tries

[#2]

"I've tried adding a new "tracer" user with the correct password on my master"

Just to make sure you understand, the Accumulo tracer user is not a linux user. It is an Accumulo user.

You need to make sure that such an Accumulo user exists with the correct password as per your Ambari configuration. This is normally done automatically for you -- not sure how you got into this state.

[#3]

Ah, okay. Thanks. I don't know either. Any help into reconfiguring Accumulo? After I received the credentials error is when I changed the passwords & created the "new" linux server account, rather than the Accumulo shell account.

[#4]

@Josh Elser

"You need to make sure that such an Accumulo user exists with the correct password as per your Ambari configuration"

Could you please be more specific on how to do that ?

[#5]

Accumulo user administration is covered at http://accumulo.apache.org/1.9/accumulo_user_manual.html#_user_administration

And defining the Accumulo user and credentials is covered in http://accumulo.apache.org/1.9/accumulo_user_manual.html#_tracers