Username and password revealed in error logs while accessing Hive metastore tables

[#1]

Hello I am using HUE 3.9.0 with MapR cluster.

When I click on the Metastore tables tab with wrong credentails it errors out but reveals the password of the user.

Is there way to hide the password.Consider LDAP authentication enabled.

Below is the exception

[10/Apr/2017 17:36:10 -0700] conf         ERROR    The application won't work without a running HiveServer2.
Traceback (most recent call last):
  File "/opt/mapr/hue/hue-3.9.0/apps/beeswax/src/beeswax/conf.py", line 151, in config_validator
    server.get_databases()
  File "/opt/mapr/hue/hue-3.9.0/apps/beeswax/src/beeswax/server/dbms.py", line 129, in get_databases
    return self.client.get_databases()
  File "/opt/mapr/hue/hue-3.9.0/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 956, in get_databases
    return [table[col] for table in self._client.get_databases()]
  File "/opt/mapr/hue/hue-3.9.0/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 587, in get_databases
    res = self.call(self._client.GetSchemas, req)
  File "/opt/mapr/hue/hue-3.9.0/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 562, in call
    session = self.open_session(self.user)
  File "/opt/mapr/hue/hue-3.9.0/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 531, in open_session
    raise QueryServerException(Exception('Bad status for request %s:\n%s' % (req, res)), message=message)
QueryServerException: Bad status for request TOpenSessionReq(username='mapr', password='mapr', client_protocol=6, configuration={'hive.server2.proxy.user': u'mapr'}):
TOpenSessionResp(status=TStatus(errorCode=0, errorMessage='Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient', sqlState=None, infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient:13:12', 'org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:266', 'org.apache.hive.service.cli.CLIService:openSessionWithImpersonation:CLIService.java:202', 'org.apache.hive.service.cli.thrift.ThriftCLIService:getSessionHandle:ThriftCLIService.java:405', 'org.apache.hive.service.cli.thrift.ThriftCLIService:OpenSession:ThriftCLIService.java:300', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1253', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1238', 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', 'java.lang.Thread:run:Thread.java:745', '*java.lang.RuntimeException:java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient:21:8', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:83', 'org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36', 'org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63', 'java.security.AccessController:doPrivileged:AccessController.java:-2', 'javax.security.auth.Subject:doAs:Subject.java:415', 'org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1595', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59', 'com.sun.proxy.$Proxy10:open::-1', 'org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:258', '*java.lang.RuntimeException:java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient:26:5', 'org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:522', 'org.apache.hive.service.cli.session.HiveSessionImpl:open:HiveSessionImpl.java:137', 'sun.reflect.GeneratedMethodAccessor9:invoke::-1', 'sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43', 'java.lang.reflect.Method:invoke:Method.java:606', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78', '*java.lang.RuntimeException:Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient:32:6', 'org.apache.hadoop.hive.metastore.MetaStoreUtils:newInstance:MetaStoreUtils.java:1523', 'org.apache.hadoop.hive.metastore.RetryingMetaStoreClient:<init>:RetryingMetaStoreClient.java:86', 'org.apache.hadoop.hive.metastore.RetryingMetaStoreClient:getProxy:RetryingMetaStoreClient.java:132', 'org.apache.hadoop.hive.metastore.RetryingMetaStoreClient:getProxy:RetryingMetaStoreClient.java:104', 'org.apache.hadoop.hive.ql.metadata.Hive:createMetaStoreClient:Hive.java:3078', 'org.apache.hadoop.hive.ql.metadata.Hive:getMSC:Hive.java:3097', 'org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:503', '*java.lang.reflect.InvocationTargetException:null:35:3', 'sun.reflect.GeneratedConstructorAccessor10:newInstance::-1', 'sun.reflect.DelegatingConstructorAccessorImpl:newInstance:DelegatingConstructorAccessorImpl.java:45', 'java.lang.reflect.Constructor:newInstance:Constructor.java:526', 'org.apache.hadoop.hive.metastore.MetaStoreUtils:newInstance:MetaStoreUtils.java:1521', '*org.apache.hadoop.hive.metastore.api.MetaException:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)\n\tat org.apache.thrift.transport.TSocket.open(TSocket.java:187)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)\n\tat org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)\n\tat sun.reflect.GeneratedConstructorAccessor10.newInstance(Unknown Source)\n\tat sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\n\tat java.lang.reflect.Constructor.newInstance(Constructor.java:526)\n\tat org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3078)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3097)\n\tat org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)\n\tat org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:137)\n\tat sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)\n\tat org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)\n\tat org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat javax.security.auth.Subject.doAs(Subject.java:415)\n\tat org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)\n\tat org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)\n\tat com.sun.proxy.$Proxy10.open(Unknown Source)\n\tat org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:258)\n\tat org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:202)\n\tat org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:405)\n\tat org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:300)\n\tat org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1253)\n\tat org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1238)\n\tat org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)\n\tat org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)\n\tat org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)\n\tat org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: java.net.ConnectException: Connection refused (Connection refused)\n\tat java.net.PlainSocketImpl.socketConnect(Native Method)\n\tat java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)\n\tat java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)\n\tat java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)\n\tat java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)\n\tat java.net.Socket.connect(Socket.java:576)\n\tat org.apache.thrift.transport.TSocket.open(TSocket.java:182)\n\t... 38 more\n:38:2', 'org.apache.hadoop.hive.metastore.HiveMetaStoreClient:open:HiveMetaStoreClient.java:466', 'org.apache.hadoop.hive.metastore.HiveMetaStoreClient:<init>:HiveMetaStoreClient.java:236', 'org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient:<init>:SessionHiveMetaStoreClient.java:74'], statusCode=3), sessionHandle=None, configuration=None, serverProtocolVersion=7)

[#2]

CDH 5.5 password redaction is enabled by default .

 Please check below link for more information. 

https://blog.cloudera.com/blog/2015/06/new-in-cdh-5-4-sensitive-data-redaction/

https://www.cloudera.com/documentation/enterprise/5-4-x/topics/sg_redaction.html#concept_nnn_m13_fr

[#3]

Hello @csguna I have HUE running on MapR cluster.

Thanks for the information though.

[#4]

Hello @csguna

What will be the correct config in hue.ini for log redaction for hiding passwords.

{

"version": 1,

"rules": [

{

"description": "Redact passwords",

"trigger": "password",

"search": "password=\".*\"",

"replace": "password=\"???\""

}

]

}

[#5]

There is no password in Hue's version, and it has nothing to do with query
redaction, I would recommend to report it to the MapR forum as this look
like they did some custom changes.

[#6 · Accepted solution]

Thanks for the update @Romainr. I will surely check with the MapR team.

I just wanted to understand if log redaction could help us as the password is spitted out in the runcpserver.log.