HDP 2.6 HIVE meta-store failed

[#1]

Team,

After i upgrade HDP 2.5 to 2.6 , Iam getting HIVE metastore below error. I dont have atlas installed. I see in HWX community to remove hive.exec.post.hooks. but i dont find option to remove that property through ambari 2.5

ERROR Metastore on velocity.test.server.com failed (Traceback (most recent call last):

File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py", line 200, in execute

timeout_kill_strategy=TerminateStrategy.KILL_PROCESS_TREE,

File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__

self.env.run()

File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run

self.run_action(resource, action)

File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action

provider_action()

File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run

tries=self.resource.tries, try_sleep=self.resource.try_sleep)

File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner

result = function(command, **kwargs)

File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call

tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)

File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper

result = _call(command, **kwargs_copy)

File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call

raise ExecutionFailed(err_msg, code, out, err)

ExecutionFailed: Execution of 'export HIVE_CONF_DIR='/usr/hdp/current/hive-metastore/conf' ; hive --hiveconf hive.metastore.uris=thrift://*****s.com:9083 --hiveconf hive.metastore.client.connect.retry.delay=1 --hiveconf hive.metastore.failure.retries=1 --hiveconf hive.metastore.connect.retries=1 --hiveconf hive.metastore.client.socket.timeout=14 --hiveconf hive.execution.engine=mr -e 'show databases;'' returned 1. log4j:WARN No such property [maxFileSize] in org.apache.log4j.DailyRollingFileAppender.

Logging initialized using configuration in file:/etc/hive/2.6.1.0-129/0/hive-log4j.properties

OK

Failed with exception java.io.IOException:java.lang.RuntimeException: Error in configuring object

Time taken: 1.957 seconds

)

[#2]

@suresh krish

can you check what the value for the hive.exec.post.hooks in hive config using ambari?

[#3]

org.apache.hadoop.hive.ql.hooks.ATSHook

[#4]

@suresh krish

then Atlas hook is not an issue here, do you find any exception/error in metastore logs?

[#5]

When i do a show database. i am getting

Failed with exception java.io.IOException:java.lang.RuntimeException: Error in configuring object

[#6]

@suresh krish

can you add the output of following command

hive --hiveconf hive.root.logger=DEBUG,console -e "show databases;"

[#7]

Debug data

Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hive.common.util.ReflectionUtil.setJobConf(ReflectionUtil.java:112)
        ... 22 more
Caused by: java.lang.IllegalArgumentException: Compression codec com.protegrity.hadoop.fileprotector.crypto.codec.PtyCryptoCodec not found.
        at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139)
        at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:179)
        at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
        ... 27 more
Caused by: java.lang.ClassNotFoundException: Class com.protegrity.hadoop.fileprotector.crypto.codec.PtyCryptoCodec not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2147)
        at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132)
        ... 29 more

[#8]

@suresh krish what the value for key io.compression.codecs in hdfs config using ambari?

[#9]

Zip, default , snappy compression codec

[#10]

@suresh krish it seems that it is taking PtyCryptoCodec from some configuration, you need to check from where it is taking this configuration, run hive -e 'set -v;' > hive.out and grep codec in hive.out to know from where it is picking the PtyCryptoCodec