All three namenode directories are pointing to same metadata. /hadoop/hdfs/namenode /var/hadoop/hdfs/namenode /var/log/hadoop/hdfs/namenode What is the use of having 3 directories pointing to same data? ... View more

when i click on grafana UI then it will take me to the a different page and there its giving This site can’t provide a secure connection <hostname> sent an invalid response. ERR_SSL_PROTOCOL_ERROR ... View more

I was creating one table in hive using beeline in which i need to compress my data using PARQUET file format. so i try to use set parquet.compression=SNAPPY; But while executing this command i am getting one error as : Error: Error while processing statement: Cannot modify parquet.compression at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1) I checked and this property is not present in whitelist of params and we dont have permissions to edit the whitelist. so i got one resolution as instead of using set parquet.compression=SNAPPY; at runtime I used the table properties TBLPROPERTIES ('PARQUET.COMPRESS'='SNAPPY') and then it works the table is successfully created. But when i loaded the data to table and by using describe table i compare the data with my other table in which i did not used the compression, the size of data is same. so that means by using 'PARQUET.COMPRESS'='SNAPPY' compression is not happening. Is there any other property which we need to set to get the compression done. For Avro i have seen the below two properties to be set to do the compression hive> set hive.exec.compress.output=true; hive> set avro.output.codec=snappy; Likewise do i need to set some other property for parquet file? ... View more

I have one application id which i got from Resource manager and since this job ran by hive as user so i am not able to find out exactly who ran that job. So i have application ID and now i need to know the code or query used by that particular application id. Is there a way to get any kind of logs from which i can get the code or query used by that application ID ... View more

The hive query which is used by my batch is taking too much time to run. Earlier when i fire the same query it took around 5 minutes and now it is taking around 22 minutes. I cant change the query. Please suggest the correct way to investigate this issue or kindly suggest any resolution. ... View more

Like Example.jar is there any sample pig scripts available with HDP? ... View more

All our jobs are failing and giving only one error : 16/12/12 01:55:49 INFO impl.YarnClientImpl: Application submission is not finished, submitted application application_1481368045892_0922 is still in NEW 16/12/12 01:55:51 INFO impl.YarnClientImpl: Application submission is not finished, submitted application application_1481368045892_0922 is still in NEW 16/12/12 01:55:53 INFO impl.YarnClientImpl: Application submission is not finished, submitted application application_1481368045892_0922 is still in NEW When i checked the resource manager UI, i clicked on the logs and it shows as: java.lang.Exception: Container is not yet running. Current state is NEW We have recently done the HDP upgrade from 2.3.2.0 to 2.4.2.0 ... View more

After manually regenerating kerberos keytabs ambari is giving the below error in hive: Connection failed on host <Hiveserver2 hostname>:10000 (Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py", line 200, in execute check_command_timeout=int(check_command_timeout)) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/hive_check.py", line 64, in check_thrift_port_sasl Execute(kinitcmd, user=smokeuser) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call tries=tries, try_sleep=try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call raise Fail(err_msg) Fail: Execution of '/usr/bin/kinit -kt /dsap/etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM; ' returned 1. kinit: Password incorrect while getting initial credentials ) ... View more