Hi, How do we find why all services went down ?We have HDP2.6 all services went down, do we just restart the all services? .We are able to connect to Ambari. ... View more

We have HDP 2.6.4 Ambari URL is not loading sudo ambari-server status Using python /usr/bin/python Ambari-server status Ambari Server running Found Ambari Server PID: 14654 at: /var/ambari-server/ambari-server.pid 01 Aug 2018 19:08:53,866 ERROR [ambari-client-thread-1088676] MetricsRequestHelper:115 - Error getting timeline metrics : Connection refused (Connection refused) 01 Aug 2018 19:08:53,867 ERROR [ambari-client-thread-1088676] MetricsRequestHelper:122 - Cannot connect to collector: SocketTimeoutException for ctcl-hdpedge1.msoit.com 01 Aug 2018 19:08:46,391 ERROR [ambari-client-thread-1088525] MetricsRequestHelper:122 - Cannot connect to collector: SocketTimeoutException for ctcl-hdpedge1.msoit.com 01 Aug 2018 19:08:53,867 ERROR [ambari-client-thread-1088676] MetricsRequestHelper:122 - Cannot connect to collector: SocketTimeoutException for ctcl-hdpedge1.msoit.com ... View more

We have HDP 2.6.4, we have user janu, when she runs the first sql it works fine, 2 sql gives below error we have kerberos enabled and users were created using ldap 1)select * from db.tbl 2) select col1 from db.tbl group by col1 AM Container for appattempt_1523646395030_2519_000002 exited with exitCode: -1000 For more detailed output, check the application tracking page: http://ctcl-hdpmaster3.com:8088/cluster/app/application_1523646395030_2519 Then click on links to logs of each attempt. Diagnostics: Application application_1523646395030_2519 initialization failed (exitCode=255) with output: main : command provided 0 main : run as user is janu main : requested yarn user is janu User janu not found Failing this attempt ... View more

We are getting below error on randomly for few minutes and then goes away, its coming in PUThiveql 2018-05-29 01:01:07,279 INFO [Timer-Driven Process Thread-95] org.apache.hive.jdbc.HiveConnection Will try to open client transport with JDBC Uri: jdbc:hive2://hdpmaster2.msoit.com:10000/default;principal=hive/_HOST@IT.COM 2018-05-29 01:01:07,281 ERROR [Timer-Driven Process Thread-95] o.apache.thrift.transport.TSaslTransport SASL negotiation failure javax.security.sasl.SaslException: GSS initiate failed at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:176) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38) at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582) at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148) at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106) at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044) at org.apache.nifi.dbcp.hive.HiveConnectionPool.lambda$getConnection$0(HiveConnectionPool.java:355) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) at org.apache.nifi.dbcp.hive.HiveConnectionPool.getConnection(HiveConnectionPool.java:355) at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:89) at com.sun.proxy.$Proxy97.getConnection(Unknown Source) at org.apache.nifi.processors.hive.PutHiveQL.lambda$new$1(PutHiveQL.java:191) at org.apache.nifi.processor.util.pattern.Put.onTrigger(Put.java:96) at org.apache.nifi.processors.hive.PutHiveQL.lambda$onTrigger$6(PutHiveQL.java:274) at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114) at org.apache.nifi.processor.util.pattern.RollbackOnFailure.onTrigger(RollbackOnFailure.java:184) at org.apache.nifi.processors.hive.PutHiveQL.onTrigger(PutHiveQL.java:274) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1147) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:175) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt) at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192 ... View more

spark-submit --master yarn --deploy-mode cluster sqlserver.py --jars sqljdbc42.jar I get Error : java.lang.ClassNotFoundException: com.microsoft.sqlserver.jdbc.SQLServerDriver Everything works when i use --deploy-mode client and copy the jar file to /usr/hdp/current/spark2-client/jars/sqljdbc42.jar Should i copy the sqljdbc42.jar to /usr/hdp/current/hadoop-yarn-client/lib/ on all data nodes? ... View more

I am trying to use python udf in hive, but i get the below error Error: Error while processing statement: FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException(Query with transform clause is disallowed in current configuration.) (state=08S01,code=12) ... View more

We have HDP 2.6, when i added table & column comments using alter table.These comments are not reflected in Atlas. I thought this should be automated ... View more

I am trying to create an external table hive , i get below error.I have defined the policy in ranger. Hive policy i have given all permissions to the user for the database(all tables & all columns). I am conecting to hive using beeline Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [hdp_ingest] does not have [ALL] privilege on [hdfs://hdpmaster1.m.com:8020/datalake/first/landing/tbltest] (state=42000,code=40000) ... View more

I have installed NIFI 1.5 on windows machine, i am trying to use puthdfs. I have coresite.xml and hdfs-site.xml copied to C:\nifi-1.5.0-bin\nifi-1.5.0\lib\. In Hadoop coniguration resources i have given C:\nifi-1.5.0-bin\nifi-1.5.0\lib\core-site.xml,C:\nifi-1.5.0-bin\nifi-1.5.0\lib\hdfs-site.xml What should i specify in Kerberos Principal & Kerberos Keytab properties? ... View more

We are currently using sqoop and Hive sql to ingest data.Can we use NIFI to ingest data into HDFS and load data into Hive tables.Is it possible to Install NIFI on a windows Server and schedule jobs to load data into HDFS and Hive? ... View more