Hi,  We are currenly using HDP 3.0.1 with ambari 2.7.1. I was trying to solve a connection issue on the zeppelin ui that give to some of the users a disconnected status while they are using the chrome browser...(curiously it seems to be working just fine with firefox ).   I am able to see on the zeppelin notebook logs the following message:  Exception in login: org.apache.shiro.authc.AuthenticationException: Authentication token of type [class org.apache.zeppelin.realm.jwt.JWTAuthenticationToken] could not be authenticated by any configured realms. Please ensure that at least one realm can authenticate these tokens.   Also on the knox logs i have the following error message not sure if it is linked: Shiro unable to login: javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308: LdapErr: DSID-0C090400, comment: AcceptSecurityContext error, data 52e, v1db1   Could you please help us.   Thank you ... View more

Hi, I am currently using HDP 3.0.1 with ambari 2.7.1. What is the best way to set up a automatic snapshot of important folder on HDFS in a multi node environment?   Thank you ... View more

Hi,  A large amount of users and groups were imported in ranger after a ldap usersync and now i need to find a way to delete more than 200 of users and groups. I know that ranger has a api... i would like to know how to remove a list of external groups or users using that api or maybe another method that allow me to do it faster?     Thank you ... View more

HI, We have a multi host hadoop environment  that use HDP-3.0.1 with Nifi 1.7.0. The environment is Kerberised and we use ranger for the security, it is also connected to the LDAP/AD server for the user and groups.     Whenever i try to connect to NIFI using a u ser_test i get the message :    The u ser_test is part of a group who has full access to nifi in ranger... -Policy Name: All-nifi-Resource -NIfi Resource Identifier: * -Allow Conditions:    -Select Group: group_test    -Permissions: Read, Write   It seems that the AD group is not used when trying to connect in nifi (If i put the username u ser_test in the Select User section in Ranger it has access to nifi...).    I have the following msg in the nifi-user.log: 2019-11-13 12:18:29,368 INFO [NiFi Web Server-65454] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=xx-xx-xx-xxXX.XxxXx.xx, O=XxxXx Inc., L=XXXX, ST=XXXX, C=XX 2019-11-13 12:18:33,097 INFO [NiFi Web Server-65454] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response. 2019-11-13 12:18:33,136 INFO [NiFi Web Server-65286] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: OpenId Connect is not configured.. Returning Conflict response. 2019-11-13 12:18:33,151 INFO [NiFi Web Server-65436] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (<Knox JWT token>) GET https://xx-xx-xx-xxXX.XxxXx.xx:xxxx/nifi-api/flow/current-user (source ip: XX.XXX.XXX.XXX) 2019-11-13 12:18:33,151 INFO [NiFi Web Server-65436] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for user_test 2019-11-13 12:12:32,122 INFO [NiFi Web Server-65436] o.a.n.w.a.c.AccessDeniedExceptionMapper identity[user_test], groups[none] does not have permission to access the requested resource. Unable to view the user interface. Returning Forbidden response.   I am not sure why none appears and not group_test. Thank you in advance for your help on that subject. ... View more

Hi, I am currently using Hortonworks HDP 3.01 with ambari 2.7.1.0 but it seems that restarting all services can be a challenge because starting services/components must be done in a certain order to avoid timeouts and errors.... Is there a way to script the process? There is a section "Service Auto Start" in ambari but it doesn't seems that Ambari is smart enough to start services/componants while taking into consideration the dependencies... Could you please help?   Thank you   ... View more

Hi, Is there a way in NIFI to compress a simple file exemple.txt to a specific compresssed gzip file like archive.gzip or archive.gz? When the compressed file (archive.gzip or archive.gz) is decompressed  the filename should be exemple.txt and not archive.   Thank you     ... View more

Hello, I am having an issue running a query on my host and i getting the following message:     Any idea how to solve this issue? ... View more

HI, I am trying to import data from a csv file into hive using Apache NiFi - Version 1.7.0.3.2.0.0-520. following the steps described on : https://community.cloudera.com/t5/Community-Articles/Stream-data-into-HIVE-like-a-Boss-using-NiFi-HiveStreaming/ta-p/247130 i have changed few things - The table olympics on hive is not external  - I am using PutHive3Streaming   My current workflow:   Getfile:   InferAvroSchema:   ConvertCSVToAvro:   PutHive3Streaming:   PutFile:   AvroReader:   I am not getting any errors so far but when i run the processors the data is not stored in hive and i have 1 failure queued on PutHive3Streaming --> PutFile.   Could you please give me some guidance? Thank you ... View more

Hi, I am getting the following message on ambari on every worker nodes: 1/1 local-dirs usable space is below configured utilization percentage/no more usable space [ /data/2/hadoop/yarn/local : used space above threshold of 90.0% ] ; I have also checked on the ResourceManager UI > Nodes. 6 nodes are currently on a unhealthy state. Could you please help me get them back to a healthy state? Thank you ... View more

Hello, I have currently a kerberized hadoop HDP 3.0.1 cluster where HIVESERVER2 INTERACTIVE has been deployed. It was working just fine but today... HIVESERVER2 INTERACTIVE reported the following alert on ambari: Hive LLAP Application The application reported a 'NOT RUNNING' state. Check took 7.116s. Any idea what could be the issue? Thank you for your help! ... View more

HI, I am new with YARN, I am not able to restart the nodemanagers on some of nodes. Could you please help me out? ... View more

Hi I am trying to start live the nodemanager previously decommisionned on a node but it stops soon after it starts... ERROR collector.NodeTimelineCollectorManager (NodeTimelineCollectorManager.java:startWebApp(314)) - The per-node collector webapp failed to start. java.io.IOException: Problem starting http server   any idea what could be the problem? ... View more

Hi , I have added new worker nodes to my Hadoop cluster. i get the following error on the yarn logs less hadoop-yarn-nodemanager-hd-c2-wn09.log: 2019-08-09 08:44:44,898 ERROR ssl.SSLFactory (SSLHostnameVerifier.java:check(357)) - Host check error {} javax.net.ssl.SSLException: hostname in certificate did't match: != OR Any idea how to fix this? ... View more

HI, I am trying to connect to the hive server of my kerberized HDP 3.0.1 using beeline on a external host. I have imported the keytab and also changed the /etc/krb5.conf on the external host, i am able to kinit using the keytab but i stilll get the following error when i am trying to connect to the hive server:   shell> kinit mapr/<FQDN@REALM> -k -t /home/user1/Desktop/hive.service.keytab shell> beeline -u "jdbc:hive2://<hostname>:10000/default;principal=mapr/<FQDN@REALM>" SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/user1/Documents/HDP/apache-hive-3.1.0-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/user1/Documents/HDP/hadoop-3.0.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Connecting to jdbc:hive2://<hostname>:10000/default;principal=mapr/<FQDN@REALM> Java config name: null Native config name: /etc/krb5.conf Loaded from native config 19/07/29 08:49:43 [main]: ERROR transport.TSaslTransport: SASL negotiation failure javax.security.sasl.SaslException: GSS initiate failed at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) ~[?:1.8.0_212] at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) ~[hive-exec-3.1.0.jar:3.1.0] at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) [hive-exec-3.1.0.jar:3.1.0] at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [hive-exec-3.1.0.jar:3.1.0] at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:51) [hive-exec-3.1.0.jar:3.1.0] at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:48) [hive-exec-3.1.0.jar:3.1.0] at java.security.AccessController.doPrivileged(Native Method) [?:1.8.0_212] at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_212] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965) [hadoop-common-3.0.1.jar:?] at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport.open(TUGIAssumingTransport.java:48) [hive-exec-3.1.0.jar:3.1.0] at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:339) [hive-jdbc-3.1.0.jar:3.1.0] at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:224) [hive-jdbc-3.1.0.jar:3.1.0] at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) [hive-jdbc-3.1.0.jar:3.1.0] at java.sql.DriverManager.getConnection(DriverManager.java:664) [?:1.8.0_212] at java.sql.DriverManager.getConnection(DriverManager.java:208) [?:1.8.0_212] at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:145) [hive-beeline-3.1.0.jar:3.1.0] at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:209) [hive-beeline-3.1.0.jar:3.1.0] at org.apache.hive.beeline.Commands.connect(Commands.java:1641) [hive-beeline-3.1.0.jar:3.1.0] at org.apache.hive.beeline.Commands.connect(Commands.java:1536) [hive-beeline-3.1.0.jar:3.1.0] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_212] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_212] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212] at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:56) [hive-beeline-3.1.0.jar:3.1.0] at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1384) [hive-beeline-3.1.0.jar:3.1.0] at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1423) [hive-beeline-3.1.0.jar:3.1.0] at org.apache.hive.beeline.BeeLine.connectUsingArgs(BeeLine.java:900) [hive-beeline-3.1.0.jar:3.1.0] at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:795) [hive-beeline-3.1.0.jar:3.1.0] at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1048) [hive-beeline-3.1.0.jar:3.1.0] at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:538) [hive-beeline-3.1.0.jar:3.1.0] at org.apache.hive.beeline.BeeLine.main(BeeLine.java:520) [hive-beeline-3.1.0.jar:3.1.0] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_212] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_212] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212] at org.apache.hadoop.util.RunJar.run(RunJar.java:239) [hadoop-common-3.0.1.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:153) [hadoop-common-3.0.1.jar:?] Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt) at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) ~[?:1.8.0_212] at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) ~[?:1.8.0_212] at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) ~[?:1.8.0_212] at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) ~[?:1.8.0_212] at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) ~[?:1.8.0_212] at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[?:1.8.0_212] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ~[?:1.8.0_212] ... 36 more 19/07/29 08:49:43 [main]: WARN jdbc.HiveConnection: Failed to connect to <hostname>:10000 Unknown HS2 problem when communicating with Thrift server. Error: Could not open client transport with JDBC Uri: jdbc:hive2://<hostname>:10000/default;principal=mapr/<FQDN@REALM>: GSS initiate failed (state=08S01,code=0) Beeline version 3.1.0 by Apache Hive     NOTE: I was following the following recommendation, section Using Beeline with Kerberos: https://mapr.com/docs/archive/mapr40x/Configuring-Hive-on-a-Secure-Cluster_28869090.html#ConfiguringHiveonaSecureCluster-UsingBeelinewithKerberos   Could you please help?   Thank you ... View more