Hi all, I have recently implemented LDAP for Nifi (works fine) - the problem is it is still using its own CA for certificates and presents the "This connection is not secure" message. Is it possible to create company signed certs to be used by Nifi although NOT for authorisation purposes? I have followed this guide and have been able to generate my own certs but these were used for authorisation and not as standard "web certificate" https://community.hortonworks.com/content/supportkb/151106/nifi-how-to-create-your-own-certs-for-securing-nif.html In short - How can I achieve Nifi LDAP while using a company signed cert? Thanks, ... View more

Hi guys, I have successfully integrated Ambari UI through through KNOX. In addition LDAP is also setup with Ambari and users can login with their LDAP credentials when accessing Ambari through the knox gateway. However, every time a user navigates to the quicklinks of a different UI and attempts to open a new UI (for example Yarn resource manager UI) an authentication popup jumps on screen asking for login credentials. The LDAP credentials the user just used to log into Ambari are no longer valid. Why is this? Here is the topology config: <topology> <gateway> <provider> <role>authentication</role> <name>ShiroProvider</name> <enabled>true</enabled> <param> <name>sessionTimeout</name> <value>30</value> </param> <param> <name>main.ldapRealm</name> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value> </param> <param> <name>main.ldapRealm.userDnTemplate</name> <value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</value> </param> <param> <name>main.ldapRealm.contextFactory.url</name> <value>ldap://{{knox_host_name}}:33389</value> </param> <param> <name>main.ldapRealm.contextFactory.authenticationMechanism</name> <value>simple</value> </param> <param> <name>urls./**</name> <value>authcBasic</value> </param> </provider> <provider> <role>identity-assertion</role> <name>Default</name> <enabled>true</enabled> </provider> <provider> <role>authorization</role> <name>AclsAuthz</name> <enabled>true</enabled> </provider> </gateway> <service> <role>NAMENODE</role> <url>hdfs://{{namenode_host}}:{{namenode_rpc_port}}</url> </service> <service> <role>JOBTRACKER</role> <url>rpc://{{rm_host}}:{{jt_rpc_port}}</url> </service> <service> <role>WEBHDFS</role> {{webhdfs_service_urls}} </service> <service> <role>WEBHCAT</role> <url>http://{{webhcat_server_host}}:{{templeton_port}}/templeton</url> </service> <service> <role>OOZIE</role> <url>http://{{oozie_server_host}}:{{oozie_server_port}}/oozie</url> </service> <service> <role>WEBHBASE</role> <url>http://{{hbase_master_host}}:{{hbase_master_port}}</url> </service> <service> <role>HIVE</role> <url>http://{{hive_server_host}}:{{hive_http_port}}/{{hive_http_path}}</url> </service> <service> <role>RESOURCEMANAGER</role> <url>http://{{rm_host}}:{{rm_port}}/ws</url> </service> <service> <role>DRUID-COORDINATOR-UI</role> {{druid_coordinator_urls}} </service> <service> <role>DRUID-COORDINATOR</role> {{druid_coordinator_urls}} </service> <service> <role>DRUID-OVERLORD-UI</role> {{druid_overlord_urls}} </service> <service> <role>DRUID-OVERLORD</role> {{druid_overlord_urls}} </service> <service> <role>DRUID-ROUTER</role> {{druid_router_urls}} </service> <service> <role>DRUID-BROKER</role> {{druid_broker_urls}} </service> <service> <role>ZEPPELINUI</role> {{zeppelin_ui_urls}} </service> <service> <role>ZEPPELINWS</role> {{zeppelin_ws_urls}} </service> <service> <role>AMBARI</role> <url>http://XXX.XXX.XXX.XXX:8080</url> </service> <service> <role>AMBARIUI</role> <url>http://XXX.XXX.XXX.XXX:8080</url> </service> <service> <role>HBASE</role> <url>http://XXX.XXX.XXX.XXX:16010</url> </service> <service> <role>HBASEUI</role> <url>http://XXX.XXX.XXX.XXX:16010</url> </service> <service> <role>YARN</role> <url>http://XXX.XXX.XXX.XXX:8088</url> </service> <service> <role>YARNUI</role> <url>http://XXX.XXX.XXX.XXX:8088</url> </service> </topology> ... View more

HDP: 2.6.2.14-5 As the title suggests, Knox is only creating a gateway for ambari. If I access any other service UI it reverts back to the original IP. It doesn't seem to be mapping at all. Taking YARN as an example, this is the config I have in the topology: <service> <role>YARN</role> <url>http://192.168.XXX.XXX:8088</url> </service> <service> <role>YARNUI</role> <url>http://192.168.XXX.XXX:8088</url> </service> ... View more

Hi all, My Livy service (version 0.3) is configured as shown in this tutorial: http://henning.kropponline.de/2016/11/06/connecting-livy-to-a-secured-kerberized-hdp-cluster/ The following command is able to access the livy server and give me a result: curl --negotiate -u : http://fqdn.host.name:8998/sessions My problem is when I try to launch a job from my API, I get an authentication error. Is this a problem due to configurations or because my code isn't asking for any negotiation methods/isn't using principals ? Before running the jobs I have tried doing a 'kinit' for SPNEGO, Zeppelin and Livy principals..no luck.. ... View more

Through tinkering I have been able to partially launch a spark submit job using the following command, however soon after starting it crashes and gives me the exception outlined below: Spark-Submit Command: su spark -c 'export SPARK_MAJOR_VERSION=2; spark-submit \ --verbose \ --master yarn \ --driver-cores 5 \ --num-executors 3 --executor-cores 6 \ --principal spark@test.com \ --keytab /etc/security/keytabs/spark.headless.keytab \ --driver-java-options "-Djava.security.auth.login.config=kafka_client_jaas.conf"\ --conf "spark.executor.extraJavaOptions=-Djava.security.auth.login.config=kafka_client_jaas.conf" \ --files "/tmp/kafka_client_jaas.conf,/tmp/kafka.service.keytab" \ --class au.com.XXX.XXX.spark.test.test test.jar application.properties' EXCEPTION: Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not logn:the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user WARN KerberosLogin: [Principal=kafka/test.com@test.com]: TGT renewal thread has been interrupted and will exit. How can I get Kerberos to KINIT two principals at the same time? I'm assuming that is the problem here? I have tried adding another set of --principal/--keytab to the initial command, although this presented more permission issues within HDFS. ... View more

Hi guys, I have installed Nifi as standalone. I have also created the user 'nifi' for that particular server and assigned it to all files/folders regarding Nifi. In the bootstrap.conf configuration file 'run.as' is set to nothing, although every time I try to access the web UI I'm met with the following error: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Found</h1> <p>The requested URL /nifi-api/flow/current-user was not found on this server.</p> </body></html> I have also gotten the following output from the nifi-user log: 2017-10-10 17:54:00,621 INFO [NiFi Web Server-43] org.apache.nifi.web.filter.RequestLogger Attempting request for (anonymous) GET http://XXX.XXX.XXX.XXX:8080/nifi-api/flow (source ip: XXX.XXX.XXX.XXX) 2017-10-10 17:54:00,622 INFO [NiFi Web Server-43] o.a.n.w.a.c.WebApplicationExceptionMapper javax.ws.rs.WebApplicationException. Returning 405 response. Has anyone experienced this before? Thanks in advance ... View more

[Apache Ambari version 2.5.1.0.] Hi guys, My current setup has all service UI's piped through a reverse proxy for security reasons. I'm currently having trouble accessing the Storm UI. Im given the following error in the browser (Chrome): org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag) After some digging it was mentioned that SPNEGO needs to be enabled for Storm UI, so I followed the following steps: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_security/content/_configuring_http_authentication_for_HDFS_YARN_MapReduce2_HBase_Oozie_Falcon_and_Storm.html This just results in a broken YARN (NodeManager Health alerts) and the same browser error.. Thank you in advance, ... View more

Hi guys, - Ambari 2.5.X, RHEL7 I'm attempting to disable the cipher: RSA_WITH_3DES_EDE_CBC_SHA I have added it to the ambari.properties file as both: RSA_WITH_3DES_EDE_CBC_SHA and TLS_RSA_WITH_3DES_EDE_CBC_SHA (They are separated by the '|' symbol) Although every time I run TestSSLServer2 on Ambari port..the ciphers show up. I have also added these ciphers to the servers java.security file with no luck. Am I missing something? ... View more