Member since
09-27-2019
48
Posts
0
Kudos Received
0
Solutions
11-30-2019
04:27 PM
@MattWho I've been trying these steps and somehow the SAN keeps getting removed when I import/export to JKS. How do I get the SAN extension to be with the key inside of the keystore file? I'm totally stuck!
... View more
11-26-2019
05:52 PM
I'm needing to clean a NiFi Node to have it stop getting errors. I've narrowed down the problem quite a bit (I'm trying to have it Authenticate via LDAP, with a self-signed SSL cert, both of these things are set up properly now). However, because I've been through so much troubleshooting and trial and error with this NiFi server, I have experienced WAY more hassle than I would've expected. I've been trying to get this to work for nearly a month now. SSL certs, truststore, keystore, node config (In CM) are all set up to the best of my ability. I stop the service roles in CM, put host in maintenance mode, restart the VM nifi is running on (to clear extra scm-agent processes). I've moved the flowfile.xml.gz, deleted users.xml, authorizations.xml, cleared the _repository folders out (moving them to a backup folder), cleared the archive folder, state/local folder. I've set initial admin to our proper LDAP identity. (I never get to this stage before the node shuts off now though) Jetty is reporting that the there is no valid keystore, but I am not sure that this is the cause of the effect of a different problem. I've been very careful to create the keystores exactly to specification following @MattWho 's article and have verified everything, also I had HTTPS working last night (csr worked, but I could not manage to log in due to "unverified keystore" on the client side I believe). Having as little experience with this sort of thing as I do, I have been so challenged and puzzled to get HTTPS set up for LDAP auth. I keep feeling like I'm only one or two steps away from having things working but then another problem springs up. Here is a piece of the log file from which I've scoured and this is the FIRST sign of something not going correctly: 7:04:23.327 PM INFO _nifi No Spring WebApplicationInitializer types detected on classpath 7:04:23.404 PM INFO ContextHandler Started o.e.j.w.WebAppContext@5b16e486{nifi,/nifi,file:///var/lib/nifi/work/jetty/nifi-web-ui-1.9.0.1.0.1.0-12.war/webapp/,AVAILABLE}{/var/lib/nifi/work/nar/framework/nifi-framework-nar-1.9.0.1.0.1.0-12.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-ui-1.9.0.1.0.1.0-12.war} 7:04:23.719 PM INFO AnnotationConfiguration Scanning elapsed time=181ms 7:04:23.748 PM INFO _nifi_api No Spring WebApplicationInitializer types detected on classpath Followed by this Error, which is followed by a MASSIVE list of missing beans which i have not included. Context initialization failed org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration': Unsatisfied dependency expressed through method 'setFilterChainProxySecurityConfigurer' parameter 1; nested exception is org.springframework.beans.factory.BeanExpressionException: Expression parsing failed; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.apache.nifi.web.NiFiWebApiSecurityConfiguration': Unsatisfied dependency expressed through method 'setJwtAuthenticationProvider' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jwtAuthenticationProvider' defined in class path resource [nifi-web-security-context.xml]: Cannot resolve reference to bean 'jwtService' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jwtService' defined in class path resource [nifi-web-security-context.xml]: Cannot resolve reference to bean 'keyService' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'keyService' defined in class path resource [nifi-administration-context.xml]: Cannot resolve reference to bean 'keyTransactionBuilder' while setting bean property 'transactionBuilder'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'keyTransactionBuilder' defined in class path resource [nifi-administration-context.xml]: Cannot resolve reference to bean 'keyDataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'keyDataSource': FactoryBean threw exception on object creation; nested exception is org.h2.jdbc.JdbcSQLException: Error opening database: "Could not save properties /var/lib/nifi/database_repository/nifi-user-keys.lock.db" [8000-176] Warn: Error opening database: "Could not save properties /var/lib/nifi/database_repository/nifi-user-keys.lock.db" [8000-176] Failed startup of context o.e.j.w.WebAppContext@2b85edc7{nifi-api,/nifi-api,file:///var/lib/nifi/work/jetty/nifi-web-api-1.9.0.1.0.1.0-12.war/webapp/,UNAVAILABLE}{/var/lib/nifi/work/nar/framework/nifi-framework-nar-1.9.0.1.0.1.0-12.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.9.0.1.0.1.0-12.war} And the final error before shutting down: 7:04:33.701 PM INFO _ No Spring WebApplicationInitializer types detected on classpath 7:04:33.755 PM INFO ContextHandler Started o.e.j.w.WebAppContext@490704a5{nifi-error,/,file:///var/lib/nifi/work/jetty/nifi-web-error-1.9.0.1.0.1.0-12.war/webapp/,AVAILABLE}{/var/lib/nifi/work/nar/framework/nifi-framework-nar-1.9.0.1.0.1.0-12.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-error-1.9.0.1.0.1.0-12.war} 7:04:33.787 PM WARN JettyServer Failed to start web server... shutting down.
java.lang.IllegalStateException: no valid keystore
at org.eclipse.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:50)
at org.eclipse.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:1071)
at org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:262)
at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:229)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:72)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:279)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:398)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:935)
at org.apache.nifi.NiFi.<init>(NiFi.java:158)
at org.apache.nifi.NiFi.<init>(NiFi.java:72)
at org.apache.nifi.NiFi.main(NiFi.java:297) 7:04:33.797 PM INFO NiFi Initiating shutdown of Jetty web server... So. With this, it is failing to even reach the stage where it will generate the users.xml and authorizations.xml file that I'm used to it auto-generating after I remove it. I've had to change the initial admin a couple of times, so I learned how to do that without reinstalling. I've removed the flowfile.xml.gz, which I suspect may be an issue here. As well as the _repository folders, which I was advised to do via other forum posts here, but I may have cleared more than I should have? I have backups saved. Really my big question is: what all CAN i remove/clean without NiFi not being able to recover / auto-gen new files on start. Any ideas would be super appreciated!
... View more
Labels:
- Labels:
-
Apache NiFi
11-21-2019
12:48 PM
CFM 1.0.1 Could someone please explain to me why, after reconfiguring to disable SSL, when I try to access the webUI it still is attempting to connect on port 8443? Is this a zookeeper issue? How can I fix this? I had another cluster previously that start out as SSL disabled, but after configuring it to have a cert via nifi toolkit CA, and getting it WORKING with LDAP auth, it would try to request things from port 8080 http, then after vote resolved it will kill the NiFi node. So here I am now. Was unable to get SSL verified while following the 5 year old guide courtesy of @MattWho (you are awesome, i really appreciate all the info you've shared) I followed that guide to a T but the server reporting the cert was not verified. Now I'm trying to just connect to the nifi node via HTTP, so i disabled all related SSL things in the config, yet: Cannot replicate request to Node nifi.domain.net:8443 because the node is not connected It still tries to connect to port 8443. Can someone tell me why it's trying to connect to the old port? Is this a zookeeper issue? If i could resolve this then I feel i could resolve all my other issues. Thanks!
... View more
Labels:
11-20-2019
04:59 PM
Hey MattWho, I gotta say, THANK YOU so much for writing this guide. Seriously, this saved me sooo much time, having zero experience with setting up SSL chains. Really appreciate everything you've done to help me/us get nifi working!!
... View more
11-18-2019
01:55 PM
Thanks so much Matt!! You've been a huge help in getting my mind wrapped around all of this. If you can't tell, I'm a bit new to authentication and authorization! I really appreciate everything you've done to point me in the right direction. If things go as planned, I *should* have a NiFi node working by the end of the day! Here's one more for you, though. I'm having to reinstall the NiFi service on a new host because some property values are messed up after all of my tinkering trying to get things to work. The node will start up, HTTPS will be working, I can successfully log into the WebUI, but then after ~5 minutes or so, something happens and it reverts to trying to use HTTP and reports that it is trying to connect to the site on the HTTP port and fails to do so. I believe everything is configured in CM properly but there are some local configs that aren't right, or profiles that need to be deleted in order for new (correct) profiles to be created automatically. If this rings any bells I would love to learn more about how to fix it, but for now it seems the best thing to do is to do a fresh install. Aloha!
... View more
11-15-2019
02:57 PM
I assumed as such, actually! My bad for not communicating that I'm configuring from within CM. We don't have a support license yet, but are hoping to ASAP. Just for basic, SIMPLE LDAP authentication should I need to configure safety valves? It seems like I should be able to get it working with the configurations available. However, it's getting stuck somewhere. I can connect to the server via HTTPS and I get a login screen. Should I be able to log in without LDAP using the initial admin + master password?
... View more
11-14-2019
07:06 PM
Thanks so much Matt!! So, security is working now. BUT no luck logging in with ldap yet. Do I need to configure the safety valves in order to access login-identity-providers.xml [root@nifi /]# find . -name login-identity-providers.xml ./run/cloudera-scm-agent/process/196-nifi-NIFI_NODE/login-identity-providers.xml ./run/cloudera-scm-agent/process/196-nifi-NIFI_NODE/aux/defaults/login-identity-providers.xml ./run/cloudera-scm-agent/process/195-nifi-NIFI_NODE/login-identity-providers.xml ./run/cloudera-scm-agent/process/195-nifi-NIFI_NODE/aux/defaults/login-identity-providers.xml ./run/cloudera-scm-agent/process/194-nifi-NIFI_NODE/login-identity-providers.xml ./run/cloudera-scm-agent/process/194-nifi-NIFI_NODE/aux/defaults/login-identity-providers.xml ./run/cloudera-scm-agent/process/181-nifi-NIFI_NODE/login-identity-providers.xml ./run/cloudera-scm-agent/process/181-nifi-NIFI_NODE/aux/defaults/login-identity-providers.xml Are there multiple processes running as a security measure, or is something on configured properly? I have no idea which login-identity-providers.xml to edit!
... View more
11-09-2019
06:01 PM
I'm attempting to enable authentication on my NiFi server that is in a cluster with several other hosts. If I enabled HTTPS for webUI authentication, without changing any of the other hosts on the cluster, will this cause any complications with their communication with one another?
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera Manager
11-09-2019
05:50 PM
I've been trying to get this to work for many days now and keep running into issues, so I would appreciate any input anyone has to offer on this. I need to enable a self-signed TLS 1.2 cert for HTTPS on the single NiFi node in my cluster in order to authenticate via LDAP. Jetty is the current webserver running on NiFi, and everything is working fine except there is no authentication method so anyone who has access to it can go in and make changes. I've read the documentation provided quite a bit, but still have many questions unanswered. Is NiFi Toolkit CA primarily for securing multiple NiFi Nodes, or should I be using it to get HTTPS set up on the server?
... View more
11-06-2019
01:51 PM
Will do, Thanks @MattWho. I actually didn't realize that NiFi can't be authenticated via HTTP until after putting many, many hours into trying to get it to work! I'm not sure if you're involved with writing the documentation for CFM or not, but it may be beneficial to make it more clear that LDAP Authentication via HTTP isn't possible. I'm on to setting up TLS now, but if I have more issues I will ask in community. Thanks so much for your help. I hope no one else spends as much time as I did trying to troubleshoot why auth wasn't working for HTTP! Derp... Aloha 🙂
... View more
- « Previous
-
- 1
- 2
- Next »