Member since
09-27-2019
48
Posts
0
Kudos Received
0
Solutions
06-01-2020
08:43 PM
Is this (now) considered a NiFi "anti-pattern"? Do you have any idea how to do this using NiFi Record serialization services? I'm under the impression that creating thousands of content files is not the best practice by today's standards, but I'm not sure how to use InvokeHTTP on a full set of records without splitting it into many flowfiles. Any ideas?
... View more
06-01-2020
02:04 PM
I am curious what the best practice is for using InvokeHTTP on an array of URLs stored in a singular Record set content file. I thought maybe it would be best to QueryRecord and store the individual URLs to be fed into InvokeHTTP one-by-one, but I'm looking for the optimal way to do this (without creating excessive content/attributes/provenance data. Is it possible, or ever a good idea to create individual attributes of each URL (with little-to-no content) to be fed into InvokeHTTP?
... View more
- Tags:
- cdf
- HDP
- invokehttp
- NiFi
04-25-2020
05:03 PM
Rebuilding a cluster and I have 8 hosts to distribute, unpack, and activate the CDH parcel onto. Distribution speed finished at 30mb/s. Creeeeeping along. What would be the best way to diagnose where the bottleneck is taking place? I've gone through the process ~4 times now and i've yet to ever optimize connection speed between my nodes. I would expect to be getting at least ten times the file transfer speeds that I'm seeing currently though, and I'd like to find a way to speed things up. Will HDFS boost cluster performance a lot typically?
... View more
Labels:
04-20-2020
05:11 PM
Had a file system crash, rebuilt Cloudera Manager host with the same everything--except the private TLS key for HTTPS. My psql db is still in tact, so the rebuilt CM host is connecting to it and getting configurations from the previous server. Is this a bad idea? Should I create a new database for the rebuilt CM host, or can I have it connect to the previous db and disable HTTPS manually somehow so I can access the web console and update it from there? Would it be a better idea to just regenerate new keys with the same password? I've never done this before, so I am unclear on any potential conflicts if a new Cloudera Manager server connects to an existing Cloudera Manager database.
... View more
Labels:
04-17-2020
10:03 PM
Thank for you this reply! This has been quite difficult for me to troubleshoot, but I finally figured it out. These machines I've been using had chrony on them all along, but the previous machines I set up did not have chrony installed. Chrony and ntpd were both enabled, and ntpd was getting exited on reboot. Because the host monitor issues "ntpq -np", and ntpd was loaded but inactive, it would report a failure to query the server, even though chrony was running. I had no idea that chrony was installed, and thus, the whole problem could've been solved by just disabling/uninstalling ntpd. I spent WAY too many hours to come to such a simple solution. It may be very helpful to someone who doesn't understand network time protocols very well if there was a suggestion to explain potential conflicts between ntpd and chronyd in the documentation, or even to take a second to check which (if any) you already have installed. Maybe it won't be an issue for most people, but for me, assuming that I didn't have chrony already running cost me a bunch of time getting my cluster healthy. I would check, find ntpd dead, see no problems reported on Host Monitor, wonder why the hell ntpd died, kill ntpd, run ntpdate, restart ntpdate, restart scm-agent, and that would "fix" it, but on reboot it would go back to using chrony and exit ntpd, and host monitor would report failure to query ntp service, even though the machine was using chrony and synced just fine all along. I appreciate your help!
... View more
04-15-2020
04:46 PM
Continuously my NTP service seems to die on some of my VMs but not others, and I cannot seem to figure out why. I've followed every piece of advice I've been given so far to no avail. NTP service is dying without drifting it seems. When I use `ntpdate` to re-sync to the pool, it's never been more than ~50ms offset. yum install ntp -y systemctl start ntpd systemctl enable ntpd ntpdate -u pool.ntp.org hwclock --systohc The VMs are hosted across 3 different ESXi servers that are connected via vCenter. The ESXi hosts all have an NTP service running and enabled, and are syncing to the same external NTP server. Is there anything in `ntp.conf` that should be changed from default in order to prevent complications? I have zero clue why ntp service will randomly die on some hosts, yet I have some hosts that have had the service running without issue for 1+ month, but only if CM doesn't have the proper PID for the ntp service. As I'm writing this I think I may have solved it. By default ntp.conf is set to not allow queries. So is Cloudera querying ntp for it's offset to the point that it kills the service as a security measure or something? from ntp.conf: # Permit time synchronization with our time source, but do not # permit the source to query or modify the service on this system. restrict default nomodify notrap nopeer noquery Is Cloudera making NTP kill itself by querying it for it's offset? Please update documentation to reflect this if this is the case!
... View more
Labels:
01-31-2020
09:21 PM
Hello. I'm installing CFM 1.0.1 on a cluster that is running CDH on OpenJDK 11. As the installations requests, I have installed OpenJDK-1.8.0 on the machines that will be using NiFi and NiFi Registry. However, upon installing Java-1.8.0, the alternative paths have all switched to Java-1.8.0, which I suspect causing some issues with CM and CDH. # alternatives --config java I have used this to set the defaults back to Java 11, however I now see that there are many more alternative paths that are pointing to the Java 8 directory (javac, javah, java_sdk, jre, etc). Do I need to change all of these to point back to Java 11? I saw nothing about this in the documentation, and it seems that having the whole cluster on Java 8 would be more simple, but I'm a bit far into the installation process to switch now. Cloudera Express works better on Java 8? I see that as of CDH 6.30 Java 11 is supported, so I've been using that. I would love to be provided with some information on the most pain-free way for me to get NiFi (CFM 1.0.1) to work on my cluster. It's a little confusing to me how things are supposed to be configured.
... View more
Labels:
01-29-2020
11:10 PM
For a single node nifi cluster, is 1 ZK host alright?
... View more
01-21-2020
01:10 AM
NiFi 1.10.0 will certainly work with Java 11. The issue I've had is that I repeatedly run into problems when using CFM 1.0.1 and having differing versions of Java across my cluster. I've had CDH java path configured to the best of my knowledge, and the NiFi node configured to run on OpenJDK 1.8 as the documentation suggests, but I have run into so many problems along the way. I need to have a product ready to demo really soon, and I've hit so many bumps in the road trying to keep Cloudera Manager happy, in addition to many of my NiFi nodes failing to start up mostly due to java pathing issues (I think), broken user/group permissions, etc. I wish I were confident enough to be able to use CFM NiFi on Java 8, but all our other machines are connected with Java 11 on CDH 6.3.2. According to my research, CFM with NiFi 1.10.0 isn't even in the schedule yet, with NiFi 1.10.0 first scheduled to appear with CDP-DC pretty soon. I'd love to know if and when a new CFM will be released!
... View more
01-18-2020
03:18 PM
I've been troubleshooting for some time now, and really at this point I'm just looking for some suggestions from people who have more experience than I do. I want to run NiFi 1.10.0 on OpenJDK 11, with NiFi Registry. Previously I was using Cloudera Flow Management 1.0.1, but ran into many issues with mixing Java 8 and Java 11, so I'm making a full switch to exclusively Java 11 now. CFM 1.0.1 is incompatible with Java 11, so I need to find another option for managing NiFi. In your opinion, would it be easiest for me to manually add NiFi and NiFi Registry nodes to CDH 6.3.2? I understand that Cloudera Data Platform - Data Cloud is going to have support for NiFi 1.10.0, however I currently do not need much more than a couple of nodes... I'm building this for demo purposes and not at scale. Any recommendations would be greatly appreciated. P.S. Does anyone know when Cloudera Flow Management will be updated to NiFi 1.10.0?
... View more
01-17-2020
07:24 PM
Any updates on this Matt? We've switched to Java 11 and would like to use CDF/CFM with NiFi 1.10.0!
... View more
01-17-2020
07:19 PM
Question about versioning... We are no longer using Java 8, but the current CFM 1.0.1 has a version of NiFi that only works with Java 8. I'm going to install a NiFi 1.10.0 node on Java 11 without CFM 1.0.1 until CFM gets an update. My question is, will NiFi Registry on CFM 1.0.1 be compatible with Java 11? Thank you!
... View more
Labels:
01-04-2020
03:58 PM
Thanks Eric! That is indeed the solution I was looking for. I ended up figuring it out myself a week or so ago before I saw your comment here just now. The cool thing was that after getting CM up and running, it was able to eventually update the new FQDNs for all of the clustered hosts on it's own (even though initially it was still displaying the old hostnames from before our migration)!
... View more
12-27-2019
11:35 PM
Aloha. After a recent server migration, all of the hostnames on our cluster got a slight modification. After getting the Cloudera Manager Server to connect to it's required database correctly, it still wants to connect to the old hostnames for everything else in the cluster. Everything is already configured for them to work together, so it should be as simple as updating the URL to connect to the hostname (rather than reinstalling cloudera-scm-agent, etc). Is there a way to do this from within Cloudera Manager? How?
... View more
Labels:
12-27-2019
12:39 AM
As the title explains, we have moved our cluster from `taco.net` to `dev.taco.net`, and in the process this has made our Cloudera Manager server unable to start up because it continues to try to connect to database.taco.net instead of database.dev.taco.net. Hosts files are updated on all servers, network interfaces are configured to their new IPs. I would imagine there is something I should update on the Cloudera Manager server, but I'm not sure where to look to update the URL.
... View more
Labels:
12-02-2019
04:48 PM
[The following question was moved here because it was posted 12-02-2019 04:48 PM to a thread marked 'Solved' 11-18-2019 05:49 AM —moderator]
How did you add a SAN extension and have it not get removed when adding the key to your JKS file? I never figured this out. Even When using Nifi Toolkit CA, the certs that are generated don't contain SAN. Soooo... still confused on this!
... View more
12-01-2019
08:28 PM
If I recall correctly, there are quite a few charts on the homepage of Cloudera Manager upon initial install. I've since removed the admin account and replaced it with my own, and lost most of the charts in the process. Where can I find some neat charts to plug in without building them myself? Thanks!
... View more
Labels:
11-30-2019
10:08 PM
I ended up loading up an older snapshot on vCenter to fix it. Prior to the issue, I had stopped the CM service roles (Host monitor, alerts, etc), and put the CM host into maintenance mode. (I have Cloudera Manager as a managed host in my list of All Hosts). Upon restarting my VMs (with all hosts managed by CM), the VM with CM booted up, and the cloudera-scm service was on, but it would not allow me to connect to the WebUI, so I guess it was in a decommissioned state still at that point. I wanted to know how make it exit maintenance mode (and allow me to connect to the WebUI again). It's worth it to note that still, after using parcels to update all other managed hosts to CDH 6.3.2, my CM host is still showing that it has no version of CDH installed, and I can't distribute and update to 6.3.2 while it functions as the Manager of the CDH. It's running 6.3.0 currently. Does the CM server need to be part of the CDH? I mean, the service roles are working just fine on it, and it's connected and heartbeating, but for whatever reason it doesn't think it has CDH running. Dunno!
... View more
11-30-2019
09:48 PM
Any luck getting it working? I'm still working on it to this day, but I think I'm gonna finish it tonight. If so, I can help you when I know my method works.
... View more
11-30-2019
05:36 PM
Hi. I just stopped all services and entered maintenance mode on all managed hosts to perform a restart on all my VMs. Upon restart, Cloudera Manager doesn't want to be recommissioned (after starting cloudera manager service). What's the command to recommission the CM host? Thanks!
... View more
Labels:
11-30-2019
04:27 PM
@MattWho I've been trying these steps and somehow the SAN keeps getting removed when I import/export to JKS. How do I get the SAN extension to be with the key inside of the keystore file? I'm totally stuck!
... View more
11-30-2019
02:16 PM
I've heard this is a "bug" with openssl/keytool. I'm following MattWho's article found here: How to create user generated keys for securing NiFi. I'm getting the following error on my NiFi WebUI:
Hostname nifi.taco.net not verified: certificate: sha256/5REuJXk5ayT2nW5J89AfpW/G3OzXY9lF4n2vE3OxHlE= DN: CN=nifi.taco.net, OU=project taco, O=taco, L=taco, ST=texas, C=US subjectAltNames: []
I'm guessing this is either because of the SAN info being removed when I use x509, or perhaps a misconfiguration in the Cloudera Flow Management NiFi Node config??
... View more
Labels:
11-27-2019
09:23 PM
At long last, I am one step away from having this thing working. My question: is lack of SubjectAlternativeName in the cert keeping this from working, or is it something else? The hostname is correct and is working to direct to the correct IP. Is having the IP address of the host in the SAN going to fix this issue? I'm authenticating via LDAP. LDAP doesn't need access to port 8443 does it? I had this working before with NiFi Toolkit CA standalone, and LDAP was working with that. So I'm pretty sure it's an issue with the SubjectAlternativeNames not being present. Thank you Failed to replicate request GET /nifi-api/flow/current-user to nifi.taco.net:8443 due to javax.net.ssl.SSLPeerUnverifiedException: Hostname nifi.taco.net not verified: certificate: sha256/ulAHv1CZSCcrJL/d9rXRCVGFNtOqvKFXH2bB4Viz6/0= DN: CN=nifi.taco.net, OU=Project taco, O=taco, L=taco, ST=taco, C=US subjectAltNames: [] javax.net.ssl.SSLPeerUnverifiedException: Hostname nifi.taco.net not verified: certificate: sha256/ulAHv1CZSCcrJL/d9rXRCVGFNtOqvKFXH2bB4Viz6/0= DN: CN=nifi.taco.net, OU=Project taco, O=taco, L=taco, ST=taco C=US subjectAltNames: [] at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:316) at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:270) at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:162) at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257) at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135) at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) at okhttp3.RealCall.execute(RealCall.java:77) at org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:138) at org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:132) at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:647) at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:839) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
... View more
Labels:
11-26-2019
05:52 PM
I'm needing to clean a NiFi Node to have it stop getting errors. I've narrowed down the problem quite a bit (I'm trying to have it Authenticate via LDAP, with a self-signed SSL cert, both of these things are set up properly now). However, because I've been through so much troubleshooting and trial and error with this NiFi server, I have experienced WAY more hassle than I would've expected. I've been trying to get this to work for nearly a month now. SSL certs, truststore, keystore, node config (In CM) are all set up to the best of my ability. I stop the service roles in CM, put host in maintenance mode, restart the VM nifi is running on (to clear extra scm-agent processes). I've moved the flowfile.xml.gz, deleted users.xml, authorizations.xml, cleared the _repository folders out (moving them to a backup folder), cleared the archive folder, state/local folder. I've set initial admin to our proper LDAP identity. (I never get to this stage before the node shuts off now though) Jetty is reporting that the there is no valid keystore, but I am not sure that this is the cause of the effect of a different problem. I've been very careful to create the keystores exactly to specification following @MattWho 's article and have verified everything, also I had HTTPS working last night (csr worked, but I could not manage to log in due to "unverified keystore" on the client side I believe). Having as little experience with this sort of thing as I do, I have been so challenged and puzzled to get HTTPS set up for LDAP auth. I keep feeling like I'm only one or two steps away from having things working but then another problem springs up. Here is a piece of the log file from which I've scoured and this is the FIRST sign of something not going correctly: 7:04:23.327 PM INFO _nifi No Spring WebApplicationInitializer types detected on classpath 7:04:23.404 PM INFO ContextHandler Started o.e.j.w.WebAppContext@5b16e486{nifi,/nifi,file:///var/lib/nifi/work/jetty/nifi-web-ui-1.9.0.1.0.1.0-12.war/webapp/,AVAILABLE}{/var/lib/nifi/work/nar/framework/nifi-framework-nar-1.9.0.1.0.1.0-12.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-ui-1.9.0.1.0.1.0-12.war} 7:04:23.719 PM INFO AnnotationConfiguration Scanning elapsed time=181ms 7:04:23.748 PM INFO _nifi_api No Spring WebApplicationInitializer types detected on classpath Followed by this Error, which is followed by a MASSIVE list of missing beans which i have not included. Context initialization failed org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration': Unsatisfied dependency expressed through method 'setFilterChainProxySecurityConfigurer' parameter 1; nested exception is org.springframework.beans.factory.BeanExpressionException: Expression parsing failed; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.apache.nifi.web.NiFiWebApiSecurityConfiguration': Unsatisfied dependency expressed through method 'setJwtAuthenticationProvider' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jwtAuthenticationProvider' defined in class path resource [nifi-web-security-context.xml]: Cannot resolve reference to bean 'jwtService' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jwtService' defined in class path resource [nifi-web-security-context.xml]: Cannot resolve reference to bean 'keyService' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'keyService' defined in class path resource [nifi-administration-context.xml]: Cannot resolve reference to bean 'keyTransactionBuilder' while setting bean property 'transactionBuilder'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'keyTransactionBuilder' defined in class path resource [nifi-administration-context.xml]: Cannot resolve reference to bean 'keyDataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'keyDataSource': FactoryBean threw exception on object creation; nested exception is org.h2.jdbc.JdbcSQLException: Error opening database: "Could not save properties /var/lib/nifi/database_repository/nifi-user-keys.lock.db" [8000-176] Warn: Error opening database: "Could not save properties /var/lib/nifi/database_repository/nifi-user-keys.lock.db" [8000-176] Failed startup of context o.e.j.w.WebAppContext@2b85edc7{nifi-api,/nifi-api,file:///var/lib/nifi/work/jetty/nifi-web-api-1.9.0.1.0.1.0-12.war/webapp/,UNAVAILABLE}{/var/lib/nifi/work/nar/framework/nifi-framework-nar-1.9.0.1.0.1.0-12.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.9.0.1.0.1.0-12.war} And the final error before shutting down: 7:04:33.701 PM INFO _ No Spring WebApplicationInitializer types detected on classpath 7:04:33.755 PM INFO ContextHandler Started o.e.j.w.WebAppContext@490704a5{nifi-error,/,file:///var/lib/nifi/work/jetty/nifi-web-error-1.9.0.1.0.1.0-12.war/webapp/,AVAILABLE}{/var/lib/nifi/work/nar/framework/nifi-framework-nar-1.9.0.1.0.1.0-12.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-error-1.9.0.1.0.1.0-12.war} 7:04:33.787 PM WARN JettyServer Failed to start web server... shutting down.
java.lang.IllegalStateException: no valid keystore
at org.eclipse.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:50)
at org.eclipse.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:1071)
at org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:262)
at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:229)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:72)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:279)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:398)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:935)
at org.apache.nifi.NiFi.<init>(NiFi.java:158)
at org.apache.nifi.NiFi.<init>(NiFi.java:72)
at org.apache.nifi.NiFi.main(NiFi.java:297) 7:04:33.797 PM INFO NiFi Initiating shutdown of Jetty web server... So. With this, it is failing to even reach the stage where it will generate the users.xml and authorizations.xml file that I'm used to it auto-generating after I remove it. I've had to change the initial admin a couple of times, so I learned how to do that without reinstalling. I've removed the flowfile.xml.gz, which I suspect may be an issue here. As well as the _repository folders, which I was advised to do via other forum posts here, but I may have cleared more than I should have? I have backups saved. Really my big question is: what all CAN i remove/clean without NiFi not being able to recover / auto-gen new files on start. Any ideas would be super appreciated!
... View more
- Tags:
- NiFi
Labels:
11-21-2019
12:48 PM
CFM 1.0.1 Could someone please explain to me why, after reconfiguring to disable SSL, when I try to access the webUI it still is attempting to connect on port 8443? Is this a zookeeper issue? How can I fix this? I had another cluster previously that start out as SSL disabled, but after configuring it to have a cert via nifi toolkit CA, and getting it WORKING with LDAP auth, it would try to request things from port 8080 http, then after vote resolved it will kill the NiFi node. So here I am now. Was unable to get SSL verified while following the 5 year old guide courtesy of @MattWho (you are awesome, i really appreciate all the info you've shared) I followed that guide to a T but the server reporting the cert was not verified. Now I'm trying to just connect to the nifi node via HTTP, so i disabled all related SSL things in the config, yet: Cannot replicate request to Node nifi.domain.net:8443 because the node is not connected It still tries to connect to port 8443. Can someone tell me why it's trying to connect to the old port? Is this a zookeeper issue? If i could resolve this then I feel i could resolve all my other issues. Thanks!
... View more
11-20-2019
04:59 PM
Hey MattWho, I gotta say, THANK YOU so much for writing this guide. Seriously, this saved me sooo much time, having zero experience with setting up SSL chains. Really appreciate everything you've done to help me/us get nifi working!!
... View more
11-19-2019
05:37 PM
Hello. I'm curious what the repercussions are if Cloudera Manager is failing to find it's version number. It should be running 6.3.0 right now, as it was previous to the install. However, something isn't correct. Long story, but I unintentionally broke my cluster by accidentally overriding the cluster to Java 8 when I should have not created a master override (All are running Java 11 and some services of dedicated Java 8 installed). After fixing this issue, CM fails to report version. I have a deadline to reach right now and am going to continue working hoping that this will not be a big issue due to something unforeseen. Everything seems to be running fine other than supervisord and failing to report CDH version number.
... View more
Labels:
11-18-2019
01:55 PM
Thanks so much Matt!! You've been a huge help in getting my mind wrapped around all of this. If you can't tell, I'm a bit new to authentication and authorization! I really appreciate everything you've done to point me in the right direction. If things go as planned, I *should* have a NiFi node working by the end of the day! Here's one more for you, though. I'm having to reinstall the NiFi service on a new host because some property values are messed up after all of my tinkering trying to get things to work. The node will start up, HTTPS will be working, I can successfully log into the WebUI, but then after ~5 minutes or so, something happens and it reverts to trying to use HTTP and reports that it is trying to connect to the site on the HTTP port and fails to do so. I believe everything is configured in CM properly but there are some local configs that aren't right, or profiles that need to be deleted in order for new (correct) profiles to be created automatically. If this rings any bells I would love to learn more about how to fix it, but for now it seems the best thing to do is to do a fresh install. Aloha!
... View more
11-18-2019
01:22 PM
Hello! I've got a misconfigured NiFi service that has been failing after a couple of minutes of being online due to something I haven't been able to diagnose, and I think the best option at this point is to just uninstall and reinstall. My question is, what is the best practice for doing so, so that it will not leave remnants of past configuration? I want to add a new NiFi service, fresh out of the box, to my cluster, and remove everything that was associated with the old one. If you could share the best practices for doing such with me, that would be greatly beneficial!
... View more
Labels:
11-15-2019
02:57 PM
I assumed as such, actually! My bad for not communicating that I'm configuring from within CM. We don't have a support license yet, but are hoping to ASAP. Just for basic, SIMPLE LDAP authentication should I need to configure safety valves? It seems like I should be able to get it working with the configurations available. However, it's getting stuck somewhere. I can connect to the server via HTTPS and I get a login screen. Should I be able to log in without LDAP using the initial admin + master password?
... View more