Member since
09-27-2019
48
Posts
0
Kudos Received
0
Solutions
06-01-2020
08:43 PM
Is this (now) considered a NiFi "anti-pattern"? Do you have any idea how to do this using NiFi Record serialization services? I'm under the impression that creating thousands of content files is not the best practice by today's standards, but I'm not sure how to use InvokeHTTP on a full set of records without splitting it into many flowfiles. Any ideas?
... View more
04-20-2020
05:11 PM
Had a file system crash, rebuilt Cloudera Manager host with the same everything--except the private TLS key for HTTPS. My psql db is still in tact, so the rebuilt CM host is connecting to it and getting configurations from the previous server. Is this a bad idea? Should I create a new database for the rebuilt CM host, or can I have it connect to the previous db and disable HTTPS manually somehow so I can access the web console and update it from there? Would it be a better idea to just regenerate new keys with the same password? I've never done this before, so I am unclear on any potential conflicts if a new Cloudera Manager server connects to an existing Cloudera Manager database.
... View more
Labels:
- Labels:
-
Cloudera Manager
04-17-2020
10:03 PM
Thank for you this reply! This has been quite difficult for me to troubleshoot, but I finally figured it out. These machines I've been using had chrony on them all along, but the previous machines I set up did not have chrony installed. Chrony and ntpd were both enabled, and ntpd was getting exited on reboot. Because the host monitor issues "ntpq -np", and ntpd was loaded but inactive, it would report a failure to query the server, even though chrony was running. I had no idea that chrony was installed, and thus, the whole problem could've been solved by just disabling/uninstalling ntpd. I spent WAY too many hours to come to such a simple solution. It may be very helpful to someone who doesn't understand network time protocols very well if there was a suggestion to explain potential conflicts between ntpd and chronyd in the documentation, or even to take a second to check which (if any) you already have installed. Maybe it won't be an issue for most people, but for me, assuming that I didn't have chrony already running cost me a bunch of time getting my cluster healthy. I would check, find ntpd dead, see no problems reported on Host Monitor, wonder why the hell ntpd died, kill ntpd, run ntpdate, restart ntpdate, restart scm-agent, and that would "fix" it, but on reboot it would go back to using chrony and exit ntpd, and host monitor would report failure to query ntp service, even though the machine was using chrony and synced just fine all along. I appreciate your help!
... View more
04-15-2020
04:46 PM
Continuously my NTP service seems to die on some of my VMs but not others, and I cannot seem to figure out why. I've followed every piece of advice I've been given so far to no avail. NTP service is dying without drifting it seems. When I use `ntpdate` to re-sync to the pool, it's never been more than ~50ms offset. yum install ntp -y systemctl start ntpd systemctl enable ntpd ntpdate -u pool.ntp.org hwclock --systohc The VMs are hosted across 3 different ESXi servers that are connected via vCenter. The ESXi hosts all have an NTP service running and enabled, and are syncing to the same external NTP server. Is there anything in `ntp.conf` that should be changed from default in order to prevent complications? I have zero clue why ntp service will randomly die on some hosts, yet I have some hosts that have had the service running without issue for 1+ month, but only if CM doesn't have the proper PID for the ntp service. As I'm writing this I think I may have solved it. By default ntp.conf is set to not allow queries. So is Cloudera querying ntp for it's offset to the point that it kills the service as a security measure or something? from ntp.conf: # Permit time synchronization with our time source, but do not # permit the source to query or modify the service on this system. restrict default nomodify notrap nopeer noquery Is Cloudera making NTP kill itself by querying it for it's offset? Please update documentation to reflect this if this is the case!
... View more
Labels:
- Labels:
-
Cloudera Manager
01-29-2020
11:10 PM
For a single node nifi cluster, is 1 ZK host alright?
... View more
01-17-2020
07:24 PM
Any updates on this Matt? We've switched to Java 11 and would like to use CDF/CFM with NiFi 1.10.0!
... View more
01-04-2020
03:58 PM
Thanks Eric! That is indeed the solution I was looking for. I ended up figuring it out myself a week or so ago before I saw your comment here just now. The cool thing was that after getting CM up and running, it was able to eventually update the new FQDNs for all of the clustered hosts on it's own (even though initially it was still displaying the old hostnames from before our migration)!
... View more
12-27-2019
11:35 PM
Aloha. After a recent server migration, all of the hostnames on our cluster got a slight modification. After getting the Cloudera Manager Server to connect to it's required database correctly, it still wants to connect to the old hostnames for everything else in the cluster. Everything is already configured for them to work together, so it should be as simple as updating the URL to connect to the hostname (rather than reinstalling cloudera-scm-agent, etc). Is there a way to do this from within Cloudera Manager? How?
... View more
Labels:
- Labels:
-
Cloudera Manager
12-27-2019
12:39 AM
As the title explains, we have moved our cluster from `taco.net` to `dev.taco.net`, and in the process this has made our Cloudera Manager server unable to start up because it continues to try to connect to database.taco.net instead of database.dev.taco.net. Hosts files are updated on all servers, network interfaces are configured to their new IPs. I would imagine there is something I should update on the Cloudera Manager server, but I'm not sure where to look to update the URL.
... View more
Labels:
- Labels:
-
Cloudera Manager
11-30-2019
09:48 PM
Any luck getting it working? I'm still working on it to this day, but I think I'm gonna finish it tonight. If so, I can help you when I know my method works.
... View more