Member since
08-02-2019
131
Posts
93
Kudos Received
13
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3288 | 12-03-2018 09:33 PM | |
4072 | 04-11-2018 02:26 PM | |
2282 | 05-09-2017 09:35 PM | |
1024 | 03-31-2017 12:59 PM | |
1961 | 11-21-2016 08:58 PM |
12-04-2018
07:25 PM
@Michael Bronson The page Designing a ZooKeeper explains that a 3 node cluster an survive 1 node failure. A 5 node cluster can survive 2 node failures.
... View more
12-04-2018
07:21 PM
@Michael Bronson A 3 node ZK cluster can survive the loss of one ZK node.
... View more
12-04-2018
07:21 PM
@Michael Bronson Physical machines are better. ZooKeeper is very sensitive to disk read latency. Virtual machines are typically connected to a networked file system. Writes to the file system can be delayed if the network connecting the VM to the networked file appliance is busy. Also VMs on hypervisors can do other fancy tricks like migrating which can cause zookeeper to fail. If you do use VMs you will need to modify some of the connection timeouts and shut off VM migration. VMs should be hosted on different hypervisors.
... View more
12-04-2018
07:16 PM
1 Kudo
@Michael Bronson ZooKeeper needs an odd number of hosts so it can build a quorum. A 3 node cluster can survive the loss of 1 node. It will fail if there is a simultaneous loss of 2 nodes (for example a node fails during an upgrade). If zookeeper goes down the brokers will not operate. Designing a ZooKeeper deployment explains: "For the ZooKeeper service to be active, there must be a majority of non-failing machines that can communicate with each other. To create a deployment that can tolerate the failure of F machines, you should count on deploying 2xF+1 machines. Thus, a deployment that consists of three machines can handle one failure, and a deployment of five machines can handle two failures. Note that a deployment of six machines can only handle two failures since three machines is not a majority. For this reason, ZooKeeper deployments are usually made up of an odd number of machines."
... View more
12-03-2018
09:33 PM
Depending on how many tenants you have using ZooKeeper you may be just fine with 3 nodes. In ZooKeeper, more nodes doesn't always yield better performance because of the communication overhead. Adding more nodes decreases write performance because of the node to node communication required to synchronize across the cluster. A few things to consider: 1. ZooKeeper will be more fault tolerant with 5 nodes vs 3 nodes. A 3 node cluster can only tolerate one down node before it loses its quorum. 2. Get the best performance out of your current 3 node deployment by following best practices. 4. Look at the current 3 node cluster performance under existing load and see the capacity you have. Check out this article. Add new kafka nodes and see how performance is affected. 3. Zookeeper needs an odd number of nodes and you will most likely not need to have more than 7. 4. Later versions of Kafka do not rely on zookeeper for consumer offsets. How Kafka uses ZooKeeper. This article describes the 0.10 release and later 5. Consider upgrading to HDP 3.0 and use Streams Messaging Manager. It makes managing Kafka a lot easier, but it only works on HDP 3.0 and above. Best of luck on your Kafka journey!
... View more
07-15-2018
02:53 AM
@Michael Dennis "MD" Danang I think there may have been an issue generating the test data. It looks like some of the benchmark tables did not get generated correctly. Try the generation again.
... View more
04-11-2018
02:26 PM
1 Kudo
I resolved the issue by using a different browser. The version of Internet Explorer I was using did not work. I resolved the issue by allowing the 8080 port through the windows firewall and then connecting remotely using Chrome.
... View more
04-10-2018
06:18 PM
I tried on Windows Server 2012 and had the same issue. I am creating my windows instance in AWS using the Windows Server with SQL Server.
... View more
04-10-2018
03:52 PM
@Wynner I see the same problem with both of those URLs.
... View more
04-10-2018
03:43 PM
2018-04-10 00:49:44,720 INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs:
2018-04-10 00:49:44,720 INFO [main] org.apache.nifi.web.server.JettyServer http://127.0.0.1:8080/nifi
2018-04-10 00:49:44,720 INFO [main] org.apache.nifi.web.server.JettyServer http://172.31.56.60:8080/nifi
2018-04-10 00:49:44,722 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap
... View more