This also includes an analysis on the fly for showing odds on a Craps game.
This example shows a simple use of Nifi - HDF - handling multiple streams of Dice data - each one simulating a separate Craps table – showing a Monte-Carlo simulation and results of a 1000 run – emulating throws each second.
To demonstrate this capability we generate some random dice data, each stream generation uses independent thread. We throttle the threads to sleep for a second between throws, mainly to demonstrate an ongoing stream of data over time.
Source for data generation:
We use Nifi to create a streaming flow of that data as it is being generated.
This simulation will use the following Nifi processors:
- HandleHttpRequest (Starts an HTTP Server and listens for HTTP Requests)
- RouteOnAttribute (Routes FlowFiles based on their Attributes using the Attribute Expression Language)
- ExecuteStreamCommand (Executes an external command on the contents of a flow file, and creates a new flow file with the results of the command.)
- HandleHttpResponse (Sends an HTTP Response to the Requestor that generated a FlowFile)
- Site-to-site (To send data from one instance of Nifi to Another)
You can use a Template to handle each stream with individual Nifi flow from:
https://github.com/eorgad/Dice-nifi-streams-example/blob/master/Multi-stream-dice-example.xml
The Nifi flow would look as follows when importing the xml template:
Web services:
We can use Nifi to host web services either on your HDP instance (can use edge node or the same host serving Ambari), or a stand along server.
However in many cases organizations already use web servers internally and externally so you can use an existing instance to link the UI example or generate one using the following steps:
Set up a local web service: You can set up you web services either on the a server or on your local mac for demo purposes.
2.1. Installation on CentOS server: To install apache, open terminal and type in this command:
sudo yum install httpd
2.2. Make configuration changes for your web services:
vi /etc/httpd/conf/httpd.conf
Place the content of the UI folder in the DocumentRoot location to be accessed via the webserver DocumentRoot "/var/www/html"
2.3. Start apache by running
sudo service httpd start
Our simple architecture will look as follows:
3. You can import the java project into eclipse or run the TwoCrapsTest from the cli to generate two files that Nifi would stream to your web instance. In the template there is a port that you can use to stream the feed via site-to-site to another Nifi instance, such as instance running on the edge node of your HDP instance (used HDP 2.5 sandbox VM for this example)
When launching the following example you would now be able to view real time streaming data from Nifi handled by your webserver showing a real time analysis of a game of Craps. Each stream represents one table. The bar shows you an accumulation of $ win or lost relating to the theoretical gamble on one of the options: pass line, six, eight, five, nine etc.
This simulation will run only 1000 iterations per thread (table in this case), so to get better approximation to the odds, you can increase this Monte Carlo simulation and run it million throws per thread.
The following is the result of launching your index.html with the two streams displayed in real time as they arrive:
The following is a Bell curve with reference to the UI/dice8.html
