In this article we will focus on configuring the IBM Bluemix IoT Cloud to except simulated IIoT messages. We will configure the Bluemix IoT cloud in gateway mode so that devices can be auto registered when IIoT messages are sent from the device.
IBM Bluemix IoT account. You can create an account here
Hortonworks Data Flow (HDF) 2.x or Apache Nifi 1.1.0
Setup The Gateway Definition on Bluemix IoT Cloud
Login to your Bluemix IoT account and navigate to your IoT Dashboard, which will look like this:
Click on the IoT icon (which looks like a microchip), which will navigate us to the Devices section. This is where you can define all the devices that you want to communicate with the Bluemix IoT cloud. We will define our gateway device here.
Click on "Add Device"
Click "Create device type". You will have the option to select a gateway or device type. Each device must have a type. In our case we are going to create a gateway device type called nifi-gateway and click next
We are going to define an attribute that will be asked each time a nifi-gateway type is selected. This attribute is called "Descriptive Location". This will allow us to specify a location for each nifi-gateway that we have deployed. This will allow us to track and report on each nifi-gateway and device that's using that gateway
Now we are going to provide the Descriptive Location a sample value, which should be overridden each time a new nifi-gateway type is used. The format of our value is country, plant number and the zone that the device is located
Optionally you can add metadata about the device using JSON. But, we aren't going to use it. So, just click "Create"
Add the Newly Created Gateway Device
Select the "nifi-gateway" device and click "Next"
We will now provide a Device ID and Descriptive location. We will set both values to us-plan14-zone12. We will scrip the Metadata section so click "Next" two times.
On the security screen you have the ability to auto-generate or define a security token. We will auto-generate. Therefore, you can just click "Next"
Now you will see a summary of your settings. Click "Add" to add the device.
Now you will see the self generated authentication token, which you will need to connect Nifi to the Bluemix IoT Cloud. Note, if you lose the token you will need to re-register the device to generate a new token.
Setup HDP/Nifi Flow
... View more
Can you access Job Metrics via JMX? I would like to see all running jobs and how long they have been running, how many containers were allocated,etc. It looks like the ResourceManager REST API provides what I'm looking for, but was wondering if it was available in JMX.
... View more
The customer wants to setup a backup to a backup cluster. They want to know how many NameNodes are required on the Backup Cluster. They want to know if there is some rule of thumb. The constraints for the backup cluster is below
Backup window: 1am-5am ET (12am-4am CT) Want to have a yarn job that runs Falcon backup jobs in another queue when the cluster isn’t busy - that way it doesn't pile up for the night. The clusters will be in Amazon, so they want the backup cluster to be backed by Amazon S3 storage (being used as HDFS)
... View more