Community Articles

Find and share helpful community-sourced technical articles.
Labels (1)
Super Guru

Simple Apache NiFi Operations Dashboard - Part 2

Part 1:

To access data to display in our dashboard we will use some Spring Boot 2.06 Java 8 microservices to call Apache Hive 3.1.0 tables in HDP 3.0 on Hadoop 3.1.

We will have our web site hosted and make REST Calls to Apache NiFi, our microservices, YARN and other APIs.

As you can see we can easily incorporate data from HDP 3 - Apache Hive 3.1.0 in Spring Boot java applications with not much trouble. You can see the Maven build script (all code is in github.)

Our motivation is put all this data somewhere and show it in a dashboard that can use REST APIs for data access and updates. We may choose to use Apache NiFi for all REST APIs or we can do some in Apache NiFi. We are still exploring. We can also decide to change the backend to HBase 2.0, Phoenix or Druid or a combination. We will see.


Spring Boot 2.0.6 Loading


JSON Output



Spring Boot Microservices and UI

To start I have a simple web page that calls one of the REST APIs.

The microservice can be run off of YARN 3.1, Kubernetes, CloudFoundry, OpenShift or any machine that can run a simple Java 8 jar.

We can have this HTML as part of a larger dashboard or hosted anywhere.



For Parsing the Monitoring Data We have some schemas for Metrics, Status and Bulletins.


Now that monitoring data is in Apache Hive, I can query it with easy in Apache Zeppelin (or any JDBC/ODBC tool)


Apache Zeppelin Screens


We have a lot of reporting tasks for Monitoring NiFi


We read from NiFi and send to NiFi, would be nice to have a dedicated reporting cluster


Just Show Me Bulletins for MonitorMemory (You can see that in Reporting Tasks)




NiFi Query To Limit Which Bulletins We Are Storing In Hive (For Now Just grab Errors)


Spring Boot Code for REST APIs


Metrics REST API Results


Bulletin REST API Results


Metrics Home Page


Run The Microservice

java -Xms512m -Xmx2048m -Dhdp.version=3.0.0 -jar target/operations-0.0.1-SNAPSHOT.jar


Maven POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi=""


 <description>Apache Hive Operations Spring Boot</description>





With some help from the Internet, we have a simple Javascript to read the Spring Boot /metrics REST API and fill some values:

HTML and Javascript (see src/main/resources/static/index.html)

<div id="output" name="output" style="align: center; overflow:auto; height:400px; width:800px" class="white-frame">
  <ul id="metrics"></ul>  

<script language="javascript">var myList = document.querySelector('ul');var myRequest = new Request('./metrics/');fetch(myRequest).then(function(response) { return response.json(); }).then(function(data) {for (var i = 0; i < data.length; i++) {var listItem = document.createElement('li');listItem.innerHTML = '<strong>Timestamp' + data[i].timestamp + '</strong>Flow Files Received: ' +data[i].flowfilesreceivedlast5minutes + ' JVM Heap Usage:' + data[i].jvmheap_usage +' Threads Waiting:' + data[i].jvmthread_statestimed_waiting +' Thread Count: ' + data[i].jvmthread_count +' Total Task Duration: ' + data[i].totaltaskdurationnanoseconds +' Bytes Read Last 5 min: ' + data[i].bytesreadlast5minutes +' Flow Files Queued: ' + data[i].flowfilesqueued +' Bytes Queued: ' + data[i].bytesqueued;myList.appendChild(listItem);}});</script>


Example API Calls to Spring Boot




TODO: We will add more calls directly to REST APIs of Apache NiFi clusters for display in our dashboard.

REST API for NiFi of Interest


Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.
Version history
Last update:
‎06-09-2020 07:18 AM
Updated by:
Top Kudoed Authors