We have recently upgraded to HDF 3.5.1 with NiFI v. 1.11.4.3.5.1.0-17.    Of five clusters, all with widely differing personalities, we've experienced an  issue on three of the clusters where the canvas becomes unavailable and the error given when we try to log in is "Error Matrix is Singular.   In each case from the logs;   ERROR [Timer-Driven Process Thread-12] o.a.n.r.ambari.AmbariReportingTask AmbariReportingTask[id=3b80ba0f-a6c0-48db-b721-4dbc04cef28e] Error running task AmbariReportingTask[id=3b80ba0f-a6c0-48db-b721-4dbc04cef28e] due to org.apache.commons.math3.linear.SingularMatrixException: matrix is singular   it appears to be an issue with the Ambari Reporting Task. In one case the issue "resolved itself". In the other two instances, restarting the Controller Service "Ambari Reporting Task, the issue was cleared.   I'm still researching this, but I can find no Customer Support cases, nothing in Community Searches or the internet in general on this "Matrix is Singular: error.   Restarting the Reporting Task helps, but is not a fix. Is this a tuning issue or a setting that might need adjusting in the new version of NiFi? Does anyone have any experience with this issue? ... View more

We recently upgraded from HDF 3.2.0 to 3.4.1 - In the process Ambari Metrics Collector failed to start. We've seen this before and most times /.../hbase/hbase.id and hbase.version have been deleted. This hbase-ams won't start. Having kept s copy of these files after past issues, We replaced the files and restarted AMC which started, however we seem to have lost all historical data. We did not delete any hbase data for files. Is it common for the hbase files to get deleted? If replacing the hbase.id and hbase.version is incorrect, what's the fix? Might there be a way to recover the historical metric data? ... View more

In Nifi my active thread count on one of our clusters won't go above 30. How can I increase the number of Active Threads? I have two clusters, that SEEM to be configured the same. On one the Active Thread count in Nifi won't go above 30. In the other we've seen as many as 70 Active Threads. Memory is the same in each cluster, but one cluster has more cores. Is this a setting or a function of available resources (possibly cores)? ... View more

We’re running Hortonworks Data Flow (HDF) Nifi actually. We have recently experienced Ambari Metrics Collector (AMC) HBase corruption. As per the multitude of posted fixes we deleted hbase and hbase-tmp and restarted. Great AMC is working again. But now I’ lost my history. This has happened 3-4 times on different HDF clusters In addition in this instance Ambari displays what appears to be a medical kit logo suggesting there are still issues. Is there any good way to recover from this corruption? Hbase hbc is not of much use although is does show that /hbase does not exist in ZooKeeper even on good clusters. For the moment I’d like to get the suitcase to go away. Long term does anyone have suggestions on avoiding AMC corruption? ... View more