Good day everyone, here are the details of my problem:
- NiFi 1.6
- A flow with 30 process groups
- Each group contains, ConsumeAMQP, MonitorActivity, JoltTransform, SplitJson, ReplaceText, ConvertRecord, PutMongoRecord, and PutCassandraQL processors
When the flow starts, I'm immediately locked out of the UI with "too many open files" errors - note that this happens when there is no data coming from the AMQP queues, which I assume means there are no flow files being created. I've followed the best practices to increase file handle limits just in case, but it's made no difference. Experimenting with using the API to stagger the start order of the processors has shown that the problem happens when all 30 PutCassandraQL processors are started. Has anyone ever encountered this issue?