Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar
Super Guru

Apache NiFi evolution from version 1.2 included in HDF 3.0 and version 1.5 included in HDF is significant. I find myself quite often puzzled when required to provide differences between releases and just reading the release notes history at https://cwiki.apache.org/confluence/display/NIFI/Release+Notes and looking at the latest list of NiFi processors is not trivial to determine which new processors were added.

I put together matrix which I hope will help developers to take advantage of new processor to improve old and develop new flows.

In a nutshell, main functionality added is around:

  • AzureEventHub
  • Kafka 0.11 and 1.0 processors
  • Record processors
  • RethinkDB
  • Flatten Json
  • Execute Spark Interactive
  • Execute Groovy Script

My favorite improvements are around record processors, flattening JSON and executing Spark interactively.

The following is a table of the matrix, arranged alphabetically from A-D:

For

NiFi 1.5NiFi 1.4NiFi 1.3NiFi 1.2
AttributeRollingWindowAttributeRollingWindowAttributeRollingWindowAttributeRollingWindow
AttributesToJSONAttributesToJSONAttributesToJSONAttributesToJSON
Base64EncodeContentBase64EncodeContentBase64EncodeContentBase64EncodeContent
CaptureChangeMySQLCaptureChangeMySQLCaptureChangeMySQLCaptureChangeMySQL
CompareFuzzyHashCompareFuzzyHashCompareFuzzyHashCompareFuzzyHash
CompressContentCompressContentCompressContentCompressContent
ConnectWebSocketConnectWebSocketConnectWebSocketConnectWebSocket
ConsumeAMQPConsumeAMQPConsumeAMQPConsumeAMQP
ConsumeAzureEventHub
ConsumeEWSConsumeEWSConsumeEWSConsumeEWS
ConsumeIMAPConsumeIMAPConsumeIMAPConsumeIMAP
ConsumeJMSConsumeJMSConsumeJMSConsumeJMS
ConsumeKafkaConsumeKafkaConsumeKafkaConsumeKafka
ConsumeKafka_0_10ConsumeKafka_0_10ConsumeKafka_0_10ConsumeKafka_0_10
ConsumeKafka_0_11ConsumeKafka_0_11ConsumeKafkaRecord_0_10ConsumeKafkaRecord_0_10
ConsumeKafkaRecord_0_10ConsumeKafkaRecord_0_10
ConsumeKafkaRecord_0_11ConsumeKafkaRecord_0_11
ConsumeKafka_1_0
ConsumeKafkaRecord_1_0
ConsumeMQTTConsumeMQTTConsumeMQTTConsumeMQTT
ConsumePOP3ConsumePOP3ConsumePOP3ConsumePOP3
ConsumeWindowsEventLogConsumeWindowsEventLogConsumeWindowsEventLogConsumeWindowsEventLog
ControlRateControlRateControlRateControlRate
ConvertAvroSchemaConvertAvroSchemaConvertAvroSchemaConvertAvroSchema
ConvertAvroToJSONConvertAvroToJSONConvertAvroToJSONConvertAvroToJSON
ConvertAvroToORCConvertAvroToORCConvertAvroToORCConvertAvroToORC
ConvertCharacterSetConvertCharacterSetConvertCharacterSetConvertCharacterSet
ConvertCSVToAvroConvertCSVToAvroConvertCSVToAvroConvertCSVToAvro
ConvertExcelToCSVProcessorConvertExcelToCSVProcessorConvertExcelToCSVProcessorConvertExcelToCSVProcessor
ConvertJSONToAvroConvertJSONToAvroConvertJSONToAvroConvertJSONToAvro
ConvertJSONToSQLConvertJSONToSQLConvertJSONToSQLConvertJSONToSQL
ConvertRecordConvertRecordConvertRecordConvertRecord
CreateHadoopSequenceFileCreateHadoopSequenceFileCreateHadoopSequenceFileCreateHadoopSequenceFile
CountText
DebugFlowDebugFlowDebugFlowDebugFlow
DeleteDynamoDBDeleteDynamoDBDeleteDynamoDBDeleteDynamoDB
DeleteGCSObjectDeleteGCSObjectDeleteGCSObjectDeleteGCSObject
DeleteHDFSDeleteHDFSDeleteHDFSDeleteHDFS
DeleteElasticsearch5DeleteElasticsearch5
DeleteRethinkDBDeleteRethinkDB
DeleteS3ObjectDeleteS3ObjectDeleteS3ObjectDeleteS3Object
DeleteMongo
DeleteSQSDeleteSQSDeleteSQSDeleteSQS
DetectDuplicateDetectDuplicateDetectDuplicateDetectDuplicate
DistributeLoadDistributeLoadDistributeLoadDistributeLoad
DuplicateFlowFileDuplicateFlowFileDuplicateFlowFileDuplicateFlowFile


1,694 Views
Version history
Last update:
‎09-16-2022 01:42 AM
Updated by:
Contributors