Actually, an easier way to ignore the column name duplication and still process the columns correctly, would be to use a schema to describe your data.   For example, say you have the following CSV:   col_a,col_b,col_b 1,2,3 4,5,6   You can configure your CSVReader with the following:       And the data will be processed correctly:   HTH, André       ... View more

@spserd ,   You're right. The Kafka 0.10 and 0.11 connectors are very old have been discontinued.   Cheers, André   ... View more

Hi @araujo ,   This is exactly what I was looking for. Thanks very much! ... View more

@gfsupp, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  ... View more

I've added a "remove" operation to the chain to solve the problem I mentioned in the last post, removing the "null" in the first position of the "measures" array: [ { "operation": "shift", "spec": { "LevelID": "assetId", "PLCTime": "datapoints[0].timestamp", "Data*Type": "datapoints[0].measures[&(0,1)].name", "Data*": "datapoints[0].measures[&(0,1)].value" } }, { "operation": "remove", "spec": { "datapoints": { "*": { "measures": { "0": "" } } } } } ]   This should give you the exact output you were looking for.   HTH, André   ... View more

Thanks a lot, Ran ... View more

@akylai ,   The Cloudera Quickstart docker image contains a very old version from CDH and is no longer supported. If you want to try the Cloudera Data Platform (CDP), I'd recommend you use the Cloudera Private Cloud trial: https://www.cloudera.com/downloads/cdp-private-cloud-trial.html   Regards, André ... View more

@CloudNaive ,   It seems that the "Message Key Field" property doesn't work with a nested structure. This property must refer to a top level field of the record that contains the key value. In the example below I used UpdateRecord to create a top level field called "id" and update it with the value of "metadata.aggregateId". Them, on the PublishKafkaRecord I was able to use the "id" field as the Kafka message key.   Cheers, André   ... View more