Support Questions
Find answers, ask questions, and share your expertise

Migrating data from RDBMS to ElasticSearch using Apache NIfi

Migrating data from RDBMS to ElasticSearch using Apache NIfi

New Contributor

Hi Team,

 

We are trying to migrate data from RDBMS to elastic search using Apache Nifi. We have created pipelines in Nifi and are able to transfer data but are facing some issues and wanted to check if someone already got over them.

Please provide inputs on the below items.

  1. How to avoid auto-generating _id in elastic search. We want this to be set from a DB column. We tried providing the column name in the "Identifier Record Path" attribute in the PutElasticSearchHTTPRecord processor but were getting an error that the attribute name is not valid. Can you please let us know the acceptable format.
  2. How to load nested objects into the index using NIfi? We are looking to maintain one to many relationships in the index using nested objects but were unable to find a configuration to do so. Do we have any processors to do this in Nifi? Please let us know.

 

Thanks in Advance!