Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Problems extracting Mongo data to insert into PostgreSQl

Problems extracting Mongo data to insert into PostgreSQl

New Contributor

Problems extracting Mongo data to insert into PostgreSQl

Hello people. I started using NiFi less than 1 week ago.
I have already made some advances but have come across a problem in which I am not finding a solution.

We have a LOG table that is stored in MongoDB and we need to extract Mongo data to put in Postgres.

We did several tests, of which:
- When I try to extract JSON files from Mongo, it generates numerous equal files (containing the same content);
- When I try to integrate MongoDb into Postgres, the same thing happens, it repeats the same record over and over.

In NiFi, I have used so far (in different processes):
1 - GetMongoDB => PutFile (to generate files)
2 - GetFile => PutDatabaseRecord (to read file and store in database)
3 - GetMongoDb => PutDatabaseRecord (To read from Mongo and store in Postgres)

In neither attempt did I get a good result.

And there is one more detail, which is the most important now.
If I try to insert Mongo pro Postgres I can only do it if I play the file structure that is in MONGO.

This is a problem for me because I wanted to take all the Mongo array {} and store it as a String for example.

Has anyone had a problem similar to this that could help me?

1 REPLY 1

Re: Problems extracting Mongo data to insert into PostgreSQl

Community Manager

@EduSantana This seems to be more of a problem with MongoDb and less of a problem concerning the use of Nifi. If that's true, your question would be better handled on the slack community for MongoDb.

 

Bill Brooks, Community Manager
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Don't have an account?
Coming from Hortonworks? Activate your account here