question Re: How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server? in Support Questions

How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

Fayza — Fri, 01 Dec 2023 13:14:55 GMT

Hey everyone,

I want to fetch all databases and its records from CouchDB (NON-SQL) and then view all records in my Apache NiFi, after confirmation I want to send all records to my Server database.?
I have not found processors of GetCouchDB. Share your thoughts that how we can do such process by using other processors ?

Is there anyone to know about this process.

#How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

Re: How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

cjervis — Fri, 01 Dec 2023 13:31:28 GMT

Welcome to the community @Fayza. While you wait for someone more knowledgeable to answer, I did find an article that hopefully gets you started in the right direction.

Getting Started with NiFi and Couchbase Server - Couchbase blog

You can also try asking on their forums.

Re: How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

Fayza — Fri, 01 Dec 2023 14:06:18 GMT

Thank you @cjervis for responding.

Actually, There is no any processor for CouchDB.

Its mandatory to use CouchDB.

So that, i purposed a solution that:
By using InvokeHTTP:

- setting the method, url, request username, request password. Got the successful flow. But it retrieve the (0 bytes of data). Although I cross check the URL with Username and Password in my POSTMAN. It contains data.

POSTMANAPACHE NiFi Flow

above are the images that shows that in POSTMAN it contains set of information but in NiFi flow contains zero bytes data.

InvokeHTTP(1)InvokeHTTP(2)

Here is the LogAttribute Processors

LogAttriutes

Please let me know if the information is missing? so that we will find solution of this query.

Thank you @cjervis again

Re: How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

SAMSAL — Fri, 01 Dec 2023 15:30:22 GMT

Hi @Fayza ,

I assume you set the credentials in the Request Username & Request Password. Can you set the "Response Generation Required" to true? this will make sure the processor is capturing all kind of responses regardless of the Http status. Another thing do you have to have any special headers attribute that needs to be part of the request ? for example Content-Type which corresponds to Request Content-Type property , or any custom header?

Re: How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

MattWho — Fri, 01 Dec 2023 19:37:05 GMT

@Fayza

Apache CouchDB looks very limited in its query capabilities.

But it does appear that a user in the community created nifi-couchDB components:
https://github.com/wasperen/nifi-couchdb

You could try adding this nar to your Apache NiFi installation to see if these components can satisfy your use case.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Re: How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

Fayza — Wed, 06 Dec 2023 10:37:06 GMT

Hi @SAMSAL

Thank you for responding.

all these "Content-Type which corresponds to Request Content-Type property , or any custom header?" can be manged by Apache NiFi by Default.

Re: How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

Fayza — Wed, 06 Dec 2023 10:39:03 GMT

@MattWho Thank you for replying.

I'm already trying this "https://github.com/wasperen/nifi-couchdb" part. can you make it more easier so that will be helpful.

Re: How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

SAMSAL — Wed, 06 Dec 2023 15:03:13 GMT

Hi @Fayza ,

You need to setup the Content-Type as required by the API. There should be "Request Content-Type" property where you can set the value. Also any custom header values can be added as Dynamic Property. The invokehttp processor should be very flexible to accommodate the different API requirements and request types.