Skip to main content
Fayza
Fayza
Silver ⭐️⭐️⭐️
December 1, 2023
Question

How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

Hey everyone,

I want to fetch all databases and its records from CouchDB (NON-SQL) and then view all records in my Apache NiFi, after confirmation I want to send all records to my Server database.?
I have not found processors of GetCouchDB. Share your thoughts that how we can do such process by using other processors ?

Is there anyone to know about this process.

 

#How to read all databases and its records from CouchDB and then by using Apache NiFi send all those records to my Server?

7 replies

cjervis
Community Manager
cjervis
Community Manager
December 1, 2023

Welcome to the community @Fayza. While you wait for someone more knowledgeable to answer, I did find an article that hopefully gets you started in the right direction.

Getting Started with NiFi and Couchbase Server - Couchbase blog

You can also try asking on their forums. 

Translate
Keep the questions coming
    Fayza
    FayzaAuthor
    Silver ⭐️⭐️⭐️
    December 1, 2023

    Thank you @cjervis  for responding.

    Actually, There is no any processor for CouchDB. 

    Its mandatory to use CouchDB.

    So that, i purposed a solution that:
    By using InvokeHTTP:

    - setting the method, url, request username, request password. Got the successful flow. But it retrieve the (0 bytes of data). Although I cross check the URL with Username and Password in my POSTMAN. It contains data.

    POSTMANPOSTMANAPACHE NiFi FlowAPACHE NiFi Flow

     

    above are the images that shows that in POSTMAN it contains set of information but in NiFi flow contains zero bytes data.

    InvokeHTTP(1)InvokeHTTP(1)InvokeHTTP(2)InvokeHTTP(2)

     

    Here is the LogAttribute Processors

    LogAttriutesLogAttriutes

     

    Please let me know if the information is missing? so that we will find solution of this query.

    Thank you @cjervis again

    Translate
    MattWho
    Titanium ⭐️⭐️⭐️
    MattWho
    Titanium ⭐️⭐️⭐️
    December 1, 2023

    @Fayza 

    Apache CouchDB looks very limited in its query capabilities.

    But it does appear that a user in the community created nifi-couchDB components:
    https://github.com/wasperen/nifi-couchdb

    You could try adding this nar to your Apache NiFi installation to see if these components can satisfy your use case.

    If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

    Thank you,
    Matt

    Translate
      SAMSAL
      SAMSAL
      Platinum ⭐️⭐️⭐️
      December 1, 2023

      Hi @Fayza ,

      I assume you set the credentials in the Request Username & Request Password. Can you set the "Response Generation Required"  to true? this will make sure the processor is capturing all kind of responses regardless of the Http status. Another thing do you have to have any special headers attribute that needs to be part of the request ? for example Content-Type which corresponds to Request Content-Type property , or any custom header?

      Translate
        Fayza
        FayzaAuthor
        Silver ⭐️⭐️⭐️
        December 6, 2023

        Hi @SAMSAL 

        Thank you for responding. 

        all these "Content-Type which corresponds to Request Content-Type property , or any custom header?" can be manged by Apache NiFi by Default.

        Translate
        SAMSAL
        SAMSAL
        Platinum ⭐️⭐️⭐️
        December 6, 2023

        Hi @Fayza ,

        You need to setup the Content-Type as required by the API. There should be "Request Content-Type" property where you can set the value. Also any custom header values can be added as Dynamic Property. The invokehttp processor should be very flexible to accommodate the different API requirements and request types.

        Translate
        Terms & ConditionsAccessibility statement