Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Salesforce integration with Hortonworks Data Flow (Apache Nifi)

avatar

The requirement is to extract data from Salesforce and to ingest into Hive. Is this a good use case for HDF?

The main requirement is to pull data from Salesforce. What processors are appropriate here?

The invokeHttp processor and/or ExtractText processor?

1 ACCEPTED SOLUTION

avatar
Master Mentor

You could use the invokeHTTP processor to connect to a restAPI to pull data. Once NiFi has the data you could do things like extract parts of the data in to NiFi FlowFIle attributes using the extractText processor.

View solution in original post

8 REPLIES 8

avatar
Master Mentor

@Ancil McBarnett their API guide shows every which possible way. I'd use the rest processors in Nifi for that.

avatar
Master Mentor

You could use the invokeHTTP processor to connect to a restAPI to pull data. Once NiFi has the data you could do things like extract parts of the data in to NiFi FlowFIle attributes using the extractText processor.

avatar
Rising Star

Matt, if you have the detailed document on importing data from Salesforce to Hadoop using nifi, please share.

avatar
New Contributor

avatar
Explorer

The invokeHTTP processor would work best, you would need to follow SalesForce authentication guidelines for the rest api: here.

avatar

Salesforce requires SOAP login to access API, linking in another discussion about WSDL soap.

avatar
Explorer

I came across this thread and my colleague published a more recent option on the DataDirect blog with Salesforce JDBC and Apache Nifi

avatar
New Contributor

Connecting Salesforce with Hortonworks Data Flow (powered by Apache NiFi) unlocks powerful data integration. Here's how:

  • NiFi Processors: Utilize processors like "InvokeHTTP" to call Salesforce APIs and retrieve data.
  • Real-Time or Batch: Move data bi-directionally (Salesforce to NiFi or vice versa) in real-time or batches.
  • Data Transformation: Cleanse, transform, and enrich data using NiFi's processors before storing it in your data lake.

This salesforce integration helps you leverage valuable Salesforce data for analytics, reporting, and deeper customer insights.