Salesforce integration with Hortonworks Data Flow (Apache Nifi)

[#1]

The requirement is to extract data from Salesforce and to ingest into Hive. Is this a good use case for HDF?

The main requirement is to pull data from Salesforce. What processors are appropriate here?

The invokeHttp processor and/or ExtractText processor?

[#2]

@Ancil McBarnett their API guide shows every which possible way. I'd use the rest processors in Nifi for that.

[#3 · Accepted solution]

You could use the invokeHTTP processor to connect to a restAPI to pull data. Once NiFi has the data you could do things like extract parts of the data in to NiFi FlowFIle attributes using the extractText processor.

[#4]

Matt, if you have the detailed document on importing data from Salesforce to Hadoop using nifi, please share.

[#5]

Please refer: https://community.hortonworks.com/idea/203714/apache-nifi-fetching-event-log-files-from-salesfor.htm...

[#6]

The invokeHTTP processor would work best, you would need to follow SalesForce authentication guidelines for the rest api: here.

[#7]

Salesforce requires SOAP login to access API, linking in another discussion about WSDL soap.

[#8]

I came across this thread and my colleague published a more recent option on the DataDirect blog with Salesforce JDBC and Apache Nifi

[#9]

Connecting Salesforce with Hortonworks Data Flow (powered by Apache NiFi) unlocks powerful data integration. Here's how:

• NiFi Processors: Utilize processors like "InvokeHTTP" to call Salesforce APIs and retrieve data.
• Real-Time or Batch: Move data bi-directionally (Salesforce to NiFi or vice versa) in real-time or batches.
• Data Transformation: Cleanse, transform, and enrich data using NiFi's processors before storing it in your data lake.

This salesforce integration helps you leverage valuable Salesforce data for analytics, reporting, and deeper customer insights.