Connecting Salesforce with Hortonworks Data Flow (powered by Apache NiFi) unlocks powerful data integration. Here's how: NiFi Processors: Utilize processors like "InvokeHTTP" to call Salesforce APIs and retrieve data. Real-Time or Batch: Move data bi-directionally (Salesforce to NiFi or vice versa) in real-time or batches. Data Transformation: Cleanse, transform, and enrich data using NiFi's processors before storing it in your data lake. This salesforce integration helps you leverage valuable Salesforce data for analytics, reporting, and deeper customer insights.  ... View more

@sri_man  Since this thread was marked 'Solved' back in 2016, you would have a better chance of receiving a relevant response by posting a new question. This will also provide the opportunity to provide details specific to your environment that could aid other members in providing a more tailored answer to your issue.      ... View more

using PUT command, need to submit the curl twice. There is "negotiate" curl command which does the same in single submission. curl --negotiate -u : -L "http://namenode:50070/webhdfs/v1/user/username/余宗阳视频审核稿-1024.docx?op=CREATE&user.name=username" -T 余宗阳视频审核稿-1024.docx ... View more

> I have millions of records in each table and hundreds of tables, so first option might not be optimal for big tables. > will try out the other options thank you ... View more

@Gayathri Reddy G The issue could be related to the mapreduce running outofmemory due to higher fetch size. Use the sqoop command with mapreduce parameters as: sqoop -import -Dmapreduce.map.memory.mb=8192 -Dmapreduce.map.java.opts=-Xmx7200m .. ... View more

Yes, that worked. after giving SELECT_CATALOG_ROLE privileges, direct option is working. ... View more

This article will help you to implement kerberos and add kerberos principal: https://sqoop.apache.org/docs/1.99.7/security/AuthenticationAndAuthorization.html ... View more

This is a problem from the hive move task (which has since been fixed in HIVE-15355) which is called by Sqoop after the import into HDFS. So, disabling move task parallelism is the right solution by adding the configuration parameter hive.mv.files.thread=0. That said, I would suggest using --hcatalog-table option with import which allows for 1. better data fidelity 2. remove one intermediate step of landing on HDFS and then invoking the hive client to do the import ... View more

@njayakumar passed the Generic arguments first to the sqoop job, now its working fine. Thanks ... View more