Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to put data into hdfs from sqlserver on azure spark hdi

Solved Go to solution
Highlighted

How to put data into hdfs from sqlserver on azure spark hdi

New Contributor

Hi I can read data from sqlserver. Converted it into ORC and now trying to push it into HDFS.

80523-im1.png

My PutHDFS config are below.

I'm just putting data into hdfs to PutHDFS is terminated at success. Can you guys please help me get past the error below

80525-error.png

80524-puthdfs.png

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to put data into hdfs from sqlserver on azure spark hdi

Super Collaborator

please check here for the config of the putHDFS processor to write to Azure: https://community.hortonworks.com/content/kbentry/71916/connecting-to-azure-data-lake-from-a-nifi-da...

4 REPLIES 4

Re: How to put data into hdfs from sqlserver on azure spark hdi

Super Collaborator

please check here for the config of the putHDFS processor to write to Azure: https://community.hortonworks.com/content/kbentry/71916/connecting-to-azure-data-lake-from-a-nifi-da...

Re: How to put data into hdfs from sqlserver on azure spark hdi

New Contributor

@Harald Berghoff I will try this and try to push data into HDFS. I'll update . Thanks

Re: How to put data into hdfs from sqlserver on azure spark hdi

New Contributor

@Harald Berghoff I tried the steps in the article. But I'm getting the same error. I was hoping that this would atleast change the error but it didn't . Any suggestions?

Re: How to put data into hdfs from sqlserver on azure spark hdi

New Contributor

The solution provided by @Harald via link is good enough as a solution. The only prequesite is that the cluster should be on top of Azure Data Lake and not azure datastorage. If its a cluster then solution should be replicated on all nodes. If it's only Edge node then solution should only be applied only edge node. By solution I mean the solution in https://community.hortonworks.com/content/kbentry/71916/connecting-to-azure-data-lake-from-a-nifi-da.... Thanks