Support Questions

Find answers, ask questions, and share your expertise

How to put data into hdfs from sqlserver on azure spark hdi

avatar
Contributor

Hi I can read data from sqlserver. Converted it into ORC and now trying to push it into HDFS.

80523-im1.png

My PutHDFS config are below.

I'm just putting data into hdfs to PutHDFS is terminated at success. Can you guys please help me get past the error below

80525-error.png

80524-puthdfs.png

1 ACCEPTED SOLUTION

avatar
Super Collaborator
4 REPLIES 4

avatar
Super Collaborator

please check here for the config of the putHDFS processor to write to Azure: https://community.hortonworks.com/content/kbentry/71916/connecting-to-azure-data-lake-from-a-nifi-da...

avatar
Contributor

@Harald Berghoff I will try this and try to push data into HDFS. I'll update . Thanks

avatar
Contributor

@Harald Berghoff I tried the steps in the article. But I'm getting the same error. I was hoping that this would atleast change the error but it didn't . Any suggestions?

avatar
Contributor

The solution provided by @Harald via link is good enough as a solution. The only prequesite is that the cluster should be on top of Azure Data Lake and not azure datastorage. If its a cluster then solution should be replicated on all nodes. If it's only Edge node then solution should only be applied only edge node. By solution I mean the solution in https://community.hortonworks.com/content/kbentry/71916/connecting-to-azure-data-lake-from-a-nifi-da.... Thanks