HDFS-Sharepoint

[#1]

Hello

I want to ingest data from a sharepoint repository, containing documents, into HDFS. What is the best way to do this? Nifi doesn't have a connector to a sharepoint repository. Has anyone faced a case like this?

Thanks

[#2 · Accepted solution]

@Ahmad Debbas

Setup HDFS NFS Gateway and copy Sharepoint files. You could also use basic script to PUT the files to HDFS. It would require to use an edge node that has access to SharePoint repository and HDFS client.

HDFS NFS Gateway: https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html

HDFS PUT: https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/FileSystemShell.html#put

If you already use HDP and it is installed with Ambari, HDFS NFS Gateway is just another service to add via Ambari.

If the response was helpful, please vote/accept best answer.

[#3]

@Ahmad Debbas Hi Ahmad I have a similar scenario where there is a need ingest data from Sharepoint into HDFS. How were you able to implement this. Could you please share a snapshot of your nifi dataflow.