Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How do i stream data from a file to hdfs using spring XD

Highlighted

How do i stream data from a file to hdfs using spring XD

Explorer

 I am trying to stream my data from a file to hdfs by using spring XD.

 

stream create --name fieltest1 --definition "file --dir=/home/hdfs/spring-xd-1.3.2.RELEASE/tmp/xd/input/filetest --outputType=text/plain | hdfs" --deploy

 

 

but I am getting below error. can some one suggest how can I achieve this?

 

org.springframework.messaging.MessageHandlingException: failed to write Message payload to HDFS; nested exception is java.io.IOException: failure to login

                at org.springframework.xd.integration.hadoop.outbound.HdfsStoreMessageHandler.handleMessageInternal(HdfsStoreMessageHandler.java:129)

1 REPLY 1

Re: How do i stream data from a file to hdfs using spring XD

Explorer

I got the solution...we need to provide fsUri after hdfs to acces hdfs location