Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to list hdfs directory files by groovy code?

Highlighted

How to list hdfs directory files by groovy code?

New Contributor

I want to get real name of file from one previous flowfile and then match how many flowfile with this name exists in certain directory , here is my code but it trows exception which tells that startup failed because unexpected char 0XFFFF, here is my code what should i change?

  1. how should i connect certain hadoop directory? and list it's file
def flowFile=session.get();
def flowFile1=session.create();
String  filename=flowFile.getAttribute('realName').toString();
def  count1 =flowFile.getAttribute('count');\
count1=coun1t as  Number;
 Configuration conf = getConf();
    Job job = Job.getInstance(conf);
    FileSystem fs = FileSystem.get(conf);
    RemoteIterator<LocatedFileStatus> fileStatusListIterator = fs.listFiles(
            new Path("master2.sda.gov.ge"), true);
int count = fs.listFiles(new Path("here  is  some   url"), true)
                                  .findAll { it.name.matches(filename.*/) }
                                  .size();


if(count1== count){
flowFile1=session.putAttribute(flowFile1,"check","AllRIGHT");
}                             
flowFile1=session.putAttribute(flowFile1,"amount",count);
flowFile1=session.putAttribute(flowFile1,"filename",filename);
session.transfer(flowFile1,REL_SUCCESS);
session.remove(flowFile)
Don't have an account?
Coming from Hortonworks? Activate your account here