Reply
New Contributor
Posts: 2
Registered: ‎12-13-2018

when a file name contains [ the job submitted to hadoop throws an error in java

Following is the exception I am facing.Could someone please help me out with this?

java.io.IOException: Illegal file pattern: Unclosed character class near index 11
T_[test.txt
at org.apache.hadoop.fs.GlobFilter.init(GlobFilter.java:71)
at org.apache.hadoop.fs.GlobFilter.<init>(GlobFilter.java:50)
at org.apache.hadoop.fs.Globber.glob(Globber.java:192)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1674)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:294)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)

 

 

 

 

 

 

If I include both [] brackets I get the following exception:-

 

 

org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input Pattern file:/home/excel/files/T_[test].txt matches 0 files
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at com.dataguise.hadoop.util.WorkFlowNew$4.run(WorkFlowNew.java:646)
at com.dataguise.hadoop.util.WorkFlowNew$4.run(WorkFlowNew.java:617)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

Posts: 57
Topics: 0
Kudos: 8
Solutions: 5
Registered: ‎05-15-2018

Re: when a file name contains [ the job submitted to hadoop throws an error in java

Hello @heena

 

From the message below, I see your job is treating as a pattern for Regular expression

 

~~

java.io.IOException: Illegal file pattern: Unclosed character class near index 11
T_[test.txt

~~

 

Have you tried running the job with different input file name  ?

 

Also, please refer the below JIRA which explains the File naming conventions and issues

 

[ https://issues.apache.org/jira/browse/HDFS-240

Thanks,
Satz
New Contributor
Posts: 2
Registered: ‎12-13-2018

Re: when a file name contains [ the job submitted to hadoop throws an error in java

Hi

Thanks for the reply.

Yes I have changed the file name and it works fine .But I want to run the job using the [ in file name.Can you suggest some way for that

Highlighted
Posts: 57
Topics: 0
Kudos: 8
Solutions: 5
Registered: ‎05-15-2018

Re: when a file name contains [ the job submitted to hadoop throws an error in java

Hello @heena

 

You can try them by using escape characters "\" in front of them:

 

Example

 

"[" can be written as "\["

Thanks,
Satz
Announcements