Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

​-rm: Fatal internal error java.lang.NullPointerException

Highlighted

​-rm: Fatal internal error java.lang.NullPointerException

Rising Star

i am trying to remove the hive external data in blob storage using below command and data got delete but i am getting error what is is error mean..can you help me to understand the error and soulution.

hdfs dfs -rm -R wasbs://data@datadev.blob.core.windows.net/backup_dwh/

-rm: Fatal internal error java.lang.NullPointerException at org.apache.hadoop.fs.azure.NativeAzureFileSystem$FolderRenamePending.execute(NativeAzureFileSystem.java:448) at org.apache.hadoop.fs.azure.NativeAzureFileSystem.rename(NativeAzureFileSystem.java:2707) at org.apache.hadoop.fs.FileSystem.rename(FileSystem.java:1340) at org.apache.hadoop.fs.TrashPolicyDefault.moveToTrash(TrashPolicyDefault.java:166) at org.apache.hadoop.fs.Trash.moveToTrash(Trash.java:109) at org.apache.hadoop.fs.Trash.moveToAppropriateTrash(Trash.java:95) at org.apache.hadoop.fs.shell.Delete$Rm.moveToTrash(Delete.java:153) at org.apache.hadoop.fs.shell.Delete$Rm.processPath(Delete.java:118) at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:317) at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:289) at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:271) at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:255) at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:119) at org.apache.hadoop.fs.shell.Command.run(Command.java:165) at org.apache.hadoop.fs.FsShell.run(FsShell.java:297) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90) at org.apache.hadoop.fs.FsShell.main(FsShell.java:356)

2 REPLIES 2

Re: ​-rm: Fatal internal error java.lang.NullPointerException

Hello @rama!
I'm not used to azure, but, could you enable the DEBUG for your logs? Will give us a clear vision of what's going on.
Looking at the code, seems you're hitting this line:

https://github.com/apache/hadoop/blob/2d2d97fa7d4224369b3c13bc4a45e8cc9e29afb1/hadoop-tools/hadoop-a...

As you can see, the code has a plenty of log.debug, this can facilitate our trace to find the issue :)

Also, take a look at this info below:

Link -> https://hadoop.apache.org/docs/current/hadoop-azure/index.html

Multithread Support

Rename and Delete blob operations on directories with large number of files and sub directories currently is very slow as these operations are done one blob at a time serially. These files and sub folders can be deleted or renamed parallel. Following configurations can be used to enable threads to do parallel processing

To enable 10 threads for Delete operation. Set configuration value to 0 or 1 to disable threads. The default behavior is threads disabled.

<property>
  <name>fs.azure.delete.threads</name>
  <value>10</value>
</property>

To enable 20 threads for Rename operation. Set configuration value to 0 or 1 to disable threads. The default behavior is threads disabled.

<property>
  <name>fs.azure.rename.threads</name>
  <value>20</value>
</property>

Hope this helps!

Re: ​-rm: Fatal internal error java.lang.NullPointerException

New Contributor

@rama, thanks for reporting the issue. It is a bug since FileSystem client should show NullPointerException to users. Would you like to file a JIRA to Apache ( https://issues.apache.org/jira/projects/HDFS )? I am happy to help.

Don't have an account?
Coming from Hortonworks? Activate your account here