Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to clear user cache for running pig job

Highlighted

how to clear user cache for running pig job

Expert Contributor

Hi,

I have cluster of 4 nodes, each having 16 GB RAM and 2 core, 50 GB disk storage on master and 25 GB on each worker node. one is master node and 3 are worker nodes. i have stored 16 GB data on HDFS.

I am trying to process this 16 GB data using PIG in map reduce mode.

When i submit a pig job, the application master is assigned on one worker node from three, job starts its execution and gets stuck at 94%. After that i came to know that the user(application) cache(7.9 GB) is full on another 2 nodes.

Is there any configuration to automatically clear the cache for running job ?


Thanks,

Heta

Don't have an account?
Coming from Hortonworks? Activate your account here