Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Pig logs consuming disk space.

Pig logs consuming disk space.

Explorer

We are running out disk very often as pig logs are huge and do not contain any useful information. we have configured log rotate to temporarily fix the problem. But was interested in knowing more about what is exactly causing this.

12G of pig log has

```Pig Stack Trace --------------- ERROR 2999: Unexpected internal error. null java.lang.NullPointerException ```

I have attached the pig configspig-configs.tar.gz

2 REPLIES 2
Highlighted

Re: Pig logs consuming disk space.

Expert Contributor

@prarthana basgod,

Can you post complete Error stacktrace?

Highlighted

Re: Pig logs consuming disk space.

Mentor

@prarthana basgod

Check your log4j properties in Ambari UI---Pig---Config---Advanced pig-properties like pig.temp.dir should point to a tmp directory

<em>pig.temp.dir=/tmp/ 
debug=INFO </em>

instead of DEBUG which generates a lot of logs.

Hope that helps

Don't have an account?
Coming from Hortonworks? Activate your account here