Support Questions
Find answers, ask questions, and share your expertise

Pig logs consuming disk space.

Explorer

We are running out disk very often as pig logs are huge and do not contain any useful information. we have configured log rotate to temporarily fix the problem. But was interested in knowing more about what is exactly causing this.

12G of pig log has

```Pig Stack Trace --------------- ERROR 2999: Unexpected internal error. null java.lang.NullPointerException ```

I have attached the pig configspig-configs.tar.gz

2 REPLIES 2

Re: Pig logs consuming disk space.

Expert Contributor

@prarthana basgod,

Can you post complete Error stacktrace?

Re: Pig logs consuming disk space.

Mentor

@prarthana basgod

Check your log4j properties in Ambari UI---Pig---Config---Advanced pig-properties like pig.temp.dir should point to a tmp directory

<em>pig.temp.dir=/tmp/ 
debug=INFO </em>

instead of DEBUG which generates a lot of logs.

Hope that helps