Reply
Highlighted
New Contributor
Posts: 1
Registered: ‎11-09-2017

pig UDF help

Hi All 

 

I am new to Java and hence new to converting the code to be used by pig (pig UDF).

 

Use Case:

I have a file consisting of different lines. Each line is a continuous string of text and packeddecimals .We got this data from Mainframe and had some luck with syncsort.

 

So the expectation from below code is to read the line and convert packed decimal (Comp-3) to String within it.

so UDF(line,0,110) should give us the a readable string line.

 

PigUDF Code: I am having problems with the byte,tuple thing below and also get error  Error I am getting: [Caught exception processing input row ]

 

import java.io.IOException;

import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.util.WrappedIOException;

public class MainframepackedconUDF extends EvalFunc<String>
{

public String exec(Tuple input) throws IOException { 
if (input == null || input.size() == 0)
return null;

try{
String str = input.get(0).toString(); // get the three arguments passed in the Tuple
byte[] b = str.getBytes();
Integer start = (Integer)input.get(1);
Integer fin = (Integer)input.get(2);

return getMainframePackedDecimal(b, start, fin);
// return null;

}catch(Exception e){
throw new IOException("Caught exception processing input row ", e);
}
}

public static StringBuilder getDecimalSB(final byte[] record, final int start, final int fin) throws IOException
{
int i;
String s;
StringBuilder ret = new StringBuilder(3 + Math.max(0, 2 *(fin - start)));
int b;

try{
for (i = start; i < fin; i++) {
b = ((int) 255) & record[i];
//toPostiveByte(record[i]);
s = Integer.toHexString(b);
if (s.length() == 1) {
ret.append('0');
}
ret.append(s);
}
}catch(Exception e){
throw new IOException("Caught exception looping through bytes", e);
}

return ret;
}

public static String getMainframePackedDecimal(final byte[] record, final int start,final int fin) throws IOException {

try {
getDecimalSB(record, start, fin);
}catch(Exception e){
throw new IOException("Caught exception calling getDecimalSB", e);
}

String ret = "0";

try {
StringBuilder hex = getDecimalSB(record, start, fin);

if (hex.length() > 0) {
switch (hex.charAt(hex.length() - 1)) {
case 'D':
case 'd' : hex.insert(0, '-');
case 'A' :
case 'B' :
case 'C' :
case 'E' :
case 'F' :
case 'a' :
case 'b' :
case 'c' :
case 'e' :
case 'f' :
hex.setLength(hex.length() - 1);
break;
default:
}
ret = hex.toString();
}


// if (ret.length() == 0) {
// ret = "0";
// }

return ret;
}catch(Exception e){
throw new IOException("Caught exception unpacking", e);
}
}
}

 

Really appreciate your help with this.

 

Thanks!

 

 

Announcements