Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Pass Pig relation value inside curl command

Pass Pig relation value inside curl command

Explorer

Hi,

I want to push only log message to external service using Pig script .I wrote a pig script which fetch log messages which is stored in Pig relation(errorLogs) . now i want to pass Pig relation(errorLogs) value inside curl command.

I tried using below script :

REGISTER /usr/hdp/2.6.1.4-2/pig/lib/piggybank.jar;
data = LOAD '/logFiles/test.log' USING org.apache.pig.piggybank.storage.MyRegExLoader('^(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(.*)$') AS (datefield:chararray,timefield:chararray,loglevel:chararray,logmessage:chararray);
logs = FOREACH data GENERATE $0 as datefield,$1 as timefield,$2 as loglevel,$3 as logmessage;
errorLogs = FILTER logs BY loglevel == 'ERROR';
g = group errorLogs all;
%declare LOG `curl -X POST http://xxx.xx.xxx.xx//api/create -H "accept: application/json" -H "authorization:authorization-token-value" -H "cache-control: no-cache" -H "content-type: multipart/form-data"  -F "title=$errorLogs.logmessage" -F "description=TEST descirption"`
result = foreach g GENERATE '$LOG';

But it throws below error while executing %declare statement :

ERROR org.apache.pig.impl.PigContext - Undefined parameter rorLogs

ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2997: Encound IOException. org.apache.pig.tools.parameters.ParameterSubstitutionException: Undefined peter : errorLogs

I am getting issue while passing Pig relation value inside curl command. How to pass Pig relation value inside curl command?

Don't have an account?
Coming from Hortonworks? Activate your account here