Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Which version of CDH supports spork (pig-on-spark)?

Which version of CDH supports spork (pig-on-spark)?

New Contributor

What version of CDH supports pig on spark?  I have CDHv5.4.3 installed (via Cloudera Manager) and I am receiving the following error when I do a "pig -x spark" from the linux command prompt:

- ERROR 2040: Unknown exec type: spark

 

Here are the versions of pig and spark installed with CDH5.4.3:

- pig-0.12.0+cdh5.4.3+58-1.cdh5.4.3.p0.9.el6.noarch

- spark-core-1.3.0+cdh5.4.3+40-1.cdh5.4.3.p0.9.el6.noarch

 

Am I missing something?  Or is it just the version of pig does not support spark?

 

If it is me -- What should I check?

If it is the version of pig --  Which version supports spark?

 

Thanks.

 

If it makes a difference -- I am using CentOS v6.5 as my OS and my test cluster is made up of six VMs - two name nodes, three data nodes, and one "everything else" node.

 

Signed,

  Shawn Scoles

 

1 REPLY 1
Highlighted

Re: Which version of CDH supports spork (pig-on-spark)?

Rising Star

It seems that historically the spork project never reached a level of maturity where Cloudera decided to support it.

 

It also does not appear to be updated in recent years, so it is unlikely to be usefull anytime soon.