Reply
New Contributor
Posts: 5
Registered: ‎06-23-2015

Which version of CDH supports spork (pig-on-spark)?

What version of CDH supports pig on spark?  I have CDHv5.4.3 installed (via Cloudera Manager) and I am receiving the following error when I do a "pig -x spark" from the linux command prompt:

- ERROR 2040: Unknown exec type: spark

 

Here are the versions of pig and spark installed with CDH5.4.3:

- pig-0.12.0+cdh5.4.3+58-1.cdh5.4.3.p0.9.el6.noarch

- spark-core-1.3.0+cdh5.4.3+40-1.cdh5.4.3.p0.9.el6.noarch

 

Am I missing something?  Or is it just the version of pig does not support spark?

 

If it is me -- What should I check?

If it is the version of pig --  Which version supports spark?

 

Thanks.

 

If it makes a difference -- I am using CentOS v6.5 as my OS and my test cluster is made up of six VMs - two name nodes, three data nodes, and one "everything else" node.

 

Signed,

  Shawn Scoles

 

Highlighted
Cloudera Employee
Posts: 39
Registered: ‎01-07-2019

Re: Which version of CDH supports spork (pig-on-spark)?

It seems that historically the spork project never reached a level of maturity where Cloudera decided to support it.

 

It also does not appear to be updated in recent years, so it is unlikely to be usefull anytime soon.

Announcements
New solutions