question Re: Datafu on HDP2.3.4.7 is supposed to be 1.3 but is actually 1.2 in Archives of Support Questions (Read Only)

Datafu on HDP2.3.4.7 is supposed to be 1.3 but is actually 1.2

steve2 — Fri, 10 Jun 2016 11:11:35 GMT

I recently installed (technically upgraded) to HDP2.3. Per the HDP2.3 release notes, it comes with datafu 1.3.

I verified datafu with HDP2.3 is installed:

yum list installed | grep datafu

datafu_2_3_4_7_4.noarch 1.3.0.2.3.4.7-4.el6 @HDP-2.3.4.7

However, if you unzip the jar files, it's actually datafu 1.2 - example:

eg, from the doc file it's 1.2: ./bags/package-summary.html:<title>datafu.pig.bags (DataFu 1.2.0)</title>

and (for example) the main jar file is missing datafu/pig/util/SelectStringFieldByName.class which is a new class added in JIRA DATAFU-69.

Is this a known issue?

Re: Datafu on HDP2.3.4.7 is supposed to be 1.3 but is actually 1.2

aervits — Sat, 11 Jun 2016 01:21:28 GMT

@Steve Steyic

first of all awesome catch! Secondly, I checked with engineering, this is a problem with documentation rather than with packaging. HDP 2.3.x suppose to contain DataFu 1.2. 1.3 is coming in next HDP release. You actually got to the bottom of something that I was researching myself but never dug deeper. Again thank you for bringing this to our attention.

Re: Datafu on HDP2.3.4.7 is supposed to be 1.3 but is actually 1.2

aervits — Tue, 26 Jul 2016 22:55:14 GMT

@Steve Steyic

closing the loop on this, I confirmed with engineering as well as on HDP 2.5 sandbox, Datafu is 1.3 in HDP 2.5 and not below.

[root@sandbox datafu]# jar -tf datafu-pig-1.3.0.2.5.0.0-817.jar | grep SelectStringFieldByName
datafu/pig/util/SelectStringFieldByName.class