Datafu on HDP2.3.4.7 is supposed to be 1.3 but is actually 1.2

[#1]

I recently installed (technically upgraded) to HDP2.3. Per the HDP2.3 release notes, it comes with datafu 1.3.

I verified datafu with HDP2.3 is installed:

yum list installed | grep datafu

datafu_2_3_4_7_4.noarch 1.3.0.2.3.4.7-4.el6 @HDP-2.3.4.7

However, if you unzip the jar files, it's actually datafu 1.2 - example:

eg, from the doc file it's 1.2: ./bags/package-summary.html:<title>datafu.pig.bags (DataFu 1.2.0)</title>

and (for example) the main jar file is missing datafu/pig/util/SelectStringFieldByName.class which is a new class added in JIRA DATAFU-69.

Is this a known issue?

[#2 · Accepted solution]

@Steve Steyic

first of all awesome catch! Secondly, I checked with engineering, this is a problem with documentation rather than with packaging. HDP 2.3.x suppose to contain DataFu 1.2. 1.3 is coming in next HDP release. You actually got to the bottom of something that I was researching myself but never dug deeper. Again thank you for bringing this to our attention.

[#3]

@Steve Steyic

closing the loop on this, I confirmed with engineering as well as on HDP 2.5 sandbox, Datafu is 1.3 in HDP 2.5 and not below.

[root@sandbox datafu]# jar -tf datafu-pig-1.3.0.2.5.0.0-817.jar | grep SelectStringFieldByName
datafu/pig/util/SelectStringFieldByName.class