Upgrade 2.3 to 2.4, cannot install packages

[#1]

I'm trying to upgrade my stack to HDP2.4 (from 2.3).

but my install is failing on 2 machines, one with this error:

2016-09-14 14:31:47,286 - Package Manager failed to install packages. Error: Execution of '/usr/bin/apt-get -q -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install libfuse2' returned 100. Reading package lists...
Building dependency tree...
Reading state information...
libfuse2 is already the newest version.
You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies:
 mahout-doc : Depends: mahout-2-4-2-0-258-doc but it is not going to be installed
E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).

and another with this:

dpkg: error processing package zeppelin-2-4-2-0-258 (--configure):
 subprocess installed post-installation script returned error exit status 2
dpkg: dependency problems prevent configuration of zeppelin:
 zeppelin depends on zeppelin-2-4-2-0-258; however:
  Package zeppelin-2-4-2-0-258 is not configured yet.

I've tried installing mahout, mahout-doc and mahout-2-4-2-0-258-doc but they all error out.

Same for the Zeppelin machine.

I've tried apt-get -f install, apt-get purge, apt-get update apt-get autoremove etc.

I did have PPAs repeated but I got rid of those.

I'm at my wit's end with this, so I am very grateful for any help anyone can give. TIA!

Update: Vkumar fixed the zeppelin one. After applying this to mahout and doing apt-get -f install i get:

ed@s1:~$ sudo apt-get -f install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following extra packages will be installed:
  mahout-2-4-2-0-258-doc
The following NEW packages will be installed
  mahout-2-4-2-0-258-doc
0 to upgrade, 1 to newly install, 0 to remove and 209 not to upgrade.
3 not fully installed or removed.
Need to get 0 B/638 kB of archives.
After this operation, 5,530 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
(Reading database ... 250308 files and directories currently installed.)
Preparing to unpack .../mahout-2-4-2-0-258-doc_0.9.0.2.4.2.0-258_all.deb ...
Unpacking mahout-2-4-2-0-258-doc (0.9.0.2.4.2.0-258) ...
dpkg: error processing archive /var/cache/apt/archives/mahout-2-4-2-0-258-doc_0.9.0.2.4.2.0-258_all.deb (--unpack):
 trying to overwrite '/usr/hdp/2.4.2.0-258/mahout/doc/NOTICE.txt', which is also in package mahout-2-4-2-0-258 0.9.0.2.4.2.0-258
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Errors were encountered while processing:
 /var/cache/apt/archives/mahout-2-4-2-0-258-doc_0.9.0.2.4.2.0-258_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

[#2]

We have observed this as a BUG in 2.4.2.0-258 release and is on the line to be fixed in upcoming release above 2.4.4.x.

I have tried as an workaround to create the missing directory on the complaining node as below:

$ mkdir -p /etc/zeppelin/conf.dist

You need to have sudo or root to create the directory said above.

After creating the directory please run:

$ apt-get -f install

I have tried to install mahout as well on a node with reported issue after creating the directory as said above and mahout installation went good.

My test environment is ubuntu14.

Please let us know if this solves the issue.

[#3]

I did that and also added the directories: /var/lib/zeppelin and /var/run/zeppelin and it fixed the zeppelin one: thank you so much!

But the mahout error is still there (I did the above on the mahout machine. When I try apt-get install zeppelin* it gives the mahout error..

[#4]

I've updated the qn to show the full mahout error.

[#5]

Trying to reproduce the issue in my environment. I would post an update right away after my verification.

[#6 · Accepted solution]

To fix the mahout issue do the following steps on the node which is complaining

1. remove mahout-doc

$ sudo apt-get remove mahout-doc

2. run

$ sudo apt-get -f install

3. see if mahout installation is perfect

$ apt-get install mahout

Please do not install mahout-doc

After running $sudo apt-get install mahout

you would see following packages installed:

$ dpkg --get-selections| grep mahout

mahout install

mahout-2-4-2-0-258 install

Please remove all mahout except above two using $ apt-get remove <name suggested by --get-selections>. Keep only two as said above.

[#7]

That did it. Thank you so much!