Zeta Alliance – Zimbra 8.7 Update Warning

Every week the members of the Zeta Alliance have an online meeting to talk about Zimbra Open Source initiatives, zimlet development etc. (Go to their web site to join our community!)

On this week’s call we learned something new and surprising about Zimbra 8.7 updates.

 

Zimbra Now Uses Repositories

As you know, the first time you run the 8.7 installer it adds packages repositories to your operating system’s list of what usually just comprise operating system update repositories.  (Like Ubuntu, which periodically respins the ISOs for LTS releases (14.04.2 etc.), Zimbra will do the same and you can expect an 8.7.1 installer release in the near future — once QA is done with it.)

So, now with Zimbra 8.7 when you run “apt-get update; apt-get dist-upgrade” you not only get updates for your operating system but also updates for much of Zimbra at the same time.  This is pretty cool and a great step forward in being able to release incremental bug fixes and security patches fast, without having to ship a whole new hundreds-of-megabytes Zimbra installer.

Indeed, Zimbra have already shipped using this method the fix for the show stopper LDAP MMR bug.  Nicely done Zimbra!

 

Zimbra’s Update Packages Need Some Tuning Though…

Seasoned Linux admins are already aware that when you update packages for things like the SSH server, BIND9 and Linux-distro-supplied Apache and MySQL, the update process itself gracefully restarts or reloads the upgraded services and performs other housekeeping as needed.

Unfortunately, Zimbra’s packages aren’t quite there yet.

Indeed, yesterday as a result of the Zeta Alliance call we learned that Zimbra’s best practice now is to have all of Zimbra stopped (except LDAP and zmconfigd) prior to running “apt-get update; apt-get dist-upgrade”.  Apparently, there has been a report of a Java keystore corruption when Zimbra updates were applied with Zimbra running. (If anyone can find the specific bug number please leave a comment and I’ll update this blog post accordingly.)

There is a Zimbra Certified wiki page that documents the supported update process; the page at this writing is marked “Work In Progress”, it has a few typos and missing bits (no mention of stopping cbpolicyd for example), so we suggest checking back there right before you plan to do an upgrade, in case anything has changed.  It’s the only 8.7 upgrade reference that exists, and it’s existence was not broadly known to most of the gurus on the Zeta Alliance call.

At this writing, the key takeaway from that certified wiki page is that ldap and zmconfigd need to be running during package upgrades, and nothing else.  The wiki proposes stopping each Zimbra service individually, but when we tested we found no commands were given to stop snmp and cbpolicyd.  We found a quicker way to apply the updates and shorten the service down time apparent to end users was instead to run as the zimbra user:

zmcontrol stop
ldap start
zmconfigdctl start

You can then become root and run “apt-get update; apt-get dist-upgrade”, and afterwards restart Zimbra with a “zmcontrol restart” to finish.

One participant suggested being clever and commenting out the Zimbra repos from /etc/apt/sources.list unless and until it was confirmed that Zimbra had released some updates. That would enable you to continue to update the operating system on demand without a service interruption to Zimbra and without risking corruption to Zimbra.  When Zimbra did release some updates, at that point one could uncomment the Zimbra repos, shut Zimbra down to do the update, and then fire Zimbra back up safely.

 

Conclusions

As with anything radically new, we always expect some teething issues. Sometimes those teething issues can be serious like this, so we continue to advise caution in deploying 8.7 — which we continue to recommend only for new smaller installs. Existing Zimbra installations we recommend should be upgraded to 8.6.0 Patch 7, at least until these kinds of issues and the remaining upgrade issues are sorted out.

If you’d like to talk more about Zimbra, explore our Barracuda-backed HIPAA-compliant email offering or just chat, feel free to give us a call at (207) 772-5678.

Take care,
L. Mark Stone
General Manager, Managed and Private/Hybrid Cloud Services
Reliable Networks
A Division of OTT Communications

The information provided in this blog is intended for informational and educational purposes only. The views expressed herein are those of Mr. Stone and do not necessarily reflect those of Reliable Networks, OTT Communications or Otelco Inc. The contents of this site are not intended as advice for any purpose and are subject to change without notice. We make no warranties of any kind regarding the accuracy or completeness of any information on this site, and we make no representations regarding whether such information is up-to-date or applicable to any particular situation. All copyrights are reserved by Mr. Stone. Any portion of the material on this site may be used for personal or educational purposes provided appropriate attribution is given to Mr. Stone and this blog.

P.S.  Here’s what today’s upgrade on a test system looked like:

zimbra@mail:~/bin$ zmcontrol stop
Host mail.testdomain.com
 Stopping zmconfigd...Done.
 Stopping zimlet webapp...Done.
 Stopping zimbraAdmin webapp...Done.
 Stopping zimbra webapp...Done.
 Stopping service webapp...Done.
 Stopping stats...Done.
 Stopping mta...Done.
 Stopping spell...Done.
 Stopping snmp...Done.
 Stopping cbpolicyd...Done.
 Stopping archiving...Done.
 Stopping opendkim...Done.
 Stopping amavis...Done.
 Stopping antivirus...Done.
 Stopping antispam...Done.
 Stopping proxy...Done.
 Stopping memcached...Done.
 Stopping mailbox...Done.
 Stopping logger...Done.
 Stopping dnscache...Done.
 Stopping ldap...Done.
zimbra@mail:~/bin$ ldap start
Started slapd: pid 3229
zimbra@mail:~/bin$ zmconfigdctl start
Starting zmconfigd...done.
zimbra@mail:~/bin$ zmcontrol status
Host mail.testdomain.com
 amavis Stopped
 amavisd is not running.
 antispam Stopped
 zmamavisdctl is not running
 antivirus Stopped
 zmamavisdctl is not running
 zmclamdctl is not running
 zmfreshclamctl is not running
 cbpolicyd Stopped
 policyd is not running.
 ldap Running
 logger Stopped
 zmlogswatchctl is not running
 mailbox Stopped
 mysql.server is not running.
 zmmailboxdctl is not running.
 memcached Stopped
 memcached is not running.
 mta Stopped
 zmsaslauthdctl is not running
 postfix is not running
 opendkim Stopped
 zmopendkimctl is not running.
 proxy Stopped
 proxy is not running.
 service webapp Stopped
 mysql.server is not running.
 zmmailboxdctl is not running.
 snmp Stopped
 zmswatch is not running.
 stats Stopped
 zimbra webapp Stopped
 mysql.server is not running.
 zmmailboxdctl is not running.
 zimbraAdmin webapp Stopped
 mysql.server is not running.
 zmmailboxdctl is not running.
 zimlet webapp Stopped
 mysql.server is not running.
 zmmailboxdctl is not running.
 zmconfigd Running
zimbra@mail:~/bin$ exit
logout
lmstone@mail:~$ sudo su - 
root@mail:~# apt-get update; apt-get dist-upgrade
Hit http://apt-longview.linode.com precise Release.gpg
Hit http://apt-longview.linode.com precise Release 
Hit http://security.ubuntu.com precise-security Release.gpg 
Hit http://us.archive.ubuntu.com precise Release.gpg
Get:1 http://us.archive.ubuntu.com precise-updates Release.gpg [198 B]
Hit http://us.archive.ubuntu.com precise-backports Release.gpg 
Hit http://apt-longview.linode.com precise/main amd64 Packages
Hit http://security.ubuntu.com precise-security Release 
Hit http://us.archive.ubuntu.com precise Release 
Get:2 http://us.archive.ubuntu.com precise-updates Release [55.4 kB] 
Hit http://apt-longview.linode.com precise/main i386 Packages 
Ign http://apt-longview.linode.com precise/main TranslationIndex 
Hit http://security.ubuntu.com precise-security/main Sources 
Hit http://us.archive.ubuntu.com precise-backports Release 
Hit http://security.ubuntu.com precise-security/restricted Sources 
Hit http://security.ubuntu.com precise-security/universe Sources 
Hit http://security.ubuntu.com precise-security/multiverse Sources 
Hit http://security.ubuntu.com precise-security/main amd64 Packages 
Hit http://security.ubuntu.com precise-security/restricted amd64 Packages 
Hit http://security.ubuntu.com precise-security/universe amd64 Packages 
Hit http://security.ubuntu.com precise-security/multiverse amd64 Packages 
Hit http://security.ubuntu.com precise-security/main i386 Packages 
Hit http://security.ubuntu.com precise-security/restricted i386 Packages 
Hit http://security.ubuntu.com precise-security/universe i386 Packages 
Hit http://us.archive.ubuntu.com precise/main Sources 
Hit http://us.archive.ubuntu.com precise/restricted Sources 
Hit http://us.archive.ubuntu.com precise/universe Sources 
Hit http://us.archive.ubuntu.com precise/multiverse Sources 
Hit http://us.archive.ubuntu.com precise/main amd64 Packages 
Hit http://us.archive.ubuntu.com precise/restricted amd64 Packages 
Hit http://us.archive.ubuntu.com precise/universe amd64 Packages 
Hit http://us.archive.ubuntu.com precise/multiverse amd64 Packages 
Hit http://us.archive.ubuntu.com precise/main i386 Packages 
Hit http://us.archive.ubuntu.com precise/restricted i386 Packages 
Hit http://security.ubuntu.com precise-security/multiverse i386 Packages 
Hit http://security.ubuntu.com precise-security/main TranslationIndex
Hit http://security.ubuntu.com precise-security/multiverse TranslationIndex
Hit http://security.ubuntu.com precise-security/restricted TranslationIndex
Hit http://security.ubuntu.com precise-security/universe TranslationIndex
Hit http://us.archive.ubuntu.com precise/universe i386 Packages 
Hit http://us.archive.ubuntu.com precise/multiverse i386 Packages 
Hit http://us.archive.ubuntu.com precise/main TranslationIndex 
Hit http://us.archive.ubuntu.com precise/multiverse TranslationIndex
Hit http://us.archive.ubuntu.com precise/restricted TranslationIndex
Hit http://us.archive.ubuntu.com precise/universe TranslationIndex 
Get:3 http://us.archive.ubuntu.com precise-updates/main Sources [499 kB] 
Hit http://security.ubuntu.com precise-security/main Translation-en 
Hit http://security.ubuntu.com precise-security/multiverse Translation-en 
Hit http://security.ubuntu.com precise-security/restricted Translation-en 
Hit http://security.ubuntu.com precise-security/universe Translation-en 
Get:4 http://us.archive.ubuntu.com precise-updates/restricted Sources [8,708 B] 
Get:5 http://us.archive.ubuntu.com precise-updates/universe Sources [132 kB]
Get:6 http://us.archive.ubuntu.com precise-updates/multiverse Sources [10.5 kB]
Get:7 http://us.archive.ubuntu.com precise-updates/main amd64 Packages [1,022 kB]
Ign http://apt-longview.linode.com precise/main Translation-en_US 
Ign http://apt-longview.linode.com precise/main Translation-en 
Get:8 http://us.archive.ubuntu.com precise-updates/restricted amd64 Packages [16.2 kB] 
Get:9 http://us.archive.ubuntu.com precise-updates/universe amd64 Packages [281 kB]
Get:10 http://us.archive.ubuntu.com precise-updates/multiverse amd64 Packages [17.1 kB]
Get:11 http://us.archive.ubuntu.com precise-updates/main i386 Packages [1,088 kB]
Get:12 http://us.archive.ubuntu.com precise-updates/restricted i386 Packages [16.1 kB]
Get:13 http://us.archive.ubuntu.com precise-updates/universe i386 Packages [291 kB]
Get:14 http://us.archive.ubuntu.com precise-updates/multiverse i386 Packages [17.3 kB]
Get:15 http://us.archive.ubuntu.com precise-updates/main TranslationIndex [208 B]
Get:16 http://us.archive.ubuntu.com precise-updates/multiverse TranslationIndex [202 B]
Get:17 http://us.archive.ubuntu.com precise-updates/restricted TranslationIndex [202 B]
Get:18 http://us.archive.ubuntu.com precise-updates/universe TranslationIndex [205 B]
Hit http://us.archive.ubuntu.com precise-backports/main Sources 
Hit http://us.archive.ubuntu.com precise-backports/restricted Sources 
Hit http://us.archive.ubuntu.com precise-backports/universe Sources 
Hit http://us.archive.ubuntu.com precise-backports/multiverse Sources
Hit http://us.archive.ubuntu.com precise-backports/main amd64 Packages
Hit http://us.archive.ubuntu.com precise-backports/restricted amd64 Packages
Hit http://us.archive.ubuntu.com precise-backports/universe amd64 Packages
Hit http://us.archive.ubuntu.com precise-backports/multiverse amd64 Packages
Hit http://us.archive.ubuntu.com precise-backports/main i386 Packages
Hit http://us.archive.ubuntu.com precise-backports/restricted i386 Packages
Hit http://us.archive.ubuntu.com precise-backports/universe i386 Packages
Hit http://us.archive.ubuntu.com precise-backports/multiverse i386 Packages
Hit http://us.archive.ubuntu.com precise-backports/main TranslationIndex
Hit http://us.archive.ubuntu.com precise-backports/multiverse TranslationIndex
Hit http://us.archive.ubuntu.com precise-backports/restricted TranslationIndex
Hit http://us.archive.ubuntu.com precise-backports/universe TranslationIndex
Hit http://us.archive.ubuntu.com precise/main Translation-en
Hit http://us.archive.ubuntu.com precise/multiverse Translation-en
Hit http://us.archive.ubuntu.com precise/restricted Translation-en
Hit http://us.archive.ubuntu.com precise/universe Translation-en 
Get:19 http://us.archive.ubuntu.com precise-updates/main Translation-en [425 kB]
Hit https://repo.zimbra.com precise Release.gpg 
Get:20 http://us.archive.ubuntu.com precise-updates/multiverse Translation-en [10.1 kB]
Get:21 http://us.archive.ubuntu.com precise-updates/restricted Translation-en [3,869 B]
Get:22 http://us.archive.ubuntu.com precise-updates/universe Translation-en [168 kB] 
Hit http://us.archive.ubuntu.com precise-backports/main Translation-en 
Hit http://us.archive.ubuntu.com precise-backports/multiverse Translation-en
Hit http://us.archive.ubuntu.com precise-backports/restricted Translation-en
Hit http://us.archive.ubuntu.com precise-backports/universe Translation-en
Hit https://repo.zimbra.com precise Release 
Hit https://repo.zimbra.com precise/zimbra Sources 
Hit https://repo.zimbra.com precise/zimbra amd64 Packages
Ign https://repo.zimbra.com precise/zimbra TranslationIndex
Ign https://repo.zimbra.com precise/zimbra Translation-en_US
Ign https://repo.zimbra.com precise/zimbra Translation-en
Fetched 4,061 kB in 2s (1,531 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree 
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
 zimbra-lmdb zimbra-lmdb-lib zimbra-openldap-client zimbra-openldap-lib zimbra-openldap-server
5 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/2,107 kB of archives.
After this operation, 1,024 B of additional disk space will be used.
Do you want to continue [Y/n]? Y
E: changelog for this version is not (yet) available; try https://launchpad.net/ubuntu/+source/zimbra-openldap/+changelog
E: changelog for this version is not (yet) available; try https://launchpad.net/ubuntu/+source/zimbra-openldap/+changelog
E: changelog for this version is not (yet) available; try https://launchpad.net/ubuntu/+source/zimbra-openldap/+changelog
E: changelog for this version is not (yet) available; try https://launchpad.net/ubuntu/+source/zimbra-openldap/+changelog
E: changelog for this version is not (yet) available; try https://launchpad.net/ubuntu/+source/zimbra-openldap/+changelog
Reading changelogs... Done
(Reading database ... 69774 files and directories currently installed.)
Preparing to replace zimbra-lmdb-lib 2.4.44-1zimbra8.7b8.12.04 (using .../zimbra-lmdb-lib_2.4.44-1zimbra8.7b9.12.04_amd64.deb) ...
Unpacking replacement zimbra-lmdb-lib ...
Preparing to replace zimbra-openldap-client 2.4.44-1zimbra8.7b8.12.04 (using .../zimbra-openldap-client_2.4.44-1zimbra8.7b9.12.04_amd64.deb) ...
Unpacking replacement zimbra-openldap-client ...
Preparing to replace zimbra-openldap-server 2.4.44-1zimbra8.7b8.12.04 (using .../zimbra-openldap-server_2.4.44-1zimbra8.7b9.12.04_amd64.deb) ...
Unpacking replacement zimbra-openldap-server ...
Preparing to replace zimbra-openldap-lib 2.4.44-1zimbra8.7b8.12.04 (using .../zimbra-openldap-lib_2.4.44-1zimbra8.7b9.12.04_amd64.deb) ...
Unpacking replacement zimbra-openldap-lib ...
Preparing to replace zimbra-lmdb 2.4.44-1zimbra8.7b8.12.04 (using .../zimbra-lmdb_2.4.44-1zimbra8.7b9.12.04_amd64.deb) ...
Unpacking replacement zimbra-lmdb ...
Setting up zimbra-lmdb-lib (2.4.44-1zimbra8.7b9.12.04) ...
Setting up zimbra-openldap-lib (2.4.44-1zimbra8.7b9.12.04) ...
Setting up zimbra-openldap-client (2.4.44-1zimbra8.7b9.12.04) ...
Setting up zimbra-openldap-server (2.4.44-1zimbra8.7b9.12.04) ...
Setting up zimbra-lmdb (2.4.44-1zimbra8.7b9.12.04) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
root@mail:~# su - zimbra
zimbra@mail:~$ zmcontrol restart
Host mail.testdomain.com
 Stopping zmconfigd...Done.
 Stopping zimlet webapp...Done.
 Stopping zimbraAdmin webapp...Done.
 Stopping zimbra webapp...Done.
 Stopping service webapp...Done.
 Stopping stats...Done.
 Stopping mta...Done.
 Stopping spell...Done.
 Stopping snmp...Done.
 Stopping cbpolicyd...Done.
 Stopping archiving...Done.
 Stopping opendkim...Done.
 Stopping amavis...Done.
 Stopping antivirus...Done.
 Stopping antispam...Done.
 Stopping proxy...Done.
 Stopping memcached...Done.
 Stopping mailbox...Done.
 Stopping logger...Done.
 Stopping dnscache...Done.
 Stopping ldap...Done.
Host mail.testdomain.com
 Starting ldap...Done.
 Starting zmconfigd...Done.
 Starting logger...Done.
 Starting mailbox...Done.
 Starting memcached...Done.
 Starting proxy...Done.
 Starting amavis...Done.
 Starting antispam...Done.
 Starting antivirus...Done.
 Starting opendkim...Done.
 Starting cbpolicyd...Done.
 Starting snmp...Done.
 Starting mta...Done.
 Starting stats...Done.
 Starting service webapp...Done.
 Starting zimbra webapp...Done.
 Starting zimbraAdmin webapp...Done.
 Starting zimlet webapp...Done.
zimbra@mail:~$ zmcontrol status
Host mail.testdomain.com
 amavis Running
 antispam Running
 antivirus Running
 cbpolicyd Running
 ldap Running
 logger Running
 mailbox Running
 memcached Running
 mta Running
 opendkim Running
 proxy Running
 service webapp Running
 snmp Running
 stats Running
 zimbra webapp Running
 zimbraAdmin webapp Running
 zimlet webapp Running
 zmconfigd Running
zimbra@mail:~$

2 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *