Upgrade

Upgrade#

This section lists known issues that may affect Carbonio during or after the upgrade process. Most of them may arise when upgrading from old releases.

Known Issues (Current Release)#

This is a list of known issues that impact Carbonio after the upgrade to 24.9.0.

Advanced modules not starting

There are a few corner cases that prevent the advanced modules, provided by the Service Roles, to be running. The problem is that the Node featuring the Mailstore & Provisioning Role can not communicate with the Database (PostgreSQL) or DB Connector (pgpool), depending on your setup. You can check if this is the case if you find in log file /opt/zextras/log/mailbox.log error messages like:

2024-07-29 10:13:58,575 ERROR [main] [] extensions -
Error getting connection from data source org.postgresql.ds.PGSimpleDataSource@4151251b org.jooq.exception.DataAccessException:
Error getting connection from data source org.postgresql.ds.PGSimpleDataSource@4151251b at com.zextras.modules.core.db.LazyDataSourceConnectionProvider.acquire(BaseDSLContextProvider.java:72) ~[?:?]

Caused by: org.postgresql.util.PSQLException: Connection to 127.78.0.25:20000 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

Caused by: java.net.ConnectException: Connection refused

2024-07-29 10:14:00,642 FATAL [main] [] extensions - Unable to initialize zextras: org.jooq.exception.DataAccessException:
Error getting connection from data source org.postgresql.ds.PGSimpleDataSource@4151251b at org.jooq_3.14.8.POSTGRES.debug (null:-1)

Caused by: org.postgresql.util.PSQLException: Connection to 127.78.0.25:20000 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

Caused by: java.net.ConnectException: Connection refused

In order to fix the problem, ensure that PostgreSQL is reachable and then restart the mailbox.

zextras$ zmmailboxdctl restart

IMAP folders with Cyrillic (UTF-7) characters

If you subscribe to an IMAP folder whose name contains Cyrillic (UTF-7) characters, chances are that you fail, even if you already subscribed to that folder. The reason is a missing jar, that you need to download and save in directory /opt/zextras/mailbox/jars/.

To verify the file’s integrity, execute the command

#  sha256sum zm-charset-22.5.0.jar

The output of the command includes the SHA256 checksum of the file, which must be:

13ce2d47274df3016ed8ec5d32439fc6a0c03d1dff04bd92e374bcc0ef049548

SLF4J related errors

In case you find on Carbonio messages like the following:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/zextras/lib/jars/carbonio-message-dispatcher-auth.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/zextras/lib/jars/log4j-slf4j-impl-2.20.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/zextras/mailbox/jars/log4j-slf4j-impl-2.20.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]

While these are only warnings, nonetheless they may become annoying. The first line is important and gives the reason for the warning messages: multiple version of the same class are present in the class path and therefore you need to remove manually the duplicates (or better rename the .jar files before to make sure you do not cause other problems): /opt/zextras/lib/jars/log4j-slf4j-impl-2.20.0.jar and /opt/zextras/mailbox/jars/log4j-slf4j-impl-2.20.0.jar.

Known Issue (Older Releases)#

During the upgrade from an older Carbonio version, you may encounter one or more of the following issues that require a manual intervention.

New GPG Key of Grafana in Ubuntu 22.04
The new package carbonio-storages must be installed
Monitoring: a new configuration file for carbonio-prometheus package must be installed
Initialise domains for Delegated Administrators
Some accounts are not migrated during upgrade
PostgreSQL Upgrade fails
A missing library causes a RHEL Netcat Issue
Domain Initialisation fails with an error
The advanced modules do not start due to miscommunication with the Database or Database Connector Roles
Folders with Cyrillic (UTF-7) characters can not be subscribed using IMAP
There are warning messages about SLF4J in the log files

GPG Key of Grafana in Ubuntu 22.04

If you migrate from Ubuntu 20.04 to 22.04 and use Monitoring Role’s Grafana, you need to update the location on which its GPG key is stored to prevent warning during upgrades. Log in to the Node on which the Carbonio Monitoring Role is installed and execute the following commands.

Remove existent repository

# add-apt-repository -r "deb https://packages.grafana.com/oss/deb
stable main"

Retrieve GPG key

# wget -q -O - https://packages.grafana.com/gpg.key | \
gpg --dearmor | sudo tee /usr/share/keyrings/grafana.gpg > \
/dev/null

Add repository

# echo "deb [signed-by=/usr/share/keyrings/grafana.gpg] \
https://packages.grafana.com/oss/deb stable main" | sudo tee -a \
/etc/apt/sources.list.d/grafana.list

New Package Installation

A new package must be installed on the Node featuring the Database Connector Role. Log in to that Node and execute command

Ubuntu

# apt install carbonio-storages

RHEL

# dnf install carbonio-storages

Carbonio Monitoring

While upgrading the carbonio-prometheus package, you will be prompted with the following text:

Configuration file '/etc/carbonio/carbonio-prometheus/prometheus.yml'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ?  Your options are:
Y or I  : install the package maintainer's version
N or O  : keep your currently-installed version

You must replace the existing file with the new one, therefore answer Y or I.

Delegated Administrators

In this release, you need to initialise again the domain(s) for Delegation: for each domain that has active Delegations, in the Carbonio Admin Panel go to Domains ‣ Manage ‣ Delegated Domain Admins and click the INIT DOMAIN button.

Backup

If the Global Administrator receives an e-mail notification about some accounts not migrated to use the new backupEnabled attribute, run the following commands as the zextras user to fix the issue.

First, stop the backup migration service

zextras$ carbonio backup dostopservice migrate-backup-enabled-setting

Then, start the service again to trigger the migration

zextras$ carbonio backup dostartservice migrate-backup-enabled-setting

Postgresql Upgrade Issue

During the upgrade of PostgreSQL, an error might be raised in case the existent databases have been created with older version of libc:

2024-03-19 12:28:14.209 UTC [909825] HINT:  Rebuild all objects in this database that use the default collation and run ALTER DATABASE activesync REFRESH COLLATION VERSION, or build PostgreSQL with the right library version.
2024-03-19 12:28:19.669 UTC [909915] WARNING:  database "abq" has a collation version mismatch

In case you find in the log some messages similar to the one shown above, you need to recreate the collation and indexes of the databases using the following commands.

First, become the postgres user

# su - postgres

Then issue these two commands

$ psql -d abq -c "REINDEX DATABASE abq"
$ psql -d abq -c "ALTER DATABASE abq REFRESH COLLATION VERSION;"

These commands should probably be given for each of the databases, so you can copy & paste all the following commands:

psql -d abq -c "REINDEX DATABASE abq"
psql -d abq -c "ALTER DATABASE abq REFRESH COLLATION VERSION;"
psql -d powerstore -c "REINDEX DATABASE powerstore"
psql -d powerstore -c "ALTER DATABASE powerstore REFRESH COLLATION VERSION;"
psql -d activesync -c "REINDEX DATABASE activesync"
psql -d activesync -c "ALTER DATABASE activesync REFRESH COLLATION VERSION;"
psql -d auth -c "REINDEX DATABASE auth"
psql -d auth -c "ALTER DATABASE auth REFRESH COLLATION VERSION;"
psql -d "carbonio-docs-connector-db" -c "REINDEX DATABASE \"carbonio-docs-connector-db\""
psql -d "carbonio-docs-connector-db" -c "ALTER DATABASE \"carbonio-docs-connector-db\" REFRESH COLLATION VERSION;"
psql -d "carbonio-files-db" -c "REINDEX DATABASE \"carbonio-files-db\""
psql -d "carbonio-files-db" -c "ALTER DATABASE \"carbonio-files-db\" REFRESH COLLATION VERSION;"
psql -d carbonio_adm -c "REINDEX DATABASE carbonio_adm"
psql -d carbonio_adm -c "ALTER DATABASE carbonio_adm REFRESH COLLATION VERSION;"
psql -d core -c "REINDEX DATABASE core"
psql -d core -c "ALTER DATABASE core REFRESH COLLATION VERSION;"
psql -d ha -c "REINDEX DATABASE ha"
psql -d ha -c "ALTER DATABASE ha REFRESH COLLATION VERSION;"
psql -d postgres -c "REINDEX DATABASE postgres"
psql -d postgres -c "ALTER DATABASE postgres REFRESH COLLATION VERSION;"
psql -d template0 -c "REINDEX DATABASE template0"
psql -d template0 -c "ALTER DATABASE template0 REFRESH COLLATION VERSION;"
psql -d template1 -c "REINDEX DATABASE template1"
psql -d template1 -c "ALTER DATABASE template1 REFRESH COLLATION VERSION;"

RHEL Netcat Issue

In systems which feature RHEL 8 or RHEL 9, you might find in journalctl’s log an error about a missing library for netcat:

May 02 09:20:02 demo.example.com systemd[1]: Started Session 3 of User zextras.
May 02 09:20:02 demo.example.com bash[3910]: netcat: error while loading shared libraries: libbsd.so.0: cannot open shared object file: No such file or directory
May 02 09:20:02 demo.example.com systemd[1]: Started Session 4 of User zextras.
May 02 09:20:02 demo.example.com bash[3912]: netcat: error while loading shared libraries: libbsd.so.0: cannot open shared object file: No such file or directory

The solution is to install the libbsd package:

# dnf install libbsd

initDomainForDelegation fails

After performing the INIT DOMAIN operation (required for creating delegated administrators), the following error may occur.

zextras@:~$ carbonio admin initDomainForDelegation local.loc

Unknown Exception com.zextras.lib.http.HTTPClientException:
<soap:Envelope
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">soap:Header<context
xmlns="urn:zimbra"/></soap:Header>soap:Bodysoap:Fault<faultcode>soap:Client</faultcode><faultstring>
no such distribution list:
__monitoring_admins@local.loc</faultstring><detail> <Error
xmlns="urn:zimbra"><Code>account.NO_SUCH_DISTRIBUTION_LIST</Code><Trace>qtp2021053848-1571:1721660801727:f88a830e260d85fa</Trace></Error></detail></soap:Fault></soap:Body></soap:Envelope>

The workaround is to manually generate the empty ACL groups BEFORE initializing the domain. This can be done with the following commands:

zextras$ carbonio prov cdl __monitoring_admins@local.loc zimbraIsAdminGroup TRUE
zextras$ carbonio prov cdl __helpdesk_admins@local.loc zimbraIsAdminGroup TRUE
zextras$ carbonio prov cdl __groups_admins@local.loc zimbraIsAdminGroup TRUE
zextras$ carbonio prov cdl __users_admins@local.loc zimbraIsAdminGroup TRUE
zextras$ carbonio prov cdl __delegated_admins@local.loc zimbraIsAdminGroup TRUE
zextras$ carbonio prov cdl __domain_admins@local.loc zimbraIsAdminGroup TRUE

After executing the suggested commands, you can then carry out the INIT DOMAIN operation, from either the Carbonio Admin Panel or the CLI.

Upgrade

Contents

Upgrade#

Known Issues (Current Release)#

Known Issue (Older Releases)#