Migration to Carbonio CE

Migration to Carbonio CE#

A migration procedure to Carbonio CE allows to import configurations and data into a mail server infrastructure based on Carbonio CE from an existent infrastructure, which is about to be decommissioned or dismissed.

This section does not discuss how configuration and data are exported from the Source infrastructure, because the tasks required to extract them may vary substantially.

This section discusses a high-level overview of the second part of such a procedure: to import an existent mail server infrastructure into Carbonio CE. Indeed, we believe it necessary to focus on some key points of the import procedure and that it is necessary to not underestimate the efforts required to successfully complete a whole export/import cycle.

We divide the import procedure into two main areas:

Provisioning

  • Import of domains, user accounts, and distribution lists

  • (optional) Import of Classes of Services

Data

  • Import of e-mails

  • Import of Calendars

  • Import of Contacts

The import procedure relies on Open Source tools, so there should be no problem during the various tasks that are required. More on this later, we need to define some concepts beforehand.

Definitions#

Before we enter the discussion, let’s define some terms to avoid misunderstandings.

Source

The Source infrastructure (Source) is the mail server from which to move away and that, at the beginning of the procedure, contains everything. It is important to stress the facts that we do not make any assumption on the Source (which can be any software), and that the Source is only needed to extract data, therefore is not touched in any way by the procedure.

Destination

The Destination infrastructure (Destination) is the Carbonio CE installation to which you want to move your existing mail server.

Procedure

The Procedure is a set of tasks that guide you in the import of all your data to the Destination. It requires access to the CLI of both Source and Destination, though it is assumed that all the data have been already exported when you start the Procedure.

Provisioning

Provisioning refers to the set of all data and metadata about the Domains, accounts, COSes, and distribution lists that are present on the Source before the start of the Procedure.

Data

Data refers to the set of all the e-mails that are present on the Source before the start of the Procedure.

Import Procedure#

The Procedure is divided in two phases. Phase one is a preparation phase, whose purpose is to make sure that you have all necessary elements to start the import, while phase two is where all the imports on the Destination are carried out.

Phase One: Requirement Checklist#

The following is the list of requirements that need to be satisfied:

Provisioning Requirements

  • provisioning files for Domains and accounts in LDIF format

Data Requirements

  • IMAP Access for e-mail-import: indeed, the size of all e-mails to be transferred might be so high that a backup/restore or other procedures might be too slow and inefficient

  • Calendar data exported in ICS format

  • Contacts data in CSV format

Carbonio CE Requirements

  • Carbonio CE must be up&running during the whole procedure and the Backup must be initialized

  • To prevent any LDAP conflicts, we suggest to configure Carbonio CE with a local domain, different from any domain that is going to be imported. You can use, for example, a domain like carbonio.local

  • All the CLI commands must be executed, unless differently specified, on the Node featuring the Mailstore & Provisioning Role as the zextras user

  • Source backup must be available from zextras user and must not use Carbonio backup path

The actual tools to be used to obtain these files may vary, depending on the Source.

Phase Two: Import to Destination#

This phase represents the operative part of the Procedure. We are currently polishing out the commands that are required and choosing the most appropriate tools to carry out the steps.

The commands in the next sections show how to accomplish the import on a Carbonio CE infrastructure and are given as examples only. You may choose any other approach or tool.

More information and how-to will soon be published and will be linked here as soon as they will be available.

Domains and Accounts Provisioning#

We assume that domain files are stored in files /tmp/backup/example.com.ldif (domain and accounts structure for example.com) and /tmp/backup/acme.corp.ldif (domain and accounts structure for acme.corp) on the LDAP Node. They can be imported using the slapadd command:

# /opt/zextras/common/sbin/slapadd -F /opt/zextras/data/ldap/config/ -v -l /tmp/backup/example.com.ldif
# /opt/zextras/common/sbin/slapadd -F /opt/zextras/data/ldap/config/ -v -l /tmp/backup/acme.corp.ldif

Import of E-mails#

The most functional and standard way to read from the Source and write into the Destination is to use the IMAP protocol. For this reason, we suggest imapsync as the tool to perform IMAP migration.

Hint

We suggest not to use the Global Administrator account to perform this task, but to create temporary Administrator accounts in the Source and in the Destination to be used for the purpose of import process only, so you can safely remove them once the synchronisation has been completed, to keep the Administrators’ action history clear. These will be denoted as SOURCE_ADMIN_ACCOUNT and CARBONIO_IMPORT_ACCOUNT in the command below.

We assume that you have already obtained credentials of an account that can access all the accounts on the Source, and you have already created a Global Administrator to be used for the import only.

In the following command, that must be executed on the Node hosting Carbonio Mails and Calendar, replace all values written in ALL_CAPS with corresponding values, suitable for your infrastructure.

See also

The command uses the official imapsync’s docker image; to download it or to learn on alternative approaches (e.g., if you want to install and use imasync) and documentation, please refer to imapsyncs’s official documentation.

zextras$ for account in $(carbonio prov -l gaa SOURCE_DOMAIN); do \
docker run --rm gilleslamiral/imapsync imapsync \
--host1 SOURCE_ENDPOINT_ADDRESS --ssl1 --user1 ACCOUNT --authuser1 SOURCE_ADMIN_ACCOUNT --password1 SOURCE_ADMIN_PASSWORD \
--host2 CARBONIO_PROXY_NODE_IP --ssl2 --user2 ACCOUNT --authuser2 CARBONIO_IMPORT_ACCOUNT --password2 CARBONIO_IMPORT_ACCOUNT_PASSWORD; \
done

Import of ICS Calendars#

We assume that the Calendar files of all users has been exported and placed into Carbonio Mails & Calendar Node’s /tmp/backup/ directory, in ICS Format, e.g., /tmp/backup/user1@example.com.ics, /tmp/backup/user1@acme.corp.ics, and so on.

The files to be imported must have the standard ICS format, for example

Example ICS file
BEGIN:VCALENDAR
X-WR-CALNAME:Calendar
X-WR-CALID:0621afc2-6801-4577-8444-bc67a85b1fbf:10
PRODID:Carbonio-Calendar-Provider
VERSION:2.0
METHOD:PUBLISH
BEGIN:VTIMEZONE
TZID:Europe/Rome
BEGIN:STANDARD
DTSTART:16010101T030000
TZOFFSETTO:+0100
TZOFFSETFROM:+0200
RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=10;BYDAY=-1SU
TZNAME:CET
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETTO:+0200
TZOFFSETFROM:+0100
RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=3;BYDAY=-1SU
TZNAME:CEST
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
UID:8c9eb99d-5197-40ef-9a3f-0cd4845e808d
SUMMARY:Meeting with CEO
DESCRIPTION:-:::_::_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_
 :_:_:_:_:_:_:_::_:_::-\n have invited you to a new meeting!\n\nSubject: Meet
 ing with CEO \nOrganizer: "undefined \n\nTime: Wednesday\, 16 October\, 2024
  09:00 - 12:00\n \nInvitees: john.doe@example.com \n\n\n-:::_::_:_:_:_:_
 :_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_::_:_::-\n"
X-ALT-DESC;FMTTYPE=text/html:<html><body id='htmlmode'>-:::_::_:_:_:_:_:_:_:
 _:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_::_:_::-<h3>undefin
 ed have invited you to a new meeting!</h3><p>Subject: Meeting with CEO</p><p
 >Organizer: undefined</p><p>Location: </p><p>Time: Wednesday\, 16 October\,
 2024 09:00 - 12:00</p><p>Invitees: john.doe@example.com</p><br/>-:::_::_
 :_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_::_:_:
 :-</body></html>
ATTENDEE;CN=john.doe@example.com;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACT
 ION;RSVP=TRUE:mailto:john.doe@example.com
ORGANIZER:mailto:stefano@example.com
DTSTART;TZID="Europe/Rome":20241016T090000
DTEND;TZID="Europe/Rome":20241016T120000
STATUS:CONFIRMED
CLASS:PUBLIC
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
TRANSP:OPAQUE
LAST-MODIFIED:20240930T105154Z
DTSTAMP:20240930T105154Z
SEQUENCE:0
BEGIN:VALARM
ACTION:DISPLAY
TRIGGER;RELATED=START:-PT5M
DESCRIPTION:Reminder
END:VALARM
END:VEVENT
BEGIN:VEVENT
UID:af6362e5-8806-4427-952b-6d33aedb93b9
SUMMARY:Meeting with CEO
DESCRIPTION:-:::_::_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_
 :_:_:_:_:_:_:_::_:_::-\n have invited you to a new meeting!\n\nSubject: Meet
 ing with CEO \nOrganizer: "undefined \n\nTime: Wednesday\, 16 October\, 2024
  09:00 - 12:00\n \nInvitees: j.doe@example.com \n\n\n-:::_::_:_:_:_:_
 :_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_::_:_::-\n"
X-ALT-DESC;FMTTYPE=text/html:<html><body id='htmlmode'>-:::_::_:_:_:_:_:_:_:
 _:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_::_:_::-<h3>undefin
 ed have invited you to a new meeting!</h3><p>Subject: Meeting with CEO</p><p
 >Organizer: undefined</p><p>Location: </p><p>Time: Wednesday\, 16 October\,
 2024 09:00 - 12:00</p><p>Invitees: j.doe@example.com</p><br/>-:::_::_
 :_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_:_::_:_:
 :-</body></html>
ATTENDEE;CN=john.doe@example.com;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACT
 ION;RSVP=TRUE:mailto:j.doe@example.com
ORGANIZER:mailto:m.jane@example.com
DTSTART;TZID="Europe/Rome":20241016T090000
DTEND;TZID="Europe/Rome":20241016T120000
STATUS:CONFIRMED
CLASS:PUBLIC
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
TRANSP:OPAQUE
LAST-MODIFIED:20240930T105159Z
DTSTAMP:20240930T105159Z
SEQUENCE:0
BEGIN:VALARM
ACTION:DISPLAY
TRIGGER;RELATED=START:-PT5M
DESCRIPTION:Reminder
END:VALARM
END:VEVENT
END:VCALENDAR

They can be imported into Carbonio CE using the following command:

zextras$ for account in $(carbonio prov -l gaa SOURCE_DOMAIN); do \
echo "prov sa $account postRestURL '/Calendar?fmt=ics' /tmp/backup/${account}.ics"; \
done | carbonio

ICS files can be also be imported manually by each user according to the direction presented in the Working with Calendars section of the User Guides.

Import of CSV Contacts#

We assume that the Contacts files of all users has been exported and placed into Carbonio Mails & Calendar Node’s /tmp/backup/ directory, in CSV Format, e.g., /tmp/backup/user1@example.com.csv, /tmp/backup/user1@acme.corp.csv, and so on.

The files to be imported must have the following format.

Example CSV file
"company","email","firstName","jobTitle","lastName","middleName","mobilePhone","namePrefix","nickname","workCity","workCountry","workPostalCode","workState","workStreet","workURL"
"ACME","j.doe@example.com","john","CEO","doe","","","","Boss",“Vicenza”,“IT”,“36100”,"","","www.example.com"
"ACME","m.jane@example.com","mary","CFO","jane","","+39 000000000","","MJ",“Vicenza”,“IT”,“36100”,"","","www.example.com"
"ACME","p.sullivan@example.com","Peter","Head of HR","Sullivan","Raymond","","mr","Ray",“Vicenza”,“IT”,“36100”,"","","www.example.com"
"ACME","d.smith@example.com","Donald","CTO","Smith",“”,"","","Don",“Vicenza”,“IT”,“36100”,“”,“”,"www.example.com"

They can be imported into Carbonio CE using the following command:

zextras$ for account in $(carbonio prov -l gaa SOURCE_DOMAIN); do \
echo "prov sa $account postRestURL '/Contacts?fmt=csv&csvfmt=thunderbird-csv' /tmp/backup/${account}.csv"; \
done | carbonio

CSV files can be also be imported manually by each user according to the direction presented in the Working with Contacts section of the User Guides.

Limitations of the Procedure#

The Procedure does migrate:

  • Domains

  • COSes

  • Accounts

  • Email Messages (sent/received)

  • Personal folders

  • Shared folders

  • Mailing/Distribution lists

  • Signatures

  • Calendars

  • Contacts

Other elements that do not belong to a Carbonio installation will not be migrated.

Contents