Migrating large amounts email data to new server


SUMMARY

In order to minimise downtime when migrating a large amount of email data, it is possible to copy the email data before having to stop services to users. Normally, when you copy email from one server to another, when moving an installation, the source server is not available to users. So email cannot be sent or read by users until the whole copy is complete. This copy can take a long time depending on the size of data and the number of files, so to minimise downtime there is a couple of options available. The simplest is to use robocopy to copy the data, but if this is not possible then using FreeFileSync with FTP provided by FileZilla is an alternative, thought there is another step involved.

DETAIL

Using Robocopy

If you are able to connect a share to the new server from the current one, you can use robocopy to copy the required files from the current server. You will need to adjust the paths to where you have installed MailEnable and what the shared path is.

robocopy "C:\Program files (x86)\Mail Enable\Config" X:\Config /mir /mt /R:0

robocopy "C:\Program files (x86)\Mail Enable\Postoffices" X:\Postoffices /mir /mt /R:0

robocopy "C:\Program files (x86)\Mail Enable\Bad Mail" "X:\Bad Mail" /mir /mt /R:0

robocopy "C:\Program files (x86)\Mail Enable\Quarantine" "X:\Quarantine" /mir /mt /R:0

Be careful with the path order. The source is before the destination. i.e. in the examples above the destination is X: drive. The /mir option will delete anything in the destination which does not match the source, so make sure your folder paths are correct. If you only want to copy you can do:

robocopy "C:\Program files (x86)\Mail Enable\Config" X:\Config /E /mt /R:0

The initial sync may take a while, but subsequent ones will be much faster. When you are ready to do the swap to the new server, you would stop all the mail services on the existing server so nothing is altering the files, then do the robocopy commands again for a final time and then point your users to the new server. If you are keeping the same IP addresses for the new server, i.e. you are going to move the IP address from the old server to the new one, then you can also include the queues, otherwise use the server migration article information on how to handle this:

robocopy "C:\Program files (x86)\Mail Enable\Queues" X:\Queues /mir /mt /R:0

If you want to also move the logs to the new server these can also be included with:

robocopy "C:\Program files (x86)\Mail Enable\Logging" X:\Logging /mir /mt /R:0

Using FTP

When migrating data from one server to another, you can use FreeFileSync which will migrate only the changed files from one location to another, using the Mirror option. This will allow you to sync the majority of the email data from one server to another instead of only copying the email data across a the time of migration, during which the source server is unavailable for users to access. When migrating to a new server it is important to maintain the creation date of index coming across, as these are used to sync clients using IMAP and ActiveSync. This is why you don't want users to connect to the new server until all the email data is across as new indexes will be created and flags lost, and IMAP clients will have to download all their email again. FTP will not preserve the creation date, so you need to copy the index files as described after the file sync. This index file copy requires that you install 7-Zip on both the current server and the new server.

You need to use FileZilla Server or another FTP service on the new server. FreeFileSync needs to be able to compare timestamps of files in order to know what to update and the Microsoft FTP server does not support the required FTP command.

So the process for the data migration assumes that MailEnable is installed and configured on the new server.

1) Install FreeFileSync onto the current server.
2) Install FileZilla on the new server. Configure it to only accept connections from the current server (there is an option to restrict connections to an IP address), and open any firewall as needed.
3) Under FileZilla create a user for the current server to use and mapped folders to the Config directory and the Postoffices directory.
4) Migrate the data to these folders using the Mirror option in FreeFileSync.
5) Keep doing step 4 until the time of the migration time is fairly small. You want to do this close to the migration date, as you will still need to do a final sync, so you want the changes to the data to be minimal.
6) Do the server migration, as described in the server migration article, and when you need to migration the email data do a last sync which will update the destination server. The mail services on the current server need to be stopped when this happens and after this users should not be able to connect to the old server since it would cause a resync of their email client.

When using FreeFileSync the time will vary, but expect it will be doing 100k files an hour. After the actual files are migrated, it may be able to check a million files for differences in 2 minutes.

After the last sync is done with FTP you will still need to copy across the index files to preserve the creation date, to avoid client resync with IMAP. This can be done by using 7-Zip in a command prompt. So on the current server open a command prompt and enter the following. The 7z.exe and 7zG.exe program location would either need to be in your PATH environment variable, or add the path to the exe in the command prompt. Change the path to the Postoffices folder if you have this in a different location.

7z a -r -t7z -mtc -mta -mtm c:\indexes.7z "C:\Program Files (x86)\Mail Enable\Postoffices\*.xml"

The indexes.7z file can then be copied to the new server and extracted with:

7zG x c:\indexes.7z -o"C:\Program Files (x86)\Mail Enable\Postoffices"

You will be prompted to overwrite the existing indexes, which you will need to indicate yes to. After the extraction you can continue with your server migration.

REFERENCES

FreeFileSync website: https://freefilesync.org/

FileZilla: https://filezilla-project.org/

7-Zip: https://www.7-zip.org/

 

 



Product:MailEnable
Article:ME020714
Module:General
Keywords:migrate,sync,move,ftp
Class:HOWTO: Product Instructions
Revised:Tuesday, December 13, 2022
Author:
Publisher:MailEnable