Thunderbird dropping/resyncing IMAP Folders on TB restart

Discussion forum for Enterprise Edition.
Post Reply
METom
Posts: 30
Joined: Wed Nov 28, 2012 6:13 pm

Thunderbird dropping/resyncing IMAP Folders on TB restart

Post by METom » Mon Oct 07, 2013 11:40 am

Hi,
I am on ME EE 7.53 and Thunderbird 24.0.
I do have 4 ME-Accounts configured as IMAP(S)/SMPT accounts and 2 Account to different (non ME-)mail servers (IMAP(S)/SMTP as well).

Every time, I close and restart TB and then open TBs 'Activity monitor' I can see, that TB is DELETING (!!) some of my subfolders (in both locations, folder below 'inbox' and folder below root like e.g.: 'sent items'), then starting do download/syncing the contained emails afterwards or on next start on TB.
This isn't going to be stable at no time.
Even if I start TB 10 times (after having waited for completing the syncs shown in 'Activity monitor), TB will either delete/resync random other folders or even the ones, TB just deleted/resynced in the last run.

This does ONLY happen with those configured email accounts that are hosted on my ME Server. The other - non-ME - accounts are not affected by this behavior.

For what is really frustrating me, that this costed me nearly 400,- EURO because I had to have my computer connected by UMTS on my last stay for a projects on site visit while being on my roaming data plan in a foreign country (and I had to have my eMail/TB connected to have proper project support).
I then wondered, why I had to reorder 150 MB data volume every hour or so... :-(((

I then turns out, that this heavy data usage could be traced down to this ME/TB-Bug.

Please advice on how to resolve/mitigate/avoid this behavior.

Best regards
Thomas


This was

MailEnable
Site Admin
Posts: 4441
Joined: Tue Jun 25, 2002 3:03 am
Location: Melbourne, Victoria Australia

Re: Thunderbird dropping/resyncing IMAP Folders on TB restar

Post by MailEnable » Mon Oct 07, 2013 1:54 pm

Hi Thomas,

here are some suggestions.

Firstly I would suggest enabling thunderbirds logging - since it will give more insight as to why it is resyncing.

Secondly, this thread seems to indicate that there have been problems with Thunderbird's behaviour: https://bugzilla.mozilla.org/show_bug.cgi?id=663069 (it is strange though that the issue only occurs with the ME server)....

In terms of diagnosing, I can explain what provisions exist in the IMAP protocol to facilitate this (and specifically what to look for in MailEnable).

When a client issues a select on a folder, the server will return a UIDVALIDITY statement for the folder. This value should always be the same, otherwise the client will detect the change and "act on it".
A change in the UIDVALIDITY will result in the client throwing away all its contents, and re-downloading the messages from the server.
As far as I am aware, this is the only way that a server can tell the client to reset its contents (basically it is telling it to forget and reload).
The value MailEnable uses for each folders UID validity is the creation date of the _index.xml. The creation date of the _index.xml file will be the date at which it was first written (the only exception to this being if you are using an obscure NAS that does not support creation dates - there is a switch that can be used read a flag file modified date in this case - but it needs to be turned on). If you are using a NAS storage solution, then you may wish to review its capabilities (specifically to see if it supports creation dates). I think though that if it were a NAS problem deficiency, then this would be the least of your problems though!

Another possibility is that something is changing the creation date (perhaps a backup utility or replication engine is hard stamping the creation date as a means of flagging incremental backups). If you are able to reproduce the issue for a given folder, perhaps take note of the UIDVALIDITY value being returned and see if it changes (or you can take note of the creation date of the folder's _index.xml file).
Regards, Andrew

METom
Posts: 30
Joined: Wed Nov 28, 2012 6:13 pm

Re: Thunderbird dropping/resyncing IMAP Folders on TB restar

Post by METom » Sun Oct 13, 2013 12:05 pm

Hi,

just a short feedback:

1.) after it was stable for about 2 days while using TB (not restarting it after having checked, that all IMAP accounts do hold all folders with all eMail, no sync activity in Activity Monitor other then the actual eMail flow like new email, deleting email, moving email) yesterday and today activity monitor reported deletion of (some - not all) folders of some (not all) of the accounts, then restarting to redowonload all of the emails in some (not all) of the folders deleted in one of the former TB restarts.
Currently again TB downloaded 3GB on eMails, that were just downloaded/synced a few days/hours ago! It is sooo frustrating.
BTW: if i turn on logging for IMAP in TB, it will grow to size of Gigabytes as well, as it will log all emails it downloads as well (10K eMails in some folders!!)
That's why I try to catch a situation, where TB is fully synced and just will drop/delete the folders on a restart.
Hopefully this will produce a small and meaningful log then.....

2.) I had a look at the ME _index.xml files of the folders being dropped/deleted/resynced in TB. The creation date will stay stable and has a date months back in the past.
So this problem seems not to be related to something messing around with _index.xml file creation date.

METom
Posts: 30
Joined: Wed Nov 28, 2012 6:13 pm

Re: Thunderbird dropping/resyncing IMAP Folders on TB restar

Post by METom » Sun Oct 13, 2013 12:52 pm

Hi Andrew,

I'll send you a (small) TB log of of a TB start, where TB deleted a couple of folders on startup, to your PM (may hold sensitive data, so no public forum upload).
As far as I can see, the deleted folders were not mentioned in the log file at all!! :-(
So it seem they were just 'missing' while TB gets the enumeration of the server side folders from ME-Sever by IMAP protocol, thus then deleting them locally, because they don't seem to be necessary on the local TB cache any more - despite the fact, that they were subscribed for local sync before.
My tests caused several Gigabytes (15+) on download data from ME-Server to my local Client now - if this will happen again, when I have to rely on my UMTS/Mobile data plan while roaming in Europe, this will cost me a fortune (again)! :-(

Post Reply