Administrators on the MacEnterprise.org mailing list recently discussed their observations about the behavior of the Microsoft Outlook for Mac Database file. Unlike Microsoft Entourage, which stores messages within the Database file and creates one multi-megabyte or even multi-gigabyte file, Outlook is suppose to store messages separately from the Database file. The purpose is to make Time Machine backups and similar management much easier.
However, one administrator noticed that with a brand new Outlook Database, one message with a 5MB attachment immediately grew the Database file itself by 5MB. This prompted him to speculate whether or not the Outlook Database file really was storing attachments separately as Microsoft claimed it should do.
By the end of the day Andy Ruff, a developer in the Macintosh Business Unit at Microsoft and Lead Program Manager for Outlook for Mac, stepped into the conversation to explain exactly how the Outlook Database works.
Data records and indices
Like most databases, the Outlook database contains primarily data records and indices. Data records are the actual messages themselves and indices are like the index of a book, helping to quickly find a message.
Indices are intended to be frequently accessed (thousands at a time) while the content is displayed in real-time. The message list seen in the Inbox, for example, is drawn from the indices of the database. When a message is opened and displayed, the content is drawn from the data records.
Outlook stores only the indices in the Database (with a capital “D”) file and stores the data records as a series of files nested within folders. The Database file and its folders are located here:
~/Documents/Microsoft User Data/Office 2011 Identities/Main Identity/Database
~/Documents/Microsoft User Data/Office 2011 Identities/Main Identity/Data Records
In Andy’s words:
The “Database” file is actually better thought of as the “Index” … Indices are generated from data records over time —when you get a new message, we update the indices and store the data record. For this reason, there’s no “value” to backing-up the index; if catastrophic failure occurs (e.g. Hard disk crashing), Outlook rebuilds the index file by reading each data record.
Why is the Outlook database growing?
Outlook does store attachments and mail messages within the Data Records folders but it stores them separately. However, as a message with an attachment is received, the attachment information is temporarily stored in the indices before moving to its final place within the Data Records folders. This causes the Database file to grow. And, like most databases, when data is removed the blocks that stored the data are not recovered to shrink the database. Instead, they are left in place to be reused by new data.
That means your Index file will typically be the size of all the indices needed by Outlook plus the largest attachment you’ve received in Outlook.