Is corruption hiding in your database?

Backup your Identity

Today we’re going to discuss the health of your Entourage Identity. Even though we encourage multiple snapshots of your Entourage Identity as backup, it’s always best to identify corruption early when there are no signs.

Entourage can work with no apparent problems, but corruption can be lurking in your database. So how do you know if you’re backing up a database that has problems?

  1. Entourage’s Database Utility can be used to verify
  2. Exporting your data as an Entourage archive (.rge) file and
  3. Exporting as MBOX files can also show problems.

Using this three step process, we’re going to verify that your database is working correctly.

Step One: Use Entourage’s Database Utility to Verify

When you hold down the Option key when launching Entourage, the Database Utility will open and allow you to check the health of your database.

However, even if the Identity, verifies as OK, it is not always 100% accurate so we are going to verify using two more steps.

Step Two: Export as Entourage archive file

Another way to check for corruption is to use the Export as Entourage archive (.rge) file under File –> Export. If you can export using this method, this is a very positive indication that your data is good. If there is corruption, it will fail and give you nothing. Just this week I was testing export and categories and found that my database had problems. I had no indication there were problems.

Step Three: Continue testing by exporting as MBOX files

This third step in the verify process will give you added assurance that your database is working correctly. Export your folders as MBOX files. You can drag a folder to the desktop to create an MBOX file, but if you have a lot of folders and subfolders, this can get to be a bit tedious. I suggest downloading Jolly Roger’s script Archive Message Folders

  • This script archives each folder in your Entourage folder list to a folder on your desktop named “Entourage Archives”.
  • The script preserves the hierarchy of the folders, as they exist in the Entourage folders list, by creating sub-folders in the “Entourage Archives” folder that correspond to the sub-folders in Entourage.
  • The exported MBOX files have the same name as the folders in Entourage, with the exception that forward slash characters (/) are replaced with dash characters (-) to avoid problems with the Mac OS X file system.

The script will fail on the folder with corruption. This allows you to identify the folder with corruption. Once you know where the corruption starts, you can try dragging the messages in this folder to the desktop. It’s possible you will be able to recover all or most of the messages. Next step is to try deleting the messages and/or the corrupt folder. If you can delete the folder try running the script again. It’s possible you have only one folder that’s corrupted, but there could be others.

Were all three steps successful?

If verifying your data using the three step method gives you three positive results, you have two different formats with your data and you can know your database is working correctly.

What if export fails?

Make a Backup Copy Now!

OK, you’ve discovered corruption. As long as you can open your database you have a good chance of recovering your data. Make a copy of your Identity in the Finder and work off a copy when you are trying to recover. You never know when Entourage is going to go belly up and refuse to open. You can always revert to a copy to continue your recovery efforts.

Rebuild your Identity

If export failed, then you will need to rebuild your database. Again let me stress, make a duplicate in the Finder first as additional backup. A rebuild does make a duplicate, but sometimes in the process the original and backup get merged resulting in total disaster. It’s rare but it happens.

How successful your rebuild will be is determinded by:

  1. Degree of corruption
  2. Size of free space on your hard drive

Previously, in Rebuilding a damaged database that exceeded size limit the article details how using a large drive (500GB) allowed a successfully rebuild of the corrupt database. Even though this article was specific to Entourage X, it applies to any rebuild. As a consultant I have successfully been able to recover damaged databases using this method. I keep a 500 GB drive especially for this purpose with only Leopard and Office X, 2004 and 2008 installed.

If rebuild fails, you will have to manually move into a new Identity. See this page for directions.

Signs of database corruption

  • Problems viewing and opening items
  • Big grey blank of about 3-4 lines in Inbox which if I click on it, freezes the screen and or makes Entourage close down
  • Blank E-mails with no text
  • Messages that won’t delete
  • Crash and freeze when opening Entourage

These are some of the most common signs, but often obscure problems are a result of database corruption. Many times you can easily eliminate database corruption by creating a new Identity in Entourage and test your problem there. If the problem goes away most likely it’s database corruption. For more help troubleshooting your Identity see Troubleshooting Basics.

Share this:
  • Twitter
  • Facebook
  • del.icio.us
  • Digg
  • StumbleUpon
  • Google Bookmarks
  • PDF
  • Print

32 comments to Is corruption hiding in your database?

  • Keith Bechard

    I discovered my DB was corrupted when I tried to recover an email attachment (Excel spreadsheet). Excel would not load the file. Other attachments were also bad. Note that the Rebuild command cannot fix this. I have been able to successfully rebuild.
    Keith

  • Be sure the entire attachment was fully downloaded.

  • helen

    My Entourage database is corrupted. I have tried rebuilding but an error occurs in step 4 of the rebuild and it cannot complete. How do i fix this problem?
    thanks..

  • It could be the amount of free space you have on your drive. see this article that explains how more space allows a rebuild to complete.
    Rebuilding a damaged database that exceeded size limit
    I have 500GB free for rebuilding my clients databases. The key to recovering is to have the original copy of your Identity and work on clones of this. Each rebuild can actually make the problem worse so going back to a clone of the original database that has problems is your best option.
    Were you still able to open the Identity? If yes, in cases like this that rebuild fails, I manually move out the data and create a new Identity.
    How to manually move your data

  • Sue Toscano

    Hoping someone can help. My inbox fails load – I can see that I have messages in there and I can successfully get into my other folders like sent, etc. I had to rebuild on 5/31 – after reading some posts here I realized I probably just copied the corruption but don’t have enough free space to another another rebuild. I do have an mbox file of my inbox from 5/31.
    The error I receive is “Cannot open that feature there is an i/o error. ”
    Any suggestions on the steps to take here – I’d like to at least recover as much as I can.

  • I/O error almost always means disk damage.
    Use Disk Utility to verify your drive. It will give you the SMART status as well. If you don’t have a backup, I suggest you do this now. You drive could fail and you would loose everything.
    I recommend this drive that was also recommended by Macworld.
    Seagate ST310005FJA105-RK FreeAgent Desk for Mac 1 TB USB 2.0/FW800 External Hard Drive (Silver) Amazon Price: $159.95
    It comes with FireWire 800 cable; FireWire 800 to 400 cable; USB 2.0 cable. 5
    yr. warranty.
    Moving into a new Identity is your best option, but you need to have a stable drive first. If you need help I do recovery for Entourage Identities.

  • Jennifer

    entourage mac 2008 keeps freezing/crashing when I try to delete old files. I’ve checked, compressed and rebuilt DB twice but has not fixed it. Any suggestions?

  • A rebuild does not always fix corruption, but as long as you can open Entourage you can salvage the data. I suggest working off a copy in case it crashes so badly it won’t open.
    How to manually move your data. (when import fails and/or you need to move to a new Identity same version or revert to an older version)
    How to manually move your data when you need to move to a new Identity.
    You start exporting all items as .rge.
    If that fails, then export one item at a time as .rge. This will tell you where the corruption is located. If it’s messages, run the export as MBOX files next. The script will fail on the folder with corruption. You divide the folder and manually drag the subfolder until you find the corrupt message(s).
    Export folders as MBOX files (includes subfolders)
    Exporting as .rge does not get categories for calendar events, notes or tasks. I use Paul Berkowitz’s Import-Export scripts to recover the categories in these items.

  • Peter wynn

    I’m afraid I dont know what the “option key” is. None of the keys on my keyboard are labelled “option” I think I need to re-build my database but I have fallen at he first hurdle!
    Peter

  • The Option key is also known as the Alt key. See this Apple KB for help with the keyboard and shortcuts.

  • Parisa Urquhart

    Hi there, thank you for such valuable information.
    I updated my microsoft office 2008 and the office reminders window stopped showing my reminders.
    I verified my databse and it said it was corrupted with the following:
    ERROR: Corruption detected on index [CLRC:1]
    (what does this mean? )
    I rebuilt my database and the reminders successfully appeared again. BUT I followed your three step strategy to check if the database is definitely not corrupted. I tried to ‘export as entourage archive file’ and it seemed to work at first but then entourage suddenly quit.
    I had to rebuild my database 3 or 4 times when i had microsoft 2004. had hoped upgrading would solve the problem but it has not.
    Can you explain what the corruption message above means and what you think is causing the corruption?
    thanks

  • Rebuilding is not a guaranteed fix. When I do a recovery for a client’s database, after I rebuild, I export the data and create a new Identity. Often after rebuilding the export as .rge will fail, then I use the MBOX method to recovery the messages. It often fails and I have to go into the folders and find the corruption by dividing them into segments testing each segment by dragging it to the desktop. Often it takes over 20 tries to find the corrupt messages in one folder. Once you move your data into a new Identity, you’ll find 08 very stable.
    I’m not sure what the error, Corruption detected on index [CLRC:1], indicates other than you have problems. It could be hardware related like a bad block on your drive. Low disk space and not enough RAM can cause odd things to happen as well. See this page for how to check: Problems from Insufficient RAM and Free Hard Disk Space

  • Robert Kerby

    When I open Entourage I get this error message.
    Database Daemon Fatal Error
    The database Daemon has encountered an unknown error. The daemon will need to quit, and Entourage may need to be force quit.
    What do i do?

  • This error probably means database corruption. I would recommend rebuilding the database.
    Make a duplicate in the Finder first as additional backup. A rebuild does make a duplicate, but sometimes in the process the original and backup get merged resulting in total disaster. It’s rare but it happens.
    How to rebuild:

  • Adam

    I was able to successfully Rebuild the database where it was a corrupted Exchange Server sync’ed database. Only the corruption took hold a few days ago and synced deleted folders with the Exchange server before I could stop it. So now that I restored the database and thought that I was back in business, the Exchange server had other ideas and suggested my old folders/messages needed to be deleted upon sync.
    Now I’m reattempting the same Rebuild and will try to manually copy out the data (as an .rge file for example) before Exchange server sync activates (I’ll remove my network connections if I need to), then create a new Identity with the Exchange server account info and let it sync, then I’ll import the data and hope that it will sync with the server in such a way that it picks up the folders and emails as new data to sync.
    Any thoughts on whether or not my plan will work? It definitely takes some time since I’m working with a 10 GB database file and I’ve been working on this for 2 days straight now, day and night.

  • Before you start your recovery, I would ask your IT to make a backup of your messages on the server. You should also make a backup of your Identity.
    Consider deleting all messages on the server (make sure Entourage is set to work off line first). You need a primary source that you know is good, then delete all other data so old corrupted data does not sync back.
    Rather than rebuilding, I would export all messages as .rge or .MBOX files. Then create a new Identity and import. This would give you the best results rather than trying to rebuild.

  • Vince Benedetti

    I’m not able to access my database in Office 2004 for Mac. I get the message that it cannot be opened with my version of Entourage.
    But this should not be because the version of the Entourage is the same as for Word, i.e. 11.0.
    The database utility says that the database is not corrupted. Any ideas anyone?
    Thanks
    Vince Benedetti

  • You are running software that has not been updated.
    “This Identity Cannot Be Opened With This Version of Entourage”
    This message usually means, for example, any Identity saved with a 12.1.0 or later version will not open in 12.0.0 or 12.0.1. Since there was a database format change in 12.1.0 (SP1), Entourage 12.0.0 or 12.0.1 cannot open a database saved in the later version.
    Entourage 2004 also had a database upgrade from the original release. Once you upgrade, the Identity will open.

  • pram99

    currently running entourage version 12.0.0
    all of a sudden, upon launch, entourage would hang and ever launch. i tried rebuilding desktop numerous time, resulting in “your database is fine” message.
    tried creating new identity, entourage launched just fine. tried to switch to old identity, entourage hangs and won’t launch.
    at this point, naturally, i am very concerned about years of email archives a well as very important recent work related emails, site registration confirmation emails etc.
    please help 🙂

  • You need to update to 12.2.3. There have been many bug fixes since Office 2008 was released.
    Steps to update Office after installing from DVD
    You don’t mention which version of the OS you are running? If you upgraded to Snow Leopard then you need to clear old fonts.
    Microsoft fonts to delete after installing Snow Leopard (includes help deleting font caches)

  • Frank Kandt

    Diane – I’ve just recovered from a corruption problem, the process wasn’t fun. But I still don’t know why it occurred. Everything purring along for a year and a half using Entourage. Then bang…..data daemon errors and a mess.
    Problem is, I don’t know what I did to cause it (so I don’t know how avoid it again). Any tips for avoiding corruption? Are there things we should or shouldn’t do when working with Entourage? Are there other programs or functions that don’t “play nice”? It’s a helpless/anxious feeling just wondering when/if it will strike again.
    Any thought or comments you have based on your expertise here would be appreciated.
    – Frank

  • There is no test for what caused the corruption. All you can do is have a good backup strategy and test using the File > Export as .rge file to test occasionally.
    1) Don’t let your database get too big.
    2) Attachments that are large can cause probems
    Watching free drive space is crucial if you have a small drive. When you get down to the minimum allowed all sorts of bad things can start happening.
    Problems from Insufficient RAM and Free Hard Disk Space

  • jlv

    I can’t launch entourage, I get an error and then the Microsoft Database Util launches. I have treid to rebuild but it fails. I removed the Documents/Microsoft/User Data/ folder (kept a copy of the DB) and lib/microsoft folder. Then I created a new user account and tried to import my old data. Still corrupt. Then I dragged the old DB back in my new user profile. Same result… Any help?

  • You don’t mention what size your database is or what version. Read this article on rebuilding.
    Rebuilding a damaged database
    I do recovery for clients if you are unable to do this yourself. Send a comment and I’ll reply off list.

  • JB

    When I place the Jolly Roger script to Archive Message Folders in the Entourage Script Menu items folder and open Entourage, the script does not appear in the list. Entourage from Office 2004 (11.4.0) for Mac.

  • Verify that you placed the script correctly.
    ~/Documents/Microsoft User Data/Script Menu Items folder
    If you moved the folder into the Script Menu Items folder, you will need to select the script from that folder.
    Don’t confuse the System Script icon with the Entourage Script icon. Entourage’s Script menu is next to Help in the Menu bar.

  • JB

    Script checked removed from Archive Message folder to MUD Script Menu Items folder on second computer with Entourage 2004. It does not show up in Entourage on that computer either, strange.
    On my PowerBook (first computer evaluated) I found that if I double click on the Script, it opens Script Editor. I then selected run from within script editor and it opened Entourage on my PowerBook and created a full MBOX message folder backup. When I tried this on my MacBook (which is the database I’m most concerned with) it only created an archive of the Inbox. Does this suggest a corrupted database file on my MBook?
    Any thoughts why this script is not showing up in the AppleScript symbol menu in Entourage (to the right of the Help menu)?
    PB is G4/MB is Core Duo (2.2) – Entourage 2004

  • First, if the script stopped on the Inbox, there is corruption in the Inbox. You’ll need to make subfolders to find the corrupt message(s). You’ll have to drag folders to see if the other folders are OK, or move all the mail from the Inbox to a folder that lists at the bottom like xxxarchive.
    As far as the script not showing up, first drag the script to the Shared folder. Go to System Preferences –> Create a New User in Accounts. Switch to the New User by logging out/in or use Fast User Switching.
    Drag the script from Shared to the Microsoft User Data folder > Script Menu Items folder. Now open Entourage. You can skip setup. Do you see the script? If yes, the problem is in your user’s folder. Try repairing permissions first.

  • Neil Campbell

    I seem to have some stability problems since upgrading to Snow Leopard, Entourage seems to crash on a regular basis. I have reinstalled Snow Leopard, but the regular crashing continues. Any suggestions?
    Additionally, my ‘Main Identity’ database is 9.6GB. All of my Entourage usage is work correspondence & associated attachments. the attachments are used for future reference. Is there a limit to the size of the Entourage database?

  • Try removing the older Microsoft fonts that were replaced by newer ones in Snow Leopard.
    Microsoft fonts to delete after installing Snow Leopard includes help deleting font caches)
    There is no limit on the Entourage database, but as database get larger performance is affected. I do suggest that you sort older mail by year and make backups of those folder as MBOX files. E.G. Inbox 2008, Inbox 2009, Sent 2008, Sent 2009 etc. This will save your important older archives in case your database gets corrupted.
    I’ve also seen Entourage balk at exporting a folder that has too many messages. Once subdivided the folders would export. The limit is not firm but use 2GB as a guideline. There is no way to get size info without exporting as MBOX to view size in the Finder.

  • Gregory S. Marsiglia

    What now? I ran the “Archive Message Folders” script. It stopped on my “Junk E-mail” folder, so I deleted those messages by selecting the “Empty Junk E-mail” contextual menu item. After that, the badge on the folder indicates “2” messages, but none appear when I view the contents of the folder. The contents are sorted by date received, and “yesterday”‘s messages are shown as “2,” but the messages themselves are not listed. I have a screenshot of this. The Junk mail folder in Entourage is special; I am unable to delete it.
    I have am using version 12.2.6, with a database~11 gb in size, with years and years of mail in a very elaborate folder hierarchy. Exporting Mbox files would take a long time. I would like to repair and salvage my current database file, if at all possible.

  • You will need to rebuild your database. The symptom of messages showing in the junk mail folder is a sign of database corruption. Since the junk folder is a default folder, there is no way other than to rebuild.
    Because your database is so large, you need LOTS of free drive space. I recommend 500GB free.