iPhone 4 Battery Life Issue Falls Squarely on the iCloud service [Solved!]

After spending a while continuing to track down the issues I brought up in my last post, I was able to finally track down the source of my battery drain: the iCloud service could not handle my large contact list.  I went ahead and turned off the iCloud service on my iPhone completely and did, in fact, dramatically (nearly 2x) increase my battery life.  However, that's a weak technical solution, so I decided to dive just a bit deeper to find out exactly what was going on.  Before I get into the crazy details, let me offer a few real recommendations on how you should respond if your iPhone is seeing a big drop in battery life after an iOS 5 upgrade:

iPhone Battery Life Checklist

  1. If your phone feels HOT (higher than when you were using iOS 4) after sitting in standby:
    1. I'd bet you've got a iCloud issue in front of you.
      1. HOWTO: Open the settings app, select the iCloud option, and then turn off each service.  Then restart your device.
      2. NOTE:  If you really want to use the iCloud service, you should use the strategy I describe below and you can at least target the problem.  Honestly, the solution for most people may be to actually do the hard restore (i.e., back to factory defaults) IF you can isolate an iCloud problem.


    2. If you discover the iCloud service is not the problem (tell me!), you've got three more major options:

      1. HOWTO: Reset your network settings
        1. Settings app -> General -> Reset -> Reset Network Settings
        2. Then restart your device.
      2. HOWTO: Turn Location Services off
        1. Settings app -> Location Services -> Location Services = OFF
        2. Restart your device.
      3. HOWTO: Reset e-mail accounts (especially exchange accounts, apparently):
        1. Delete-mail accounts (Settings app -> Mail, Contacts, Calendars -> Account (select one) -> Delete Account).
        2. Restart your device.
        3. Add e-mail accounts back (Settings app -> Mail, Contacts, Calendars -> Add Account...)


  2. If your seeing battery life problems after iOS 5 and you've not got a hot phone, I'm not sure what's wrong with your phone, but you should attempt to follow the options Apple provides and see if you see an improvement.  Other people offer options too.

Honestly, if you upgraded to iOS 5 and one of these issues isn't causing your battery life to be degraded, you may be having another problem that's pretty unique.

Identifying Battery Drain Root Cause

I went ahead and checked my hypothesis from my previous post and identified the iCloud service as the source of my battery drain issues.  Turning off the service increased my battery life by north of 2 times my post-upgrade battery life.  Unfortunately, I kind of want to use this service (despite all the problems I've seen with it so far) and so I had to dive a bit deeper into the process.  Things are going to start getting technical, but I'll describe how to jailbreak your phone (using redsn0w 0.9.9b4), a method of manually dealing with your contacts using the sqlite contact databases on the phone, and a solution to iCloud churn around contacts.  It'll be fun (if you're an enormous nerd).  Here goes.

Disclaimer and Backups

DISCLAIMER: I don't know what the appropriate way to put this is, but let me keep it simple: anything you do here will probably cause the earth to collapse around you.  Don't do it and certainly don't blame me, but if you have questions or need help, I'd be happy to help.

BACKUPS:  You need to backup everything.  Don't go forward unless you are certain you can afford to lose everything on your phone.  Trust me, I lost stuff on my phone while playing with this and had to restore using backups during different stages of this process.

Jailbreak iOS 5 on iPhone 4 using redsn0w 0.9.9b4

After you're up and running with your jailbreak, you essentially want to start debugging your iPhone using a set of unix tools -- in this case, I got lucky and was able to individually turn on and off components of the iCloud service (which I previously identified as the source of my battery drain issues) and measure very specific CPU utilization using top.  I'll describe this in detail next.

[caption id="attachment_110" align="alignright" width="216"]Redsn0w Extras Screen Select the IPSW file or you're not going to get this to work![/caption]

  1. I'm a Windows user, so I picked up redsn0w 0.9.9b4, unzipped it, and ran the redsn0w application.
  2. You will want to click the "Extras" button on the main page and then click the "Select IPSW" button on the next screen.  You want to navigate to %appdir%RoamingApple ComputeriTunesiPhone Software Updates and select the iPhone3,1_5.0_9A334_Restore.ipsw file and you're set.
  3. Head back to the main screen and click jail break and follow the direction.  You should be good to go.
  • NOTE:  If you turn on your phone and you end up with a white Cydia icon and Cydia crashes immediately after starting, you simply need to turn off your phone, make sure it's tethered to your machine, and then click the "Just boot" component in the redsn0w extras (the image on the right shows the button).

Enable Unix Tools on iPhone and Debug!

I'm sure this isn't a surprise to people that have made it this far, but the next step here is to enable OpenSSH and install a set of unix tools through Cydia.  Cydia works the  magic for you, so you shouldn't worry too much about this (though there is a bug in lsof that remains).  For the unix tools, you'd be smart to select and install the BigBoss Recommended Tools.

After all of this was in place for me, I went ahead and assessed the specific problem I was having with my iPhone.  Among a set of tools, top ended up being the easiest to measure the iCloud issue.  While I was syncing my iPhone Contacts, the iCloud service was churning doing some kind of data access (how can this be even reasonably efficient, ugh.).

[caption id="attachment_111" align="alignnone" width="591"]Problematic iPhone CPU Usage Check out the CPU utilization and the CPU time -- something seems wrong.[/caption]

After turning off the contacts syncing, the iCloud service settled into a reasonable background process with low CPU, Memory, and Bandwidth utilization.  So, I hunted down the iPhone contact database (/private/var/mobile/Library/AddressBook) and noticed the files were absolutely huge.  Something here was clearly wrong.

[caption id="attachment_118" align="alignnone" width="645"]iPhone AddressBook Files Accessible (awesome!) sqlite databases that hold my contacts are too large here.[/caption]

So, the debugging here wasn't too hard.   The hard part follows on how I was able to figure out why the phone was churning (why this IS NOT okay) and how I fixed the problem.

Isolating the iPhone Contacts Problem

So, the files in the AddressBook directory were all sqlite databases.  This is not going to be a SQL introduction, but to access these databases, you basically need a sqlite client (sqlite3 was provided by the BigBoss package I installed earlier) and through a few select statements here and there, it's clear the Facebook application (probably in sync with either Google or MobileMe) were responsible for creating a huge amount of duplicated contacts and associated images.  For whatever reason, the iCloud service simply couldn't get this information into the cloud (though the actual size of the files it would have to move are on the order of hundreds of megabytes in size).  There is, apparently, also a tool that the phone has to notice this kind of duplication, but it was unfortunately not being used.  In total, it appeared I had several thousand contacts (which, for someone important might not be unusual).  Either way, the iCloud service choked on all of this (which is unreasonable.).

The solution was to go ahead and cut out all of the duplicate information in four tables in both the AddressBook.sqlitedb and AddressBookImages.sqlitedb databases.

  1. ABPerson
  2. ABMultiValue,
  3. ABFullSizeImage
  4. ABThumbnailImage

NOTE: If you are doing and are using groups, you may need to adjust your group information too.  The table names are ABGroup and ABGroupMembers.  If you've made it this far, chance are you're going to be able to figure this out, but feel free to ask me for help if you need it.

THINGS TO DO FIRST!

[caption id="attachment_119" align="alignright" width="202"]iCloud Empty Contacts Screenshot

[/caption]

  1. Turn off EVERY application on your phone.  The contacts app, the phone app, the SMS app, etc. should all be closed on your phone.  Look at your phone and make sure!
  2. Turn off ALL of the iCloud services.  It had better not be syncing contacts while you're doing messy things with the database it's using.
  3. Delete ALL of your iCloud contacts (this takes time and is buggy as hell -- the iCloud service was rushed to market, ugh.).  Make sure you end up with a screen that looks like the one on the right.  If not, you're going to lose your contacts when you first sync.  You will have to restore from backup.
  4. Create a backup of your AddressBook folder (I had to use this multiple times while messing around) or you're going to lose your contacts.  Run the following command to do this:
cd /private/var/mobile/Library/AddressBook  
tar -cf ~/backup.tar *sqlite*
NOTE: If you end up needing to restore from backup (you mess up a sql statement, for example), you will want to run the following command:
cd /private/var/mobile/Library/AddressBook  
tar -xf ~/backup.tar

Deleting Duplicate Contacts [Problem Solved!]

The final step is actually not that hard (it takes a while to get all the SQL working like magic especially if you don't have a good idea about the sqlite nuances.  The short and fast is that you basically open the sqlite3 in a console and execute each of the SQL commands I've written and you'll clear out duplicate contacts in a heartbeat. The first step is to clear out duplicate entries in the AddressBook database and leave the images database for another entry.  The following commands will do that when run in the sqlite3 console:
DELETE FROM ABPerson WHERE ROWID NOT IN (SELECT MIN(ROWID) AS ROWID FROM ABPerson AS SaveMe GROUP BY First,Last);  
DELETE FROM ABMultiValue WHERE record_id NOT IN (SELECT ROWID FROM ABPerson);  
DELETE FROM ABMultiValue WHERE UID NOT IN (SELECT UID FROM ABMultiValue AS SaveMe GROUP BY record_id,property,value);

Starting up the sqlite3 console looks like the image below.  In particular, you want to make sure you run "sqlite3 AddressBook.sqlitedb" and that will make sure you are using the appropriate database.

SQLite3 Console STartup

After you've cleared out the duplicates in your AddressBook.sqlitedb database, you will want to run the commands below for the AddressBookImages.sqlitedb database.

ATTACH 'AddressBook.sqlitedb' as AddressBook;  
DELETE FROM ABFullSizeImage WHERE record_id NOT IN (SELECT ROWID FROM 'AddressBook'.ABPerson);  
DELETE FROM ABThumbnailImage WHERE record_id NOT IN (SELECT ROWID FROM 'AddressBook'.ABPerson);

And that's that.  You can turn on your iCloud contacts sync and you will see the contacts show up again like magic and your problems will be solved

Conclusion

And that's that. My iPhone was getting slammed by the iCloud service as it was trying to sync my contacts. There was no warning, there was no attempt to mitigate the problem, and the solution was absolutely impossible. There has to be a better way for Apple to provide a service like this -- I can imagine people who do actually have lots of contacts and probably just suffer through a poor experience. Oh well. Done!