Category: Chef’s Book


There are some interesting challenges with incorporating iCloud support, especially into an existing application. One, which I’ve been working through, is that you can no longer copy in a pre-existing store for the purposes of loading data. Such a move appears to work but does not upload anything to the cloud, the cloud store will only see future changes and not have the initial data set.
The solution to this is to use NSPersistentStoreCordinator’s migratePersistentStore:toURL:options:withType:error method. However I found the documentation somewhat lacking in clarity so here is the solution that now appears to be working for me.

There are some things I wanted to guarantee, which are probably not applicable to many so I’ll document them here:

- The application has gone through a few object model changes and I wanted to support people that may have skipped versions.
- The application’s data store location has changed on one occasion, when I moved the store out of Documents and into the Library directory due to adding iTunes document sharing.

Before we look at the code we’ll have a look at a rough outline of the procedure, in handy bullet points, step is 4 is the one that took some figuring out:

  1. Run everything following this asynchronously.
  2. Determine location of any existing stores, if not default to initial data set.
  3. Check to see if application supports iCloud, e.g. is running iOS 5.
  4. Configure store options if it does support iCloud.
  5. Lock the persistent store coordinator.
  6. If there is a cloud data set already, simply add it.
  7. If there isn’t DON’T add it, the migrate function adds it.
  8. Unlock persistent store coordinator.
  9. Send messages out so that view controllers that need to can redo their fetches.

A few caveats about the following, it’s not tested aside from ensuring it does migrate data into the store. I’ve not yet verified that the data then careers across the cloud to another device but I’ll update the post as soon as I do.

The code isn’t optimal, but that should be obvious! ;) The method should be called where in your code you’d normally add the store. A lot of the framework, minus the migrations stuff, is based on some example iCloud code.
Also note that the if block with the “//Migrate old store to new AND migrate into cloud” comment is devoid of code. In that block I would need to update the old store to match the new one and THEN migrate the store as I’m using my own migrations code which also allows importing of new data and a few other tweaks, which I think I’ve previously blogged.

CLOUD_NAME,STORE_NAME are #defines. DLog is a #define macro to NSLog that is blank in production environments.

- (void)aSynchronouslyAddPersistentStore {
  NSString *cloudPath = [[self applicationLibraryDirectory] stringByAppendingPathComponent:CLOUD_NAME];
  NSString *preCloudPath = [[self applicationLibraryDirectory] stringByAppendingPathComponent:STORE_NAME];
  NSString *defaultStorePath = [[NSBundle mainBundle]
                                pathForResource:@"InitialData" ofType:@"sqlite"];
  NSString *oldStorePath = [[self applicationDocumentsDirectory] stringByAppendingPathComponent:STORE_NAME];

  // do this asynchronously since if this is the first time this particular device is syncing with preexisting
  // iCloud content it may take a long long time to download
  dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
    NSFileManager *fileManager = [NSFileManager defaultManager];
    NSURL *storeUrl = [NSURL fileURLWithPath:cloudPath];
    NSURL *migrateUrl = nil;

    /**
     Find out which store has the data we need to migrate,
     if any.
     */
    if (![fileManager fileExistsAtPath:cloudPath]) {
      if ([fileManager fileExistsAtPath:preCloudPath]) {
        // Migrate old application data.
        migrateUrl = [NSURL fileURLWithPath:preCloudPath];
      } else if ([fileManager fileExistsAtPath:oldStorePath]) {
        // Migrate old store to new AND migrate into cloud.
      } else if ([fileManager fileExistsAtPath:defaultStorePath]) {
        // Migrate (copy) in initial recipe data set.
        migrateUrl = [NSURL fileURLWithPath:defaultStorePath];
      }
    }

    // this needs to match the entitlements and provisioning profile
    NSURL *cloudURL = nil;
    if([fileManager respondsToSelector:@selector(URLForUbiquityContainerIdentifier:)])
    {
      cloudURL = [fileManager URLForUbiquityContainerIdentifier:nil];
      NSString* coreDataCloudContent = [[cloudURL path] stringByAppendingPathComponent:@"chefsbook_v14b3"];
      cloudURL = [NSURL fileURLWithPath:coreDataCloudContent];
      DLog(@"cloudURL : %@", cloudURL);
    }

    //  The API to turn on Core Data iCloud support here.
    NSDictionary *options = nil;
    if (cloudURL) {
      options = [NSDictionary dictionaryWithObjectsAndKeys:
                 @"com.thelostsouls.chefsbook", NSPersistentStoreUbiquitousContentNameKey,
                 cloudURL, NSPersistentStoreUbiquitousContentURLKey,
                 [NSNumber numberWithBool:YES], NSMigratePersistentStoresAutomaticallyOption,
                 [NSNumber numberWithBool:YES], NSInferMappingModelAutomaticallyOption
                 ,nil];
    } 

    NSError *error = nil;
    [persistentStoreCoordinator_ lock];
    DLog(@"Persistent Store ****LOCKED****");

    /**
     If I migrate url is set don't just add, migrate!
     Otherwise proceed as planned.
     */
    if (migrateUrl) {
      NSPersistentStore *srcPS = [persistentStoreCoordinator_ addPersistentStoreWithType:NSSQLiteStoreType
                                                configuration:nil
                                                          URL:migrateUrl
                                                      options:nil
                                                        error:&error];
      if (![persistentStoreCoordinator_ migratePersistentStore:srcPS
                                                         toURL:storeUrl
                                                       options:options
                                                      withType:NSSQLiteStoreType
                                                         error:&error]) {
        DLog(@"Error migrating data: %@, %@", error, [error userInfo]);
        abort();
      }
    }
    else
    {
      if (![persistentStoreCoordinator_ addPersistentStoreWithType:NSSQLiteStoreType
                                                     configuration:nil
                                                               URL:storeUrl
                                                           options:options
                                                             error:&error]) {
        DLog(@"Error adding persistent store: %@, %@", error, [error userInfo]);
        abort();
      }
    }
    [persistentStoreCoordinator_ unlock];
    DLog(@"Persistent Store ****UNLOCKED****");

    // tell the UI on the main thread we finally added the store and then
    // post a custom notification to make your views do whatever they need to such as tell their
    // NSFetchedResultsController to -performFetch again now there is a real store
    dispatch_async(dispatch_get_main_queue(), ^{
      DLog(@"asynchronously added persistent store!");
      [[NSNotificationCenter defaultCenter]
       postNotificationName:AppDelegateSharedPSCAddedStore
       object:self userInfo:nil];
    });
  });

}

So there you go.

Still a worry are scenarios where iCloud support is enabled/disabled and/or the data deleted from the cloud via the settings menu. Hopefully nobody would do that and leave iCloud enabled or they’d end up with an empty application and no recovery. That’s also the reason I’m still supporting and improving the code that saves the docs out to iTunes.

Another thought just occurred to me. The following scenario needs serious consideration:

  1. User starts app with iCloud support, initial data set loaded.
  2. User proceeds to add a handful of data.
  3. User installs Chef’s Book on second device with iCloud enabled for app.
  4. App starts and copies in default data PLUS receiving iCloud data.

Will it fail, magically know it’s the same (eminently probable), or duplicate data?

Time to find out!

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Well I found out, it hangs permanently. Thanks to some helpful people on the apple forums I know why. So the above code is only partially correct. What needs to be done is to also query data on the cloud using NSMetadataQuery and examine the results, if there is cloud data you should use that and not add a persistent store in the normal way (I guess?). What I now need to find out, having successfully been able to query the cloud and find LOTS of data on there from my previous tests, is how you should add that data.

Then I’m going to rewrite the entire core data start up code from the bottom up.

The MD5 checksum bug identified.

Happily it’s easy to fix and exactly what I thought it was. When retrieving values from an NSManaged object like so:

NSString *foo = recipe.value;

or

NSString *foo = [recipe valueForKey:@"value"];

If the value is not present you get a nil. Messages to nil go nowhere but do not crash, so if value should have been an NSString the following is OK:

[foo isEqualToString:@"bar"]

However if the object you are retrieving from is an NSDictionary you will get an NSNull. NSNull is a class and a valid object so sending it messages it does not understand will fail:

NSString *foo = [dictionary valueForKey:@"value"];
[foo isEqualToString:@"bar"]; // CRASH

Seeing as I need to compute the md5 sum for a dictionary based recipe and an managed object based one and the sums need to be the same the code that produces the string to hash must produce identical results. In the case of a null value the code to produce the key to hash from the recipe managed object inserted this:

(null)

The code to produce it from the dictionary simply crashed. So I changed it in both the dictionary and the managed object routines to check:

if ([object isEqualToClass:[NSNull class]]) {
object = @"";
}

Which meant the keys changed and I didn’t update the initial data set to reflect that as I didn’t run OCUnit and thus didn’t get bashed upside the head that something had changed.

Always run your unit tests folks.

So the latest version of my main app is in the app store and…. it has a minor minor bug.  In a rush to eliminate a highly unlikely, in fact to pre-empt, a bug I updated some code to check for the possibility of NSNull being assigned to an NSString.  For some reason this changed the md5 sum assigned to two recipes (Shepherd’s Pie and Seafood Linguine Carbonara) which means they will be duplicated for every user I expect.  Shame.  Not life ending but annoying.  I submitted the update metadata around 21:20 on Friday, the app at around 21:57.  I fixed the bug at 21:45.  I didn’t run the OCUnit tests which would have spotted the md5 checksum changing when it should not have.

Easy to fix though and on the upside all of this code and unit tests means development is much faster now.  So I’m well on the way to iTunes document synchronisation support.  I’ve had to relocate a plist file and the sqlite database out of documents and into library otherwise they show up in iTunes.  So all you people that write core data tutorials putting sqlite in the documents folder… stop it!  It’s not a good idea use NSLibraryDirectory instead of NSDocumentDirectory when calling NSSearchPathForDirectoriesInDomains.

On another note, to determine what files were new in the documents folder I thought I’d use file info, like dates etc.  Figuring that when the file is written to the iPhone by iTunes it would write a new file but it doesn’t… it preserves the host machines modified and creation dates:

      NSDictionary *fileInfo = [[NSFileManager defaultManager] attributesOfItemAtPath:filePath
                                                                                error:nil];
      NSDate *lastModified = [fileInfo valueForKey:NSFileModificationDate];

(gdb) po fileInfo
{
    NSFileCreationDate = "2011-05-09 11:30:53 +0000";
    NSFileExtensionHidden = 0;
    NSFileGroupOwnerAccountID = 501;
    NSFileGroupOwnerAccountName = mobile;
    NSFileModificationDate = "2011-05-09 11:30:53 +0000";
    NSFileOwnerAccountID = 501;
    NSFileOwnerAccountName = mobile;
    NSFilePosixPermissions = 420;
    NSFileProtectionKey = NSFileProtectionNone;
    NSFileReferenceCount = 1;
    NSFileSize = 1788192;
    NSFileSystemFileNumber = 2608135;
    NSFileSystemNumber = 234881027;
    NSFileType = NSFileTypeRegular;
}

A shame but totally understandable as the file is not being modified or created as such. Looks like I’ll need to track a bit more metadata myself or waste time import/exporting everything every time. Err… no.

It’s not brilliantly efficient but it does reuse existing importing code and it does work reliably and well.  If I update the app to include new recipes I only want to include them once.  When the app is first launched a preloaded Core Data sqlite file is copied into the Documents directory.  To provide new recipes to existing users I could have included them as a set of chefdoc files to import from the applications bundle but that would be wasteful as they’d already be in the preloaded file.  So, expanding a bit on the previous use of automatic migrations we can use an NSEntityMigrationPolicy just like before, it happens that I have added more information to the DB so I needed one anyway.  Alongside a routine much like the one used previously we can add the following routine:

- (BOOL)endEntityMapping:(NSEntityMapping *)mapping
                 manager:(NSMigrationManager *)manager
                   error:(NSError **)error {
    if (![super endEntityMapping:mapping manager:manager error:error]) return NO;

    [self addNewRecipes:manager];

    return YES;
}

Inside the addNewRecipes routine we first grab our own NSPersistentStoreCoordinator and NSManagedObjectContexts:

    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
    NSError *error = nil;
    NSString *modelPath = [[NSBundle mainBundle] pathForResource:@"Chefs_Book" ofType:@"momd"];
    NSURL *modelURL = [NSURL fileURLWithPath:modelPath];
    NSManagedObjectModel *managedObjectModel = [[NSManagedObjectModel alloc]
                                                initWithContentsOfURL:modelURL];    

    NSPersistentStoreCoordinator *persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:managedObjectModel];
    NSString *initialDataPath = [[NSBundle mainBundle]
                                 pathForResource:@"InitialData" ofType:@"sqlite"];
    NSURL *initialStoreURL = [NSURL fileURLWithPath:initialDataPath];
    [persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType
                                             configuration:nil
                                                       URL:initialStoreURL
                                                   options:nil
                                                     error:&error];

    NSManagedObjectContext *context = [[NSManagedObjectContext alloc] init];
    [context setPersistentStoreCoordinator:persistentStoreCoordinator];

Don’t forget to drain the pool at the end of the above! We can then set about iterating through the Recipes. I simply build a list of unique ID’s from our preloaded db, compare it to the users database using the passed in NSMigrationManager to reference the destination context like so:

    NSArray *destRecipeUUIDs = [KookaDIS allRecipeUUID:[manager destinationContext]];

Once I’ve got a list of which ones are missing I convert them into the same format that’s used for emailing then use the same routines to import the data that are used when somebody opens a file in the email application or in safari. It’s not the most efficient as images are converted and base64 encoded and decoded etc. but it has two major advantages:

- The routines are proven and reliable.
- I only have one set of bulk import/export routines to maintain should my needs change in the future.

Having said that, the routines involved are recursive, data independent and obey a set of rules passed in to avoid looping recursion so not many data base schema changes will make it throw a fit.

That is the last big bit of code done for the update. To do now is the progress bar code for data updating/import, some polish (simple UI changes like hiding icons instead of deactivating them to fall more inline with apples UI recommendations) then test it for memory likes and finally do end to end tests on hardware. Probably another week so perhaps two weeks until the update is in the store.

Ownership of user created content.

I want to implement some concept of ownership/authorship in Chef’s Book.  Not for any reason of control but as a way of letting users swap recipes without data loss or unnecessary duplicates.  The problem is if I email a recipe to myself and load it on the iPad I get a new recipe.  That was fine for launch but now I want to keep the iPad copy up to date with changes I make on the iPhone (it has the camera!) so I don’t want duplicates.  That’s easy.. if the recipe exists on my iPad replace it.  That’s fine, and that’s what will happen, but what if I’ve sent it to Bob and Bob has made changes?  Bob would be pissed if he lost them when I sent him a new version with more milk and less photos.

To solve this each user needs to be identified and thus if Bob makes changes to a recipe he’s got then he becomes the owner of that recipe, in fact it becomes a new recipe.  This way if I send an updated version of my recipe to Bob he’ll get it as a new recipe, if Bob doesn’t make changes then his recipe will be updated if necessary.

So far, so simple.  Any change to a recipe that you do not own will make you the owner and generate a new unique identifier.  Any change to the title of a recipe you own will also generate a new unique identifier such that I can make chocolate cake and send it out, edit the dish to become a spanish omelette and send that out too.  Peoples chocolate cakes won’t change into spanish omelettes overnight then.

Good good… what about if I edit the recipe on my iPad and send it back to the iPhone?  Ah.. without a centralised system my iPad will grab ownership of the recipe after changes and make it a new recipe, sending it back to the iPhone will therefore import it anew.  That’s also fine, but not ideal.

Such centralised behaviour is something I want to do but it wasn’t something I planned to have to do with this update.  It was something that would be necessary for uploading recipes to a server for others to search and download.  It would allow them to receive updates to recipes they have downloaded but not altered.  Of course that behaviour isn’t cast in stone, I would probably store the ID’s of downloaded recipes against the users profile to notify them of updates.  That allows for some flexibility too, users can modify recipes to their hearts content and yet still get poked if a new version is available.  Which would then download as a fresh copy.

So one part of me has had a flippant and amusing idea.  If it’s possible to grab some unique info back from Game Center would that be a good thing to implement?  It’s not a game so how intrusive would it be?  If I felt truly silly I could bung in some achievements too.  Part of me wants to do that just because it’s daft and amusing.

It has to be practical though.  So I’ll have to check on a few things:

  • Is there an ID I can safely store from game center.
  • Do Apple mind if it’s used for non gaming applications?  I know of a To Do list dressed up as a game that uses it.
  • How unobtrusive can it be made?

Of course the ideal solution is to code something quickly that just provides the basics of authentication using OpenID.  That would get the foundations laid for the future which needs to be done anyway and would not exclude any flippant future use of game center.

Whilst I ponder this I’ll just implement the flow I’ve got so I can test it extensively.  One thing I want to check is the speed with which I can do a deep comparison of a large recipe incoming as a chefdoc with a present recipe NSManagedObject hierarchy.  I want to know how long it would take to determine whether even an image in a recipe has been modified.  For example someone taking an existing received recipe and updating it with their own photos.

Application Accepted!

Well that was quite smooth, the application is now in the App Store Right Here!. In other news it appears I’d completely forgotten in my rush to get the site live to include LV partitions for the site and it’s backup. Cue me completely forgetting the LVM commands and having to have a quick refresh on tldp.org. Oh how things have changed so fast! A few weeks ago I stumbled briefly with grant privs in MySQL!

Powered by WordPress | Theme: KLG based on Motion by 85ideas.