Tag Archive: migrations


There are some interesting challenges with incorporating iCloud support, especially into an existing application. One, which I’ve been working through, is that you can no longer copy in a pre-existing store for the purposes of loading data. Such a move appears to work but does not upload anything to the cloud, the cloud store will only see future changes and not have the initial data set.
The solution to this is to use NSPersistentStoreCordinator’s migratePersistentStore:toURL:options:withType:error method. However I found the documentation somewhat lacking in clarity so here is the solution that now appears to be working for me.

There are some things I wanted to guarantee, which are probably not applicable to many so I’ll document them here:

- The application has gone through a few object model changes and I wanted to support people that may have skipped versions.
- The application’s data store location has changed on one occasion, when I moved the store out of Documents and into the Library directory due to adding iTunes document sharing.

Before we look at the code we’ll have a look at a rough outline of the procedure, in handy bullet points, step is 4 is the one that took some figuring out:

  1. Run everything following this asynchronously.
  2. Determine location of any existing stores, if not default to initial data set.
  3. Check to see if application supports iCloud, e.g. is running iOS 5.
  4. Configure store options if it does support iCloud.
  5. Lock the persistent store coordinator.
  6. If there is a cloud data set already, simply add it.
  7. If there isn’t DON’T add it, the migrate function adds it.
  8. Unlock persistent store coordinator.
  9. Send messages out so that view controllers that need to can redo their fetches.

A few caveats about the following, it’s not tested aside from ensuring it does migrate data into the store. I’ve not yet verified that the data then careers across the cloud to another device but I’ll update the post as soon as I do.

The code isn’t optimal, but that should be obvious! ;) The method should be called where in your code you’d normally add the store. A lot of the framework, minus the migrations stuff, is based on some example iCloud code.
Also note that the if block with the “//Migrate old store to new AND migrate into cloud” comment is devoid of code. In that block I would need to update the old store to match the new one and THEN migrate the store as I’m using my own migrations code which also allows importing of new data and a few other tweaks, which I think I’ve previously blogged.

CLOUD_NAME,STORE_NAME are #defines. DLog is a #define macro to NSLog that is blank in production environments.

- (void)aSynchronouslyAddPersistentStore {
  NSString *cloudPath = [[self applicationLibraryDirectory] stringByAppendingPathComponent:CLOUD_NAME];
  NSString *preCloudPath = [[self applicationLibraryDirectory] stringByAppendingPathComponent:STORE_NAME];
  NSString *defaultStorePath = [[NSBundle mainBundle]
                                pathForResource:@"InitialData" ofType:@"sqlite"];
  NSString *oldStorePath = [[self applicationDocumentsDirectory] stringByAppendingPathComponent:STORE_NAME];

  // do this asynchronously since if this is the first time this particular device is syncing with preexisting
  // iCloud content it may take a long long time to download
  dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
    NSFileManager *fileManager = [NSFileManager defaultManager];
    NSURL *storeUrl = [NSURL fileURLWithPath:cloudPath];
    NSURL *migrateUrl = nil;

    /**
     Find out which store has the data we need to migrate,
     if any.
     */
    if (![fileManager fileExistsAtPath:cloudPath]) {
      if ([fileManager fileExistsAtPath:preCloudPath]) {
        // Migrate old application data.
        migrateUrl = [NSURL fileURLWithPath:preCloudPath];
      } else if ([fileManager fileExistsAtPath:oldStorePath]) {
        // Migrate old store to new AND migrate into cloud.
      } else if ([fileManager fileExistsAtPath:defaultStorePath]) {
        // Migrate (copy) in initial recipe data set.
        migrateUrl = [NSURL fileURLWithPath:defaultStorePath];
      }
    }

    // this needs to match the entitlements and provisioning profile
    NSURL *cloudURL = nil;
    if([fileManager respondsToSelector:@selector(URLForUbiquityContainerIdentifier:)])
    {
      cloudURL = [fileManager URLForUbiquityContainerIdentifier:nil];
      NSString* coreDataCloudContent = [[cloudURL path] stringByAppendingPathComponent:@"chefsbook_v14b3"];
      cloudURL = [NSURL fileURLWithPath:coreDataCloudContent];
      DLog(@"cloudURL : %@", cloudURL);
    }

    //  The API to turn on Core Data iCloud support here.
    NSDictionary *options = nil;
    if (cloudURL) {
      options = [NSDictionary dictionaryWithObjectsAndKeys:
                 @"com.thelostsouls.chefsbook", NSPersistentStoreUbiquitousContentNameKey,
                 cloudURL, NSPersistentStoreUbiquitousContentURLKey,
                 [NSNumber numberWithBool:YES], NSMigratePersistentStoresAutomaticallyOption,
                 [NSNumber numberWithBool:YES], NSInferMappingModelAutomaticallyOption
                 ,nil];
    } 

    NSError *error = nil;
    [persistentStoreCoordinator_ lock];
    DLog(@"Persistent Store ****LOCKED****");

    /**
     If I migrate url is set don't just add, migrate!
     Otherwise proceed as planned.
     */
    if (migrateUrl) {
      NSPersistentStore *srcPS = [persistentStoreCoordinator_ addPersistentStoreWithType:NSSQLiteStoreType
                                                configuration:nil
                                                          URL:migrateUrl
                                                      options:nil
                                                        error:&error];
      if (![persistentStoreCoordinator_ migratePersistentStore:srcPS
                                                         toURL:storeUrl
                                                       options:options
                                                      withType:NSSQLiteStoreType
                                                         error:&error]) {
        DLog(@"Error migrating data: %@, %@", error, [error userInfo]);
        abort();
      }
    }
    else
    {
      if (![persistentStoreCoordinator_ addPersistentStoreWithType:NSSQLiteStoreType
                                                     configuration:nil
                                                               URL:storeUrl
                                                           options:options
                                                             error:&error]) {
        DLog(@"Error adding persistent store: %@, %@", error, [error userInfo]);
        abort();
      }
    }
    [persistentStoreCoordinator_ unlock];
    DLog(@"Persistent Store ****UNLOCKED****");

    // tell the UI on the main thread we finally added the store and then
    // post a custom notification to make your views do whatever they need to such as tell their
    // NSFetchedResultsController to -performFetch again now there is a real store
    dispatch_async(dispatch_get_main_queue(), ^{
      DLog(@"asynchronously added persistent store!");
      [[NSNotificationCenter defaultCenter]
       postNotificationName:AppDelegateSharedPSCAddedStore
       object:self userInfo:nil];
    });
  });

}

So there you go.

Still a worry are scenarios where iCloud support is enabled/disabled and/or the data deleted from the cloud via the settings menu. Hopefully nobody would do that and leave iCloud enabled or they’d end up with an empty application and no recovery. That’s also the reason I’m still supporting and improving the code that saves the docs out to iTunes.

Another thought just occurred to me. The following scenario needs serious consideration:

  1. User starts app with iCloud support, initial data set loaded.
  2. User proceeds to add a handful of data.
  3. User installs Chef’s Book on second device with iCloud enabled for app.
  4. App starts and copies in default data PLUS receiving iCloud data.

Will it fail, magically know it’s the same (eminently probable), or duplicate data?

Time to find out!

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Well I found out, it hangs permanently. Thanks to some helpful people on the apple forums I know why. So the above code is only partially correct. What needs to be done is to also query data on the cloud using NSMetadataQuery and examine the results, if there is cloud data you should use that and not add a persistent store in the normal way (I guess?). What I now need to find out, having successfully been able to query the cloud and find LOTS of data on there from my previous tests, is how you should add that data.

Then I’m going to rewrite the entire core data start up code from the bottom up.

It seems that when you create a mapping model for core data migrations using XCode (3 or 4) it will not create mapping models for abstract entities. If you do not create this manually within your mapping model any data contained therein will be lost. In my case I needed to tie the image data back to the entity it belonged to. I could create several entities each with the requisite attributes but it made more sense to create one abstract image entity and have the others inherit from it. This enabled me to have a recipeimage and stepimage that could have relationships back to their parent without having to create lots of attributes. RecipeImage has Image as a parent and has a defined relationship with Recipe etc. The downside is that after migration if you do not create the Image mapping manually all you images vanish from the database.
I cannot fathom a good reason for this mapping not being created automatically, under what circumstances would you not want the data contained in an abstract entity migrated when you update the schema? At least it was easy to fix.

It’s not brilliantly efficient but it does reuse existing importing code and it does work reliably and well.  If I update the app to include new recipes I only want to include them once.  When the app is first launched a preloaded Core Data sqlite file is copied into the Documents directory.  To provide new recipes to existing users I could have included them as a set of chefdoc files to import from the applications bundle but that would be wasteful as they’d already be in the preloaded file.  So, expanding a bit on the previous use of automatic migrations we can use an NSEntityMigrationPolicy just like before, it happens that I have added more information to the DB so I needed one anyway.  Alongside a routine much like the one used previously we can add the following routine:

- (BOOL)endEntityMapping:(NSEntityMapping *)mapping
                 manager:(NSMigrationManager *)manager
                   error:(NSError **)error {
    if (![super endEntityMapping:mapping manager:manager error:error]) return NO;

    [self addNewRecipes:manager];

    return YES;
}

Inside the addNewRecipes routine we first grab our own NSPersistentStoreCoordinator and NSManagedObjectContexts:

    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
    NSError *error = nil;
    NSString *modelPath = [[NSBundle mainBundle] pathForResource:@"Chefs_Book" ofType:@"momd"];
    NSURL *modelURL = [NSURL fileURLWithPath:modelPath];
    NSManagedObjectModel *managedObjectModel = [[NSManagedObjectModel alloc]
                                                initWithContentsOfURL:modelURL];    

    NSPersistentStoreCoordinator *persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:managedObjectModel];
    NSString *initialDataPath = [[NSBundle mainBundle]
                                 pathForResource:@"InitialData" ofType:@"sqlite"];
    NSURL *initialStoreURL = [NSURL fileURLWithPath:initialDataPath];
    [persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType
                                             configuration:nil
                                                       URL:initialStoreURL
                                                   options:nil
                                                     error:&error];

    NSManagedObjectContext *context = [[NSManagedObjectContext alloc] init];
    [context setPersistentStoreCoordinator:persistentStoreCoordinator];

Don’t forget to drain the pool at the end of the above! We can then set about iterating through the Recipes. I simply build a list of unique ID’s from our preloaded db, compare it to the users database using the passed in NSMigrationManager to reference the destination context like so:

    NSArray *destRecipeUUIDs = [KookaDIS allRecipeUUID:[manager destinationContext]];

Once I’ve got a list of which ones are missing I convert them into the same format that’s used for emailing then use the same routines to import the data that are used when somebody opens a file in the email application or in safari. It’s not the most efficient as images are converted and base64 encoded and decoded etc. but it has two major advantages:

- The routines are proven and reliable.
- I only have one set of bulk import/export routines to maintain should my needs change in the future.

Having said that, the routines involved are recursive, data independent and obey a set of rules passed in to avoid looping recursion so not many data base schema changes will make it throw a fit.

That is the last big bit of code done for the update. To do now is the progress bar code for data updating/import, some polish (simple UI changes like hiding icons instead of deactivating them to fall more inline with apples UI recommendations) then test it for memory likes and finally do end to end tests on hardware. Probably another week so perhaps two weeks until the update is in the store.

Powered by WordPress | Theme: KLG based on Motion by 85ideas.