This happens quite a lot in programming I find. There’s the ideal way of doing something and the way you actually end up doing it, be it due to buggy libraries, quirks of code, time or inexperience. I found one earlier today that had caught me out before. Modern cameras and images can have the orientation of the picture embedded. For example older cameras held normally may produce an image that is (widthxheight) 1024×768 and vertically an image of 768×1024. However a lot of the time now both images will internally be 1024×768 and have an orientation field (probably held in exif data I expect although in UIImage it’s a @!£%£@$^ read only property called imageOrientation) that informs you which way up the camera was when the shot was taken. Excellent. Not.
The problem with this occurs if you grab the raw image data to manipulate it and then convert it back. Say from a UIImage in cocoa/ukit whatever to an IplImage in opencv. What happens, if you had the camera held vertically, when you display your processed image is that it has rotated 90º ccw because you have not ported back the orientation data. That’s fine, copy the orientation data, then port it back. Except you can’t as apple in their infinite wisdom protect you from manipulating such things! I worked around this by avoiding having to manipulate my image that way within opencv. Many many others have worked around these problems by copying the orientation data and then using that to rotate their image before copying it back thus nullifying the need for orientation.
So to my specific problem. Opencv provides many many useful tools for discovering image data. It tends to present a lot of this data back to us as linked lists of points and the like. It also provides you with tools to draw said points in a meaningful way. One of which is cvDrawContours which takes a sequence of points and draws them using ‘pen’ information you provide (size, colour, etc.) onto your image. If this is done with a vertically taken iPhone camera image, copied to an IplImage, drawn on with cvDrawContours and then copied back to a UIImage to display your picture rotates 90º ccw. There are two potential solutions here:
- Write aforementioned code that takes the old orientation into account and rotates the image post manipulation.
- Rewrite the functions you need to to do any image drawing direct on your UIImage or Image object.
It might not be obvious but the former method is probably the worst of the two. It’s the quickest in terms of implementation. I think there are around 8 types of orientation of which 4 are mirrors so you only have to write code to rotate and/or mirror your image. However the other option is to simply doodle on your original using the data from opencv. Far more complicated for sure but it should be more efficient than rotating all those pixels about. I’ll find out when I’m done.
To do this I took the cvDrawImage code directly out of cxdrawing.cpp along with it’s helper functions and inline data structures in opencv and pasted it into my ImageView.m file. I renamed the function to uiDrawImage and altered the first parameter away from an IplImage to a UIImage. I then changed all function parameters away from a c++ style to an objective-c++ style. Of course it’s c++ so I had to rename the source file to ImageView.mm to get it compiled as objective-c++. No other changes needed except judicious deleting of functions that I shouldn’t have copied. The linker warnings gave them away so no problem there.
Now, a sequence of points in opencv is thankfully a doubly linked list with no tail and end. It’s effectively a circle of co-ordinates. This makes it easy to traverse and as it’s a list of structures that contain x&y floats or ints or whatever we should be able to rip out a lot of the code from cvDrawImage and get it so when it’s got the points it just adds them to a CGMutablePathRef. That’s effectively the same thing in cocoa/ukit whatever it is. After that there is a function provided that does the same thing as cvDrawImage but with a UIImage or Image. I’ve not looked that far yet.
If you hadn’t guessed this post is mostly laying out my current thinking on how to tackle this in the best way, not the way I’ll probably end up doing it. We’ll see.