It’s a heck of a lot faster than my code, the canny/sobel operators especially. Also it works. See photos for more.
The first is the original with 5×5 blur to reduce the noise, the second is adaptive threshold and the third is a canny edge detection. Takes about a 1/100th of the time to run on my netbook compared to image magick and my own stuff. Awesome.
Save more often than you would in a hardcore FPS. FFS. With other choice 4 letter words and 3 letter acronyms. I just lost almost 2 hours work patching up the awful perspective and scale on this sketch I did ages ago:
Sketch of a girl climbing.
Based on one done by Hawk around chrimbo time of Applegeeks fame.
In other news APB is awesome, despite me dying more times than a villain in hammer horror.
Still haven’t quite got the hang of the MagickWand/MagickCore stuff so I’ve been fiddling on the command line so I can get the best sequence of operators together. I’ve found one. A few actually. If I smooth the image (morphology open -> close) it obliterates numbers writen by hand but preserves the printed ones. Which is amusing. This however is great, it’s a skeletonisation, repeated thinning until no more can be done:
Morphological Skeletonisation
If you look closely you can still make it all out. 😛
It turns out these features must be new because they’re not in the ubuntu binaries or source. So here comes a quick guide to compiling image magick under ubuntu.
Continue reading
So err… I guess I’ve learnt a lot writing this crap on my own, and I am at least comfortable in the knowledge that when my routines DO work they are faster but I just found this:
MagickMorphologyImage
MagickMorphologyImage() applies a user supplied kernel to the image according to the given mophology method.
The format of the MagickMorphologyImage method is:
MagickBooleanType MagickMorphologyImage(MagickWand *wand,
MorphologyMethod method,const ssize_t iterations,KernelInfo *kernel)
MagickBooleanType MagickMorphologyImageChannel(MagickWand *wand,
ChannelType channel,MorphologyMethod method,const ssize_t iterations,
KernelInfo *kernel)
Shite. That’s everything right there. Corner detection in theory but definitely thinning, my routine for excessive thinning is called starving. :), and thickening. I’m pretty sure I can also use it to do RIP stuff. So new workflow could be something like:
SigmaContrast(via. UpdatePixelViewIterator)->MagickAdaptiveThresholdImage->MagickQuantizeImage->MagickEvaluateImage(Negatise)->MagickMorphologyImage(RIP)->MagickEvaluateImage(Negatise)->MagickMorphologyImage(Thinning/Starving)->MagickMorphologyImage(CornerDetect)
Hmm… First things first, replace my code with theirs to see what the detectable differences are.
I’ve hit somewhat of a brick wall. I managed to get some morphological hit&miss code that thinned the images down to 1px wide and it worked fine on the previously posted test. However after that I started getting horrible loops and things attached to the digits ruining all chance of OCR from them. Coupled with the fact that additional photos taken with an actual iPhone camera as opposed to my Lumix don’t come out too well it’s a bit of a dead loss right now. On the plus side I have implemented my own code to write out bitmap files so once I’ve got my thresholded quantized image I’m free of MagickWand. Potentially I can implement the magic wand stuff myself now too and do away with it completely. However these problems are in dire need of solving:
1) Identifying some criteria upon which to threshhold and image (currently what radius to use in the process). Minor changes make the world of difference although 40-45px is still somewhat optimal.
2) Some way of RIPing the image. (Remove Isolated Pixels).
3) Some way of perhaps filling in the obvious holes, morphological thickening sucks.
4) A better approach to thinning, perhaps, if 2&3 don’t help with the current method.
5) A corner detection algorithm that works and isn’t either: (c)Dera/QinetiQ, Some Student or Patented and aww screw let’s face it, I need to do my own.
On the plus side a trip to the Central Resources Library for Hertfordshire has yielded useful material.
So this is a section of a sudoku puzzle as the code sees it:
A small section of a Sudoku converting 1’s and 0’s.
It’s a screen cap of some debug output put into GEdit with a small font. It’s just 1’s and 0’s. A few interesting things here apart from the noise around the lower left of 8 is that it’s clearly obvious which numbers were the ones printed in the paper, the 2 and 3, along with the fact that the perfect horizontal lines are visibly distorted compared to the verticals. This is entirely down to the angle that I suspect most people will photograph things from unless they try for an exactly overhead shot. Now to test my detection routines.
EDIT@22:14 I take it back. The data was getting borked on exit from the function. Turns out that the code in use had RETURN() as a macro which freed the pointer I was using when I called RETURN(TRUE). Haha…. still doesn’t answer my original question below though.
/EDIT
I thought they were essentially the same. A pointer to a pointer is the address of an address and surely the address of a pointer is an address of an address? I’ve clearly forgotten something fundamental here so on the off chance somebody can help. I’ve done this a few times now and fixed it without really thinking what I’m doing. This time however it caught me out. For the whole of today I’ve been fiddling with one routine only to find a final problem when I’d hashed out the algorithm to use. I have this defined as a function:
bool read_bitmap_file_data(unsigned char *cstream, unsigned int *width, unsigned int *height, unsigned char **data);
It used to take a file handle but it’s easier to bung it a stream of characters as variable cstream. data is what I receive back. A pointer to a pointer. Within the function it goes through and parses the stream adding bits to data as follows:
unsigned char *ptr;
int bytes;
for (bytes = 0, ptr = bits; bytes < size; bytes++, ptr++) { if ((value = next_int (&p_cstream)) < 0) RETURN (FALSE); *ptr=value; }
The above snippet is actually stolen from an old file called xbm.c (google it). After that loop it does this assignment if bits has data:
*data = bits;
If I define the variable passed to it as a pointer to a pointer like so:
unsigned char **hex_sudoku;
read_bitmap_file_data(xbm_sudoku,&width,&height,hex_sudoku);
It segfaults on the assignment mentioned above. If I declare the parameter and call the function like so:
unsigned char *hex_sudoku;
read_bitmap_file_data(xbm_sudoku,&width,&height,&hex_sudoku);
it is fine. In both instances the contents of the variable bits is identical so it is something about declaring a pointer to a pointer and just passing it as opposed to declaring a pointer and passing the address. What am I actually passing in the first instance if not an address?
Ah well... it all works now and I have not exactly wasted my time here.
OK so I’ve praised MagicWand now I feel I can curse it. I want a 1bit BMP out of the thing. A bitmap is easy, it’s smart. I write something ending in jpg it writes a jpg, I want a bmp it writes one. However it was writing 24bit images. If I want to faff with it on a binary level myself I need as little data as humanly possible. So I call all sorts of functions to set the bit depth and such like:
MagickSetImageDepth(contrast_wand,8);
or
MagickSetType(contrast_wand,GrayscaleType);
or
MagickMANYOTHERS
None work. I get the same 24bit image. So I finally thought fine… I’ll write my own routing to strip everything out and call it as a function:
static MagickBooleanType FadeToBlack(PixelView *black_view, void *context)
{
MagickPixelPacket pixel;
PixelWand **pixels;
register long x;
pixels=GetPixelViewPixels(black_view);
for (x=0; x < (long) GetPixelViewWidth(black_view); x++)
{
/* Do magic */
PixelGetMagickColor(pixels[x],&pixel);
pixel.red=0;
pixel.green=0;
pixel.blue=0;
PixelSetMagickColor(pixels[x],&pixel);
}
}
--SNIP--
black_view=NewPixelView(contrast_wand);
if(black_view == (PixelView *)NULL)
ThrowWandException(contrast_wand);
status=UpdatePixelViewIterator(black_view,FadeToBlack,(void*) NULL);
if(status == MagickFalse)
ThrowWandException(contrast_wand);
black_view=DestroyPixelView(black_view);
Well you can guess how that ended up… with a black image (I thought wrongly being CMYK there might be a pixel.black. The above routine would have worked with a bit of analysis in it. Still with a 24bit image though. So I thought… well WWPD? (Photoshop) Quantize! So it turns out it’s one line:
/* Quantize image to 2? Please! */
MagickQuantizeImage(contrast_wand,2,GRAYColorspace,0,MagickFalse,MagickFalse);
And I get my precious:
diziet@ono-sendai:~/Programming/C/OCRSudoku$ file sudoku2bt50.bmp
sudoku2bt50.bmp: PC bitmap, Windows 3.x format, 800 x 739 x 1
I am hoping that on a binary level this is really simple to parse. I mean… It doesn’t have much data surely? 800×739 bits plus header? Nope sadly not. Ah well it’s only 73kB.
EDIT
Oh fucksocks, wish it’d dawned on me sooner. Isn’t the X11 xbm format err… ascii? Old school icons etc… So…..:
diziet@ono-sendai:~/Programming/C/OCRSudoku$ ./thresh-sigmoidal sudoku.jpg sudoku2bt50.xbm 50
diziet@ono-sendai:~/Programming/C/OCRSudoku$ file sudoku2bt50.xbm
sudoku2bt50.xbm: ASCII C program text
Did that say C? C? Hell I can do C!
#define sudoku2bt50_width 800
#define sudoku2bt50_height 739
static char sudoku2bt50_bits[] = {
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x30, 0xC3, 0xFC, 0xFF, 0x0F, 0x1C, 0x04, 0x00,
0x00, 0x00, 0xC0, 0xFB, 0x7F, 0x1F, 0x00, 0xC0, 0xE1, 0xFF, 0xFE, 0x00,
etc.
Oh baby! Right… time for more Modnations Racer or Final Fantasy ∞.
EDIT
OK I can load and display OK:
diziet@ono-sendai:~/Programming/C/OCRSudoku$ ./testxbm test_bits.xbm
1111111111111111
1000000000000001
1011111111111101
1010000000000101
1010111111110101
1010100000010101
1010101111010101
1010101001010101
1010101001010101
1010101111010101
1010100000010101
1010111111110101
1010000000000101
1011111111111101
1000000000000001
1111111111111111
Obv. it’s a bit messy to display the 800pixel sudoku:
diziet@ono-sendai:~/Programming/C/OCRSudoku$ ./testxbm sudoku2bt50.xbm |more
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000110011000011001111111111111111110000001110000010000000000000000000000000000000000011110111111111111011111000000000
00000000111000011111111111011111110000000011111111111100001111000011110000
etc. etc. etc. etc. etc. etc. etc. etc. etc. etc. etc. etc. etc. etc. for another 738 lines… It works! Now for the hard part.
