OpenCV digits merging into surrounding boxes
See the question and my original answer on StackOverflowHere is some code that seems to work quite well. There are two phases:
- One can observe that numbers are slightly bolder than boxes. Plus the whole image has strong horizontality. So we can apply a dilatation stronger horizontally to get rid of most vertical lines.
- At this point, OCRs, for example, Google's one, can detect most numbers. Unfortunately, it's somewhat too good and sees other stuff, so I have added another phase that is more complex and quite related to your particular context.
Here is one image's result after 1st phase:
And here are all results after 2nd phase:
As you see it's not perfect, 8 can be seen as B (well, even a human like me sees it as a B... but it can be easily fixed if you have only numbers in your world). There is also like a ":" character (a legacy from a vertical line that has been removed) that I can't get rid of either w/o tweaking the code too much...
The C# code:
static void Unbox(string inputFilePath, string outputFilePath)
{
using (var orig = new Mat(inputFilePath))
{
using (var gray = orig.CvtColor(ColorConversionCodes.BGR2GRAY))
{
using (var dst = orig.EmptyClone())
{
// this is what I call the "horizontal shake" pass.
// note I use the Rect shape here, this is important
using (var dilate = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(4, 1)))
{
Cv2.Dilate(gray, dst, dilate);
}
// erode just a bit to get back some numbers to life
using (var erode = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(2, 1)))
{
Cv2.Erode(dst, dst, erode);
}
// at this point, good OCR will see most numbers
// but we want to remove surrounding artifacts
// find countours
using (var canny = dst.Canny(0, 400))
{
var contours = canny.FindContoursAsArray(RetrievalModes.List, ContourApproximationModes.ApproxSimple);
// compute a bounding rect for all numbers w/o boxes and artifacts
// this is the tricky part where we try to discard what's not related exclusively to numbers
var boundingRect = Rect.Empty;
foreach (var contour in contours)
{
// discard some small and broken polygons
var polygon = Cv2.ApproxPolyDP(contour, 4, true);
if (polygon.Length < 3)
continue;
// we want only numbers, and boxes are approx 40px wide,
// so let's discard box-related polygons, if any
// and some other artifacts that passed previous checks
// this quite depends on some context knowledge...
var rect = Cv2.BoundingRect(polygon);
if (rect.Width > 40 || rect.Height < 15)
continue;
boundingRect = boundingRect.X == 0 ? rect : boundingRect.Union(rect);
}
using (var final = dst.Clone(boundingRect))
{
final.SaveImage(outputFilePath);
}
}
}
}
}
}