What is OCR and how does it even work?

Full disclosure: I'm trying to achieve something between "explain like I'm five" and a computer science lecture, but leaning slightly towards ELI5 here.

OCR = optical character recognition. Fancy name for a simple job: take a picture of some text and turn it into text you can actually select and copy.

Sounds easy. It isn't. To a computer, an image is just a bunch of colored dots. Nobody told it some of those dots make an A.

Clean book scan? Usually fine. Blurry phone pic of a receipt, angled menu, weird font? That's where things get ugly.

Rough order of things in order

Clean the image up a little. Straighten it, knock down noise, crank contrast so ink stands out from paper. Garbage in, garbage out — so you try not to feed it garbage.

Figure out where the words even are. Half the picture might be a logo or empty space. Draw boxes around the stuff that looks like lines of text; ignore the rest.

Guess the letters inside those boxes. Left to right (or whatever reading order applies).

Old-school OCR did this with rules some programmer typed in by hand — “hole in the middle and no tail? call it an O. little squiggle on the side? maybe Q.” That works until the font gets cute or the photo gets bad, and then the whole rulebook falls over.

Newer OCR mostly learns from piles of example images instead. Show it enough messy text and it starts recognizing patterns without anyone writing a rule for every font.

Use language to unfuck the obvious mistakes. Alone, l / I / 1 can look identical. With neighbors, “he11o” becomes “hello” instead of whatever nonsense the shape-matcher coughed up.

Old vs new

Older stuff: human-written rules, great on tidy print, brittle everywhere else.

Newer stuff: machine learning, better on the mess phones and scanners actually produce. Still not magic. Glare and handwriting will still ruin your day.