Captchas: the go-to solution to keeping bots away from sensitive forms. The problem is that they often don’t work.

It’s not uncommon for applications to protect sensitive forms exposed to unauthenticated users by showing an image of text, usually with extra lines through the writing, some letters blown up large and others quite small and any number of other distortions applied. Ideally, these should be easy for humans to solve, but not computers. Unfortunately, this has proven to be very difficult to achieve, and many of these captchas are now difficult for humans to read. Other solutions, such as reCaptcha, work by requiring people to select from various images using contextual knowledge.

Captcha bypasses are not new, but some applications still rely on them as a primary defense against automated attacks. When used as a defense-in-depth control to supplement other security measures, they can provide significant protection. When used alone, they can frequently be defeated which allows attackers to target sensitive application functionality or data.

During a recent mobile application assessment, I found a login form which protected sensitive user data - debit card numbers - from enumeration by attackers with a captcha. Unfortunately for the app, this captcha turned out not to be as strong as the developers hoped and I was able to defeat it with a few Python modules and a free Optical Character Recognition (OCR) program. The scripts below require only the following:

This blog post demonstrates how I developed a script to solve the captchas I encountered. The final script is not the most robust captcha solver, but it was sufficient to attack this application. More importantly, it shows the process I used to attack this captcha, highlights some of the difficulties image captchas face and shows why they should not be considered a primary security control.

What’s unique about my text?

The first thing we should take a look at is what is it about the text that’s different from the rest of the image. The captchas below were taken from a mobile app during a real assessment. Defeating them was all that stood between me and automated harvesting of debit card numbers.

Image

The text we want to recover is red, which doesn’t match the two lines meant to obscure the text. One of the first things we might try is to split these images up by their color channels: red, green and blue.

We can actually do this quite easily with a Python script, using the libraries above. He’s the one I put together to try and achieve this:

This produces the following images:

Red	Image
Green	Image
Blue	Image

At first glance, it might seem surprising that the text is completely absent in the red channel. However, in the RGB colorspace, the color white is represented by setting the red, green and blue channels to their maximum value. As a result, both white and red have the same value for the red channel.

Perhaps there are other colorspaces in which this red text is more recoverable? Pillow allows us to do a lot of conversions between color spaces. The Image.convert method can convert between some of the most common colorspaces, but we can perform some less common ones with only a little more effort:

Hue	Image
Saturation	Image
Value	Image
Cyan	Image
Magenta	Image
Yellow	Image
L	Image
a	Image
b	Image

Two of the most promising images are the “Yellow” channel image and the “a” channel. During the assessment, I chose to use the “a” channel. The strategy below could likely be applied to the yellow channel with similar results.

Removing Distractions

To “fix” this obscured image, we’re going to need to first identify a strategy that retains the text and discards the lines. But before we dive into this problem, we need to start thinking about our OCR library. In particular, Tesseract likes images with at least 300 dots per inch (DPI). These images were meant to be immediately displayed in a mobile application, so their DPI is a little dubious. If the data DPI isn’t encoded into the image metadata, we’re going to make a conservative assumption that it’s 72 DPI and scale up. The method below implements this:

Now we’re ready to return to the problem of removing the non-text lines. One of the simplest ways to do this is thresholding: If a pixel is “bright” enough (in the “a” channel) we’ll set it to full white. If it’s less than our threshold, we’ll set it to black. Within the images, pixel values range from 0 to 255. I initially tried a few threshold values starting with 128, but found a value of 180 worked best:

The code above produces the following image:

Image

We’ve successfully removed the obscuring lines and retained the text, but we’re missing some pieces of the text image. One way we might recover this missing data is by “expanding” the dark pixels. With Pillow, we can do this with a MinFilter. (Black has a value of 0, so expanding the black area means causing more pixels to become 0.) How much to expand the text was difficult to figure out in advance; by trial and error, 11 pixels did a decent job of closing up some of those missing pixel gaps:

Image

Next, we’re going to apply a MaxFilter to “contract” the text. Because the text is so thick on some images, letters ran together. The MaxFilter attempts to help fix this. Here’s the result:

Image

Here’s our new code after adding the filters:

OCR: solving the captcha

Finally, we need to actually run the image through Tesseract to get the text back. The Tesseract documentation indicates that it works best when images have a border, so we’ll have to add that to our image. This, too, is pretty straightforward with Pillow. We’ll add the function below, with the understanding that this method will crash if its border_size parameter is odd:

With the pytesseract module, this is actually quite straightforward. This module works by starting the tesseract program and passing our image data. Here’s the new code:

And the result:

We can refactor the code above into a function and attempt to test three of the images above. This gives us some confidence that we will solve all the captchas the server sends and that our code doesn’t just solve this one image. Here are the results for the three images above:

Image

The script correctly solves 2 out of the three, and missed the middle one by a single letter.

Making an educated guess

I then pulled 100 captcha images from the server and ran the script against these pages. The script correctly identified 29 of the 100 images. This is actually good enough for an attack, although it won’t be as fast as possible. If this is the only rate-limiting control of the application, we’ve effectively circumvented it. About one in four requests will pass the captcha, and I can attack the application endpoint effectively. At this point, the captcha has been defeated. Since it’s the only security control preventing automated attacks, the debit card numbers are now at risk.

One in four odds means we’re resending the same request, with different captcha solutions, frequently, however. With only a few more changes, we can do significantly better and create a more effective attack. Let’s take a look at some of the cases where we’re wrong, but close.

We can see three common patterns for mistakes. By far, the most common was confusing a “7” in the image with a forward slash (“/”). Because I had 100 captcha images to sort through, I was confident that there would never be a forward slash. The two next-most common errors were confusing an “8” with an amperpsand (“&”) and incorrectly identifying capital versions of some letters with little variance between upper- and lowercase forms. Replacing any forward slash with “7,” ampersand with “8” and converting the string to lowercase meant that the script recognized 44 of the 100 images.

We can make a few other corrections because we know some facts about all captcha solutions. All captchas were five characters, consisting of lowercase letters or numbers only. Applying all of the corrections below to the OCR result improved the success rate to 52 out of 100 images.

At this point, I launched the attack and was able to recover many debit card numbers. Since Optiv’s Application Security testing tries to avoid denial-of-service attacks, I made no effort to attack the application with a multithreaded, high-volume attack. Still, I was able to recover many valid card numbers in a few minutes. This demonstrates that a captcha is not a robust single line of defense against automated attacks.

When the captcha solution was correct, the remote API would either indicate that the card number wasn’t valid or would prompt the user to provide a passcode to finish logging in. After five login failures, the account would lock and it would only unlock if the customer called the bank. This is an effective rate-limiting control that prevents access to the accounts. The application could have implemented a similar protection for debit card numbers or rejected multiple debit card number requests from the same IP address in a short span. The captcha would have enhanced that protection, since failing the captcha half of the time means I would make twice as many requests to identify a single debit card number, on average, and been locked out sooner.

Captchas, Huh! What are they good for?

Captchas provide defense-in-depth protection at the cost of an easy user experience (UX). My final captcha solving script is under 70 lines of Python code and uses a collection of free software. This isn’t the strongest set of captcha images, but it took about six hours of hands-on-keyboard work to put together and costs nothing. If the reward is worth enough, there are more advanced paid captcha-solving tools attackers might use, and simply paying humans to solve captchas may be a viable approach. Applications (mobile and web) should not rely on captchas as a primary form of defense. Even things like reCatpcha aren’t a panacea, although they’re quite difficult to solve with computers.

Instead, consider more robust approaches. Locking user accounts after a series of failed logins dramatically limits the effectiveness of password-guessing attacks. These locks should be logged and monitored. Sudden spikes of account lockouts may indicate that an attacker is targeting the application, and security personnel can take action against the attack. Password reset endpoints, which take a user identifier and send an email with a reset link, can perform rate-limitng based on IP address. If a client makes a large number of requests in a short period, more than is reasonable for a single user trying to reset their password, future requests from the same IP can be rejected without processing them. A captcha might be added as a defense-in-depth measure to make these controls even stronger, but should not be relied upon as a primary security control.

References: