Fortunately I found this post in German which Google kindly translated for me, and it solved my problems.

Since the translation screws up some of the code, I thought I would post the steps here in English in case anyone finds it useful.

Step 1: LAMP

If you don’t already have PHP 5 and Apache2 set up, this will get you started. Open up a terminal and type:
[bash]sudo tasksel install lamp-server[/bash]

Guiding you through setting up PHP and Apache is beyond the scope of this guide, so if you get stuck, check the multitude of guides available on Google.

Step 2: Upgrade LAMP

With PHP 5 and Apache installed, the next step is to install the developer versions.

• php5-dev (otherwise you’ll get an error about “phpize”)
• apache2-threaded-dev (or else you’ll get an “apxs” error)

[bash] sudo apt-get install php5-dev apache2-threaded-dev [/bash]

Step 3: Getting build tools

We’re going to be building from source. If you’ve never done that before it’s pretty easy, but first you’ll need some extra tools.
[bash] sudo apt-get install build-essential [/bash]

Step 4: PEAR and memcached

You’re also going to need PEAR and the memcached binary.
[bash] sudo apt-get install php-pear memcached [/bash]

Step 5: Building libmemcached

You’ll need to get the latest source package from here. You may have to try different versions until you get it working. In 9.04, libmemcached-0.33 worked. For 9.10, it was libmemcached-0.37. As of Ubuntu 10.04′s launch, the latest version was libmemcached-0.40, which appears to install correctly but under actual usage conditions the Memcached::add() method failed to write every time, returning error code 5. I recommend sticking with libmemcached-0.37 for now if you’re running Ubuntu 10.04.

Download the appropriate version with your preferred method, or type:
[bash] wget http://download.tangent.org/libmemcached-0.37.tar.gz [/bash]

Extract the files with your preferred method, or type:
[bash] tar -xzf libmemcached-0.37.tar.gz [/bash]

Move into the extracted directory:
[bash] cd libmemcached-0.37/ [/bash]

Configure the package.
[bash] ./configure [/bash]

Make.
[bash] make [/bash]

Make install. Note this probably will require sudo rights.
[bash] sudo make install [/bash]

Step 6: PECL

Finally, you can install the memcached extension through PECL. [Thanks to Andy Lav for pointing out this very important missing step in the comments.]
[bash] sudo pecl install memcached [/bash]

When this process completed, you should see something like this:
[bash]
Build process completed successfully
Installing ‘/usr/lib/php5/20060613+lfs/memcached.so’
install ok: channel://pecl.php.net/memcached-1.0.0
Extension memcached enabled in php.ini
[/bash]

If you see this, you should be ready to go.

You might instead get a message that says: You should add “extension = memcached.so” to php.ini. If that’s the case, open the file with the following and paste extension = memcached.so at the top:
[bash] gksudo gedit /etc/php5/apache2/php.ini [/bash]

After you save and close the file, restart Apache.
[bash] sudo /etc/init.d/apache2 restart [/bash]