A little while ago I ran into memory problems on an VPS. It turned out to be due to the GD library having issues with large images.
I discovered is that the PHP memory limit only applies to actual PHP code and not C libraries like GD that are called from PHP.
To put a cap on the memory usage of Apache I set process limits for the main Apache process using ulimit.
On Debian this can be done by adding this to the bottom of /etc/default/apache2:
ulimit -v 1048576
NOTE: This is for a limit of 1GB of virtual memory.