I have serious trouble with nvidia driver 331.20 and 331.38 here in Ubuntu 12.04. Both results in an "out of memory" error of the gfx driver here, after some minutes. This somehow only happens when you play with "selectBuf" selection method ( which is the recommended standard for nvidia cards ).
If you experience this trouble too, best switch you your selection method to "colorpicking". At least for me it looks like the problem is fixed doing this.
