I've used 2.5.x with patched nvidia drivers, and the results are nothing
short of amazing. The whole system feels a lot faster and more responsive
with the kernel improvements & fast drivers.
Did you get 2.5 working with the latest nvidia drivers? I'm running shrike here, with nvidia driver version 1.0-4363 - Frankly that's the big thing keeping me on 2.4 now - if nvidia plus e.g. 2.5.68 is a go, then I'm there.
One other thing: Does xscreensaver unlock for you when you type in your password, in a recent 2.5.6x kernel?
Joe