Unlike other architectures with 32-bit and 64-bit flavors, where the biggest change in 64-bit binaries is using twice as much space for a lot of things, the x86-64 architecture also has twice as many registers as the 32-bit x86 and so 64-bit compiled code tends to perform noticeably better.
