Using a MIPS64 config (built using gas-abi=o32 as suggested by Kumba), it doesn't even get that far:
This is certainly wrong. What you really want is gas-abi=o64. Take a look at the O2 minimum patchset at http://www.total-knowledge.com, as the arch/mips/Makefile changes are what you need. Using a 64-bit kernel on your indy is pointless unless you want to run n32 userland anyhow.
As of this moment (may change in the future), 2.4 kernels are much better for ip22 anyhow. Serial console, indycam, and sound all work properly in 2.4. In 2.6, serial console is broken, there is no indycam support, and I'm running into some issues with sound where the cpu usage runs way up.
Steve