Did you miss the part about NOT NEEDING kernel source in order to build modules against that kernel? FC2 and FC3 kernel provides headers that are sufficient 99% of the time, while FC4 split that out into kernel-devel which serves a similar purpose.
BTW, has anyone attempted to build netfilter modules using the kernel build system? It would be nice to be able to try out some of the experimental stuff without having to build a full custom kernel. (I'd particularly like to use the U32 match module to detect/block some malicious UDP packets.) IIRC, most of the modules include a matching userspace shared object that's used by the iptables binary so of course that would also be needed, but in principle one could package the two together.