It has been done. For example, we leverage the ASYNC interface to make the private-key operations remote. There was also an RT ticket that had an implementation for a version years ago. (I can't find the RT but I know it was there.) The PKCS#11 stuff might also be appropriate.
