I did check the obvious things, and they were working before as well. cyrus is member of that ssl-cert group and that group should have permissions for /etc/ssl/private/ (not that it's the file in the error log). Actually /etc/ssl/certs, and all its files have read permissions to all, including the "others".
Permissions are the same on both the working and non-working server.
That's why I couldn't figure it out for hours. And I don't even have an idea what it could be.