I haven't had this issue on any of my machines, but have you tried re-creating the panel, or wayland vs X11? It looks like there are a few similar bug reports, but I imagine this issue may be present for different reasons on different systems. It may be worth checking system logs also. And not to get off topic, but I'd be interested to know what problems you're having with kmail? My experience has been really good with it.