I have lots of sockets tied to pid 1 (systemd) in CLOSE-WAIT, anyway to get rid of these?

[ccox@xxxxxxxxxxxxxx (Christopher Cox)]

On 06/20/2018 06:33 AM, Lennart Poettering wrote:
> On Di, 19.06.18 17:21, Christopher Cox (ccox at endlessnow.com) wrote:
>
>> Not sure what happened, but on one of our CentOS 7 host we had like 64
>> socket connects owned by pid 1 (systemd) in CLOSE-WAIT state.  And systemctl
>> -a shows a bunch of lines like:
>>
>> check_mk at 11305-10.32.50.31:6556-10.32.3.10:55106.service  (corresponding to
>> the 64 CLOSE-WAIT connections)
>>
>> I tried shutting down the socket and service and was able to get rid of the
>> CLOSE-WAIT sockets, however the entries in systemctl persist.  Is there
>> anyway to clear out those (they are now in  loaded failed failed
>> state).
>
> What's the precise "systemctl status" output for a "check_mk at .service"
> instance like this?
>
> My educated guess is that these instances of this per-connection
> service has failed, and is thus not GC'ed by systemd. It's generally
> wise to use the "-" modifier in ExecStart= in services like this, so
> that failures of the per-connection processes are
> ignored. Alternatively, with more recent systemd versions you can also
> set CollectMode=inactive-or-failed in the service's unit file, so that
> the failure is recorded normally, but the unit is unloaded anyway on GC.
>
> Lennart
>

The problem was they were "hung".  In this case on a locked up "df" that will 
never return.  Not sure if there's a solution for this case.