Hi Erich,
hope it helps. Let us know.
Dietmar
Am 26. April 2024 15:52:06 MESZ schrieb Erich Weiler <weiler@xxxxxxxxxxxx>:
>Hi Dietmar,
>
>We do in fact have a bunch of users running vscode on our HPC head node as well (in addition to a few of our general purpose interactive compute servers). I'll suggest they make the mods you referenced! Thanks for the tip.
>
>cheers,
>erich
>
>On 4/24/24 12:58 PM, Dietmar Rieder wrote:
>> Hi Erich,
>>
>> in our case the "client failing to respond to cache pressure" situation is/was often caused by users how have vscode connecting via ssh to our HPC head node. vscode makes heavy use of file watchers and we have seen users with > 400k watchers. All these watched files must be held in the MDS cache and if you have multiple users at the same time running vscode it gets problematic.
>>
>> Unfortunately there is no global setting - at least none that we are aware of - for vscode to exclude certain files or directories from being watched. We asked the users to configure their vscode (Remote Settings -> Watcher Exclude) as follows:
>>
>> {
>> "files.watcherExclude": {
>> "**/.git/objects/**": true,
>> "**/.git/subtree-cache/**": true,
>> "**/node_modules/*/**": true,
>> "**/.cache/**": true,
>> "**/.conda/**": true,
>> "**/.local/**": true,
>> "**/.nextflow/**": true,
>> "**/work/**": true
>> }
>> }
>>
>> ~/.vscode-server/data/Machine/settings.json
>>
>> To monitor and find processes with watcher you may use inotify-info
>> <https://github.com/mikesart/inotify-info>
>>
>> HTH
>> Dietmar
>>
>> On 4/23/24 15:47, Erich Weiler wrote:
>>> So I'm trying to figure out ways to reduce the number of warnings I'm getting and I'm thinking about the one "client failing to respond to cache pressure".
>>>
>>> Is there maybe a way to tell a client (or all clients) to reduce the amount of cache it uses or to release caches quickly? Like, all the time?
>>>
>>> I know the linux kernel (and maybe ceph) likes to cache everything for a while, and rightfully so, but I suspect in my use case it may be more efficient to more quickly purge the cache or to in general just cache way less overall...?
>>>
>>> We have many thousands of threads all doing different things that are hitting our filesystem, so I suspect the caching isn't really doing me much good anyway due to the churn, and probably is causing more problems than it helping...
>>>
>>> -erich
>>> _______________________________________________
>>> ceph-users mailing list -- ceph-users@xxxxxxx
>>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>>>
>>
>>
>> _______________________________________________
>> ceph-users mailing list -- ceph-users@xxxxxxx
>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
