Hello, Daniel. On Tue, Sep 03, 2019 at 09:48:22PM +0200, Daniel Vetter wrote: > I think system memory separate from vram makes sense. For one, vram is > like 10x+ faster than system memory, so we definitely want to have > good control on that. But maybe we only want one vram bucket overall > for the entire system? > > The trouble with system memory is that gpu tasks pin that memory to > prep execution. There's two solutions: > - i915 has a shrinker. Lots (and I really mean lots) of pain with > direct reclaim recursion, which often means we can't free memory, and > we're angering the oom killer a lot. Plus it introduces real bad > latency spikes everywhere (gpu workloads are occasionally really slow, > think "worse than pageout to spinning rust" to get memory freed). > - ttm just has a global limit, set to 50% of system memory. > > I do think a global system memory limit to tame the shrinker, without > the ttm approach of possible just wasting half your memory, could be > useful. Hmm... what'd be the fundamental difference from slab or socket memory which are handled through memcg? Is system memory used by GPUs have further global restrictions in addition to the amount of physical memory used? > I'm also not sure of the bw limits, given all the fun we have on the > block io cgroups side. Aside from that the current bw limit only > controls the bw the kernel uses, userspace can submit unlimited > amounts of copying commands that use the same pcie links directly to > the gpu, bypassing this cg knob. Also, controlling execution time for > gpus is very tricky, since they work a lot more like a block io device > or maybe a network controller with packet scheduling, than a cpu. At the system level, it just gets folded into cpu time, which isn't perfect but is usually a good enough approximation of compute related dynamic resources. Can gpu do someting similar or at least start with that? Thanks. -- tejun
