On 2020-03-12 06:56, Nirmoy wrote: > > On 3/12/20 9:50 AM, Christian König wrote: >> Am 11.03.20 um 21:55 schrieb Nirmoy: >>> >>> On 3/11/20 9:35 PM, Andrey Grodzovsky wrote: >>>> >>>> On 3/11/20 4:32 PM, Nirmoy wrote: >>>>> >>>>> On 3/11/20 9:02 PM, Andrey Grodzovsky wrote: >>>>>> >>>>>> On 3/11/20 4:00 PM, Andrey Grodzovsky wrote: >>>>>>> >>>>>>> On 3/11/20 4:00 PM, Nirmoy Das wrote: >>>>>>>> [SNIP] >>>>>>>> @@ -1257,6 +1258,9 @@ static int amdgpu_cs_submit(struct >>>>>>>> amdgpu_cs_parser *p, >>>>>>>> priority = job->base.s_priority; >>>>>>>> drm_sched_entity_push_job(&job->base, entity); >>>>>>>> + if (ring->funcs->no_gpu_sched_loadbalance) >>>>>>>> + amdgpu_ctx_disable_gpu_sched_load_balance(entity); >>>>>>>> + >>>>>>> >>>>>>> >>>>>>> Why this needs to be done each time a job is submitted and not >>>>>>> once in drm_sched_entity_init (same foramdgpu_job_submit bellow ?) >>>>>>> >>>>>>> Andrey >>>>>> >>>>>> >>>>>> My bad - not in drm_sched_entity_init but in relevant amdgpu code. >>>>> >>>>> >>>>> Hi Andrey, >>>>> >>>>> Do you mean drm_sched_job_init() or after creating VCN entities? >>>>> >>>>> >>>>> Nirmoy >>>> >>>> >>>> I guess after creating the VCN entities (has to be amdgpu specific >>>> code) - I just don't get why it needs to be done each time job is >>>> submitted, I mean - since you set .no_gpu_sched_loadbalance = true >>>> anyway this is always true and so shouldn't you just initialize the >>>> VCN entity with a schedulers list consisting of one scheduler and >>>> that it ? >>> >>> >>> Assumption: If I understand correctly we shouldn't be doing load >>> balance among VCN jobs in the same context. Christian, James and Leo >>> can clarify that if I am wrong. >>> >>> But we can still do load balance of VNC jobs among multiple contexts. >>> That load balance decision happens in drm_sched_entity_init(). If we >>> initialize VCN entity with one scheduler then >>> >>> all entities irrespective of context gets that one scheduler which >>> means we are not utilizing extra VNC instances. >> >> Andrey has a very good point here. So far we only looked at this from >> the hardware requirement side that we can't change the ring after the >> first submission any more. >> >> But it is certainly valuable to keep the extra overhead out of the hot >> path during command submission. > > > >> >>> Ideally we should be calling >>> amdgpu_ctx_disable_gpu_sched_load_balance() only once after 1st call >>> of drm_sched_entity_init() of a VCN job. I am not sure how to do that >>> efficiently. >>> >>> Another option might be to copy the logic of >>> drm_sched_entity_get_free_sched() and choose suitable VNC sched >>> at/after VCN entity creation. >> >> Yes, but we should not copy the logic but rather refactor it :) >> >> Basically we need a drm_sched_pick_best() function which gets an array >> of drm_gpu_scheduler structures and returns the one with the least >> load on it. >> >> This function can then be used by VCN to pick one instance before >> initializing the entity as well as a replacement for >> drm_sched_entity_get_free_sched() to change the scheduler for load >> balancing. > > > This sounds like a optimum solution here. > > Thanks Andrey and Christian. I will resend with suggested changes. Note that this isn't an optimal solution. Note that drm_sched_pick_best() and drm_sched_entity_get_free_sched() (these names are too long), are similar in what they do, in that they pick a scheduler, which is still a centralized decision making. An optimal solution would be for each execution unit to pick work when work is available, which is a decentralized decision model. Not sure how an array would be used, as the proposition here is laid out--would that be an O(n) search through the array? In any case, centralized decision making introduces a bottleneck. Decentralized solutions are available for scheduling with O(1) time complexity. Regards, Luben > > >> >> Regards, >> Christian. >> >>> >>> >>> Regards, >>> >>> Nirmoy >>> >> > _______________________________________________ > amd-gfx mailing list > amd-gfx@xxxxxxxxxxxxxxxxxxxxx > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=02%7C01%7Cluben.tuikov%40amd.com%7C903b7b3a6faf480f1a7908d7c6738bae%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637196071895640764&sdata=RrPrZ5aHVOhMd5H8wEqCt%2FPPSBNLCyRVwDoLBU4p3Iw%3D&reserved=0 > _______________________________________________ amd-gfx mailing list amd-gfx@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/amd-gfx
