[Bug 110702] segfault in radeonsi HEVC hardware decoding with yuv420p10le

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Comment # 11 on bug 110702 from
(In reply to Christian König from comment #10)
> (In reply to Pierre-Eric Pelloux-Prayer from comment #9)
> > The driver fails when trying to allocate a buffer for this video with a
> > ENOMEM error (the requested size is 3 GB).
> 
> Well that strongly sounds like we miscalculated the necessary size somewhere.

Indeed.
The size is computed by the `calc_ctx_size_h265_main10()` function.

I'm not familiar enough with hevc to fix it though (but the calculation seems
to overflow because context_buffer_size_per_ctb_row is 1GB and is multiplied by
max_references (= 23) and the result is stored in an unsigned int).


You are receiving this mail because:
_______________________________________________
dri-devel mailing list
dri-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Index of Archives]     [Linux DRI Users]     [Linux Intel Graphics]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [XFree86]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux