On 7/14/2023 4:23 PM, Will Deacon wrote:
On Fri, Jul 14, 2023 at 09:56:27AM +0800, Aiqun(Maria) Yu wrote:
On 7/14/2023 3:08 AM, Mark Rutland wrote:
On Thu, Jul 13, 2023 at 10:08:34PM +0800, Aiqun(Maria) Yu wrote:
On 7/13/2023 7:20 PM, Mark Rutland wrote:
Are you saying that LSE atomics to *cacheable* mappings do not work on your
system?
Specifically, when using a Normal Inner-Shareable Inner-Writeback
Outer-Writeback mapping, do the LSE atomics work or not work?
*cacheable* mapping have the LSE atomic is not working if far atomic is
performed.
Thanks for confirming; the fact that this doesn't work on *cacheable* memory is
definitely a major issue. I think everyone is confused here because of the
earlier mention of non-cachable accesses (which don't matter).
Maybe I can have the information collected in a summary to see if that
helps.
I know that some CPU implementations have EL3 control bits to force LSE atomics
to be performed near (e.g. in Cortex-A55, the CPUECTLR.ATOM control bits),
which would avoid the issue while still allowing the LSE atomics to be used.
If those can be configured in EL3 firmware, that would be a preferable
workaround.
Can you say which CPUs are integrated in this system? and/or can you check if
such control bits exist?
We have CPUECTLR_EL1.ATOM bit can force LSE atomics to be perform near.
CPUECTLR_EL1 is also an option to EL1 kernel drivers to be configuarable.
Try to a detailed summarise of the whole discussions, anyone can ignore some
part if you are already know.
* Part 1: Solution for this issue.
While we still want to have options to let third party and end users can
have options:
1.Disable lse atomic cap.
2.*Disallow* far atomic by "CPUECTLR_EL1.atom force near atomic" and
non-cachable mappling for lse atomic only.
Sorry, but this still isn't making sense to me. Which CPUs do you have on
this SoC?
cpu is cortex A78/A55.
My understanding of the CPUs from ARM is that LSE atomics are not supposed
to be sent to blocks that don't support them. That doesn't mean you have to
do everything near, however -- you can still execute them at e.g. L2.
For example, the Cortex-X1 TRM states:
| Atomic instructions to cacheable memory can be performed as either
| near atomics or far atomics, depending on where the cache line
| containing the data resides.
|
| When an instruction hits in the L1 data cache in a unique state, then
| it is performed as a near atomic in the L1 memory system. If the atomic
| operation misses in the L1 cache, or the line is shared with another
| core, then the atomic is sent as a far atomic on the core CHI interface.
lse atomic is optional to CHI-B for example, some system may have cpu
feature register have lse atomic feature, but the far atomic is not
accpeted by CHI side. It will be simiar issue that we do.
|
| If the operation misses everywhere within the cluster, and the
| interconnect supports far atomics, then the atomic is passed on to the
| interconnect to perform the operation.
|
| When the operation hits anywhere inside the cluster, or when an
| interconnect does not support atomics, the L3 memory system performs
| the atomic operation. If the line is not already there, it allocates
| the line into the L3 cache. This depends on whether the DSU is configured
| with an L3 cache.
So something doesn't add up.
* Part 2: Why we need the solution
1. There is also some case far atomic is better performance than near
atomic. end user may still can still try to do allow far atomic.
while this driver is also use kerenl LSE ATOMIC macro, so it can be running
on cpu don't support lse atomic and cpu support lse atomic already.
while current system, cpu have feature register said lse atomic is
supported, but memory controller is not supported is currently not yet
supported.
I think you're forgetting the fact that these instructions can be executed
by userspace, so the kernel option is completely bogus. If you're saying
that cacheable atomics can cause external aborts, then I can write an app
which will crash your device even if you've set this command line option.
For apps like userspace also needed to check the system capbility as far
as I know.
Will
--
Thx and BRs,
Aiqun(Maria) Yu
