--- Begin Message ---
Good finding.
I’m not sure what motivated Paul to make this change. Perhaps we should inquire on list. He is an open source contractor who Nokia has under contract to make code ‘ready’ for open source. I suspect he felt it was an optimization. I am aware he filed some question to TI about the fixed delay value but not about the enable/disable sequence. It has been my experience that he is quick to make changes with code purity in mind. He does gladly take back bug fixes for those changes. There are many cases that the TI code has arrived at a working sequence which is re-coded in open source ‘clean ups’. Generally the code cleanness does improve but functional regressions are not uncommon. This is part of open source maturity.
I’m not sure of answer to sequencing issue you raise. Some research is necessary to find the answer. Maybe Girish or Rajendra know off hand.
Regards,
Richard W.
________________________________
From: Wang Limei-E12499 [mailto:E12499@xxxxxxxxxxxx]
Sent: Wednesday, July 08, 2009 3:26 PM
To: Wang Limei-E12499; Woodruff, Richard; Hunter, Jon; Ghongdemath, Girish; Nayak, Rajendra; mikechan@xxxxxxxxxx
Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836; De Chanterac Cyril-cdlc01; WIDZER NOAH-KFQG76; Liu Haiyang-DGRW68
Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in K29 open source PM
Just talked with Noah, he confirmed DLL should be enabled even when DLL is in unlock mode. DLL is disabled in unlock_dll is the reason of mem corruption.
>From the snapshot captured yesterday, SDRC_DLLA_CTRL is changed from 0x0F000008 to 0x0F000004 after omap3_configure_core_dpll is executed, means DLLLOCK is changed from lock mode to unlock mode,this is expected,but ENADLL bit is cleared, so DLL is disabled. SDRC_DLLA_CTRL is updated when running unlock_dll code, see 1 and 2 snapshot attached, it is masked when setting default FixedDelay. If enable DLL manually in LB, code can be executed correctly till hitting this function again. Since FixedDelay is already set to 0x0F before running these piece of code, pls see snapshot, if skipping these two instructions in RED, vdd2 dvfs works well. BTW, in K27 kernel sram-fn_34xx.S, does not include these.
unlock_dll:
ldr r11, omap3_sdrc_dlla_ctrl
ldr r12, [r11]
- and r12, r12, #FIXEDDELAY_MASK
- orr r12, r12, #FIXEDDELAY_DEFAULT
orr r12, r12, #DLLIDLE_MASK
str r12, [r11] @ (no OCP barrier needed)
bx lr
This change is made orignally in http://git.kernel.org/?p=linux/kernel/git/khilman/linux-omap-pm.git;a=commit;h=102bfe0bcb4e5985322b5f30b1321f2be1ad1c20 <http://git.kernel.org/?p=linux/kernel/git/khilman/linux-omap-pm.git;a=commit;h=102bfe0bcb4e5985322b5f30b1321f2be1ad1c20>
If it meant to set SDRC_DLLA_CTRL.FIXEDDELAY to 0x0f, need to be changed to below way or similar:
diff --git a/arch/arm/mach-omap2/sram34xx.S b/arch/arm/mach-omap2/sram34xx.S
index 91e3213..860a252 100644
--- a/arch/arm/mach-omap2/sram34xx.S
+++ b/arch/arm/mach-omap2/sram34xx.S
@@ -41,8 +41,7 @@
#define SDRC_UNLOCK_DLL 0x1
/* SDRC_DLLA_CTRL bit settings */
-#define FIXEDDELAY_SHIFT 24
-#define FIXEDDELAY_MASK (0xff << FIXEDDELAY_SHIFT)
+#define FIXEDDELAY_MASK 0x00FFFFFF
#define DLLIDLE_MASK 0x4
Richard and Girish,
Do you agree with this? Any comments?
Thanks,
limei
________________________________
From: Wang Limei-E12499
Sent: Wednesday, July 08, 2009 1:45 PM
To: Wang Limei-E12499; 'Woodruff, Richard'; 'Hunter, Jon'; 'Ghongdemath, Girish'; 'Nayak, Rajendra'; 'mikechan@xxxxxxxxxx'
Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836; De Chanterac Cyril-cdlc01; WIDZER NOAH-KFQG76; Liu Haiyang-DGRW68
Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in K29 open source PM
Girish and Ricard,
when unlock dll, clear SDRC_DLLA_CTRL. LOCKDLL , is DLL supposed to be disabled(SDRC_DLLA_CTRL.ENADLL) ? if it is disabled, what will happen?
I think when dll is unlocked, DLL should still be enabled, right?
thanks,
limei
________________________________
From: Wang Limei-E12499
Sent: Tuesday, July 07, 2009 6:59 PM
To: Woodruff, Richard; Hunter, Jon; Ghongdemath, Girish; Nayak, Rajendra; mikechan@xxxxxxxxxx
Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836; De Chanterac Cyril-cdlc01; WIDZER NOAH-KFQG76; Liu Haiyang-DGRW68; Wang Limei-E12499
Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in K29 open source PM
Ricarhd and Girish,
I think you refer to fix_dll_freeze, correct ? It is already in sleep34xx.S. I attached the the sleep34xx.s and sram34xx.s I am using. will you take a close look at them?
And I set the breakpoint at the return address of omap3_sram_configure_core_dpll 0xC0047FAC, and found that mem content is corrupted after returning from it. pls see attached snapshot before and after omap3_sram_configure_core_dpll is run. Seems like unlock DLL lead to mem corruption.
Again, if does not unlock DLL, no this problem, that ist the only difference between good and bad.
Thanks,
limei
________________________________
From: Woodruff, Richard [mailto:r-woodruff2@xxxxxx]
Sent: Monday, July 06, 2009 9:33 PM
To: Wang Limei-E12499; Hunter, Jon; Ghongdemath, Girish; Nayak, Rajendra; mikechan@xxxxxxxxxx
Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836; De Chanterac Cyril-cdlc01; WIDZER NOAH-KFQG76; Liu Haiyang-DGRW68
Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in K29 open source PM
Did you apply my force DLL to lock patch? I think Mike did. I didn’t see it in your code.
rkw
________________________________
From: Wang Limei-E12499 [mailto:E12499@xxxxxxxxxxxx]
Sent: Monday, July 06, 2009 9:23 PM
To: Wang Limei-E12499; Woodruff, Richard; Hunter, Jon; Ghongdemath, Girish; Nayak, Rajendra; mikechan@xxxxxxxxxx
Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836; De Chanterac Cyril-cdlc01; WIDZER NOAH-KFQG76; Wang Limei-E12499; Liu Haiyang-DGRW68
Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in K29 open source PM
More update about this issue:
omap3_sdrc_actim_ctrla,omap3_sdrc_actim_ctrlb,omap3_sdrc_rfr_ctrl are not re-configured when L3 clk is changed from 160.5 to 80.25, the reason is in current sram34xx.c/configure_sdrc, a hack was added to skip ddr timing configure, when hit this function, it returns instead of setting timing registers. Mike, will you expain what is the problem with these code?
but even after removing the hack, freeze still happen. when it occur, attach LB, find that dataabt happened. pls see attached snapshot. sdrc iclk is enabled and status is accessed.... Does anybody can check if the register setting is correct after freeze happens? In another two instances when occur happened, attached LB, PC stop at FFFFxxxx,seems around exception hander/vector, pls check the snapshot. Any idea about the data abort and unexpected instructions?
BTW, Hacking unlock_dll to zero still works with below patch.
Cyril or Noah,
Will you pls spend sometime with me tomorrow to debug this issue?
Below is the patch I applied.
diff --git a/arch/arm/mach-omap2/sram34xx.S b/arch/arm/mach-omap2/sram34xx.S
index 91e3213..9831981 100644
--- a/arch/arm/mach-omap2/sram34xx.S
+++ b/arch/arm/mach-omap2/sram34xx.S
@@ -151,11 +151,15 @@ configure_core_dpll:
and r12, r12, r10
orr r12, r12, r3, lsl #CORE_DPLL_CLKOUT_DIV_SHIFT
str r12, [r11]
- ldr r12, [r11] @ posted-write barrier for CM
- bx lr
+ ldr r12, clk_stabilize_delay @ wait for the clock to stabilise
wait_clk_stable:
subs r12, r12, #1
bne wait_clk_stable
+ nop
+ nop
+ nop
+ nop
+ nop
bx lr
enable_sdrc:
ldr r11, omap3_cm_iclken1_core
@@ -187,7 +191,6 @@ wait_dll_unlock:
bne wait_dll_unlock
bx lr
configure_sdrc:
- bx lr /* Skip ddr timing configurations, these values are bogus */
ldr r11, omap3_sdrc_rfr_ctrl
str r0, [r11]
ldr r11, omap3_sdrc_actim_ctrla
@@ -221,6 +224,8 @@ omap3_sdrc_dlla_ctrl:
.word OMAP34XX_SDRC_REGADDR(SDRC_DLLA_CTRL)
core_m2_mask_val:
.word 0x07FFFFFF
+clk_stabilize_delay:
+ .word 0x000000C8
ENTRY(omap3_sram_configure_core_dpll_sz)
.word . - omap3_sram_configure_core_dpll
thanks,
limei
________________________________
From: Wang Limei-E12499
Sent: Monday, July 06, 2009 1:09 PM
To: Woodruff, Richard; Hunter, Jon; Ghongdemath, Girish; Nayak, Rajendra; mikechan@xxxxxxxxxx
Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836; De Chanterac Cyril-cdlc01; WIDZER NOAH-KFQG76; Wang Limei-E12499
Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in K29 open source PM
Richard,
Thanks for the info!
I already applied the patch before, it did not resolve the problem. Remove the posted-write barrier, still does not work.
will you pls check the attached sram34xx.S? what is the delay to program for clk stablization? The clk stable delay is 103 mpu cycles when change freq from 160.5 to 80.25M, it is calculated in clock34xx.c. Also tried 200(0xC8), same thing.
/*
* XXX This only needs to be done when the CPU frequency changes
*/
mpurate = arm_fck.rate / CYCLES_PER_MHZ;
c = (mpurate << SDRC_MPURATE_SCALE) >> SDRC_MPURATE_BASE_SHIFT;
c += 1; /* for safety */
c *= SDRC_MPURATE_LOOPS;
c >>= SDRC_MPURATE_SCALE;
if (c == 0)
c = 1;
diff --git a/arch/arm/mach-omap2/sram34xx.S b/arch/arm/mach-omap2/sram34xx.S
index 91e3213..db7fef5 100644
--- a/arch/arm/mach-omap2/sram34xx.S
+++ b/arch/arm/mach-omap2/sram34xx.S
@@ -97,6 +97,8 @@ ENTRY(omap3_sram_configure_core_dpll)
blne lock_dll
bl sdram_in_selfrefresh @ put SDRAM in self refresh, idle SDRC
bl configure_core_dpll @ change the DPLL3 M2 divider
+ mov r12, r5
+ bl wait_clk_stable @ wait for SDRC to stabilize
bl enable_sdrc @ take SDRC out of idle
cmp r4, #SDRC_UNLOCK_DLL @ wait for DLL status to change
bleq wait_dll_unlock
@@ -104,8 +106,6 @@ ENTRY(omap3_sram_configure_core_dpll)
cmp r7, #1 @ if increasing SDRC clk rate,
beq return_to_sdram @ return to SDRAM code, otherwise,
bl configure_sdrc @ reprogram SDRC regs now
- mov r12, r5
- bl wait_clk_stable @ wait for SDRC to stabilize
return_to_sdram:
isb @ prevent speculative exec past here
mov r0, #0 @ return value
@@ -151,7 +151,6 @@ configure_core_dpll:
and r12, r12, r10
orr r12, r12, r3, lsl #CORE_DPLL_CLKOUT_DIV_SHIFT
str r12, [r11]
- ldr r12, [r11] @ posted-write barrier for CM
bx lr
wait_clk_stable:
subs r12, r12, #1
Thanks,
limei
________________________________
From: Woodruff, Richard [mailto:r-woodruff2@xxxxxx]
Sent: Sunday, July 05, 2009 8:56 AM
To: Wang Limei-E12499; Hunter, Jon; Ghongdemath, Girish; Nayak, Rajendra; mikechan@xxxxxxxxxx
Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836; De Chanterac Cyril-cdlc01; WIDZER NOAH-KFQG76
Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in K29 open source PM
Hello,
Yes, what you reported has been observed in the past. It is indicative of the SDRC hardware state machine being messed up. You need to wait small bit after changing the M2 divider before proceeding. There is a small period where the clock has not stabilized. If you let this unstable clock propagate to SDRC bad things a result.
You might try attached patch. Additionally I’d remove this read back to keep traffic limited to sram post clock change
ldr r12, [r11] @ posted-write barrier for CM <-- remove
Below is behavior you want:
- setup for changing M2 divider
- change M2 divider (in purple)
- simple delay with minimal outside module paths being touched
configure_core_dpll:
ldr r4, cm_clksel1_pll
ldr r5, [r4]
ldr r6, core_m2_mask_val @ modify m2 for core dpll
and r5, r5, r6
orr r5, r5, r3, lsl #0x1B @ r3 contains the M2 val
str r5, [r4]
mov r5, r10 @ get the clk_stab delay count
cmp r3, #2
bne wait_clk_stable
bx lr
wait_clk_stable:
subs r5, r5, #1
bne wait_clk_stable
A side note is your Lauterbach window has the potential to cause issues at delicate times. Any value visible is generating a polling access to that source. This will cause a number of abort which Lauterbach fixes up under the scenes. Sometimes its hard to predict their side effect. You can play with update rules but in this case I’d just hide everything but instructions till after you step through the critical section.
Regards,
Richard W.
> -----Original Message-----
> From: Wang Limei-E12499 [mailto:E12499@xxxxxxxxxxxx]
> Sent: Saturday, July 04, 2009 6:43 PM
> To: Hunter, Jon; Ghongdemath, Girish; Nayak, Rajendra; Woodruff, Richard;
> mikechan@xxxxxxxxxx
> Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836; De
> Chanterac Cyril-cdlc01; WIDZER NOAH-KFQG76; Wang Limei-E12499
> Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in K29 open source
> PM
>
> Girish,Richard and Noah,
>
> Last Friday I stepped through sram34xx.S/omap3_sram_configure_core_dpll
> again and checked the SDRC and RPCM register setting configured by
> these code using LB. Did not find issue when running through
> unlock_dll, sdram_in_selfrefresh,configure_core_dll, the related
> registers are all set correctly. But when running to enable_sdrc, in
> this routine, iclk is re-enabled and sdrc idle status is Accessed in
> cm_idlest1_core, but SDRC register are not accessible. Pls see the
> snapshot capture by LB.
>
> Do you know what might be the reason SDCR can not be accessed even after
> iclk is enabled and ST_SDRC say it can be accessed? And what is the
> impact of unlock DLL? Since if don't unlock DLL, seems fine.
>
> BTW, In the sram34xx.S attached, I moved wait_clk_stable to be next to
> configure_core_dpll.
>
> Thanks,
> limei
>
> -----Original Message-----
> From: Hunter, Jon [mailto:jon-hunter@xxxxxx]
> Sent: Thursday, July 02, 2009 4:40 PM
> To: Wang Limei-E12499; Ghongdemath, Girish; Nayak, Rajendra;
> richardwoodruff2@xxxxxx; mikechan@xxxxxxxxxx
> Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836;
> De Chanterac Cyril-cdlc01; WIDZER NOAH-KFQG76
> Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in K29 open
> source PM
>
> Hi Limei,
>
> Thanks for the update. Unfortunately, I am leaving on vacation this
> evening. Please work with Girish on this matter.
>
> Cheers
> Jon
> ________________________________________
> From: Wang Limei-E12499 [E12499@xxxxxxxxxxxx]
> Sent: Thursday, July 02, 2009 12:26 PM
> To: Hunter, Jon; Ghongdemath, Girish; Nayak, Rajendra;
> richardwoodruff2@xxxxxx; mikechan@xxxxxxxxxx
> Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836;
> De Chanterac Cyril-cdlc01; WIDZER NOAH-KFQG76; Wang Limei-E12499
> Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in K29 open
> source PM
>
> Jon, Girish, Richard,
>
> SRAM is already noncachable in K29 Open source PM code. The same piece
> of code you mentioned below are present in
> plat-omap/sram.c/omap-map-sram().
>
> And Jocelyn found that if force unlock_dll to zero, no more freeze. With
> this hack, testing result seems okay till now. Any idea about this? do
> you already know this issue, any patch?
>
> diff --git a/arch/arm/mach-omap2/clock34xx.c
> b/arch/arm/mach-omap2/clock34xx.c index 3cee6b4..82c1e98 100644
> --- a/arch/arm/mach-omap2/clock34xx.c
> +++ b/arch/arm/mach-omap2/clock34xx.c
> @@ -518,9 +518,12 @@ static int omap3_core_dpll_m2_set_rate(struct clk
> *clk, unsigned long rate)
>
> pr_debug("clock: changing CORE DPLL rate from %lu to %lu\n",
> clk->rate,
> validrate);
> +
> + /*Force it to 0 to hack freeze at wait_dll_unlock*/
> + unlock_dll = 0;
> omap3_configure_core_dpll(sp->rfr_ctrl, sp->actim_ctrla,
> sp->actim_ctrlb, new_div, unlock_dll, c,
> sp->mr, rate > clk->rate);
> --
>
> Girish,
>
> For your question, yes, I only see freeze, the system is still running
> and no crash. I can attach LB, but the instructions of PC stoppped at
> is abnormal, Attached aretwo isntances, it was not stuck at
> omap3_sram_configure_core_dpll,cpu is keeping on running, can stop and
> run multiple times by LB, but the instructions are all strange...
>
>
> thanks,
> limei
> ________________________________
> From: Hunter, Jon [mailto:jon-hunter@xxxxxx]
> Sent: Wednesday, July 01, 2009 10:53 AM
> To: Wang Limei-E12499; Ghongdemath, Girish; Nayak, Rajendra
> Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836
> Subject: RE: Mem freeze at omap3_sram_configure_core_dpll in open source
> PM
>
> Hi Limei,
>
> Sorry for the delay. The last time we had a problem with dvfs it turned
> out to be that the SRAM was being configured as cacheable. Can you make
> sure that the SRAM is being as noncacheable in the function
> omap_map_sram() in the file arch/arm/plat-omap/sram.c?
>
> You should see something like:
>
> 205 if (cpu_is_omap34xx()) {
> 206 omap_sram_io_desc[0].virtual = OMAP3_SRAM_VA;
> 207 base = OMAP3_SRAM_PA;
> 208 base = ROUND_DOWN(base, PAGE_SIZE);
> 209 omap_sram_io_desc[0].pfn = __phys_to_pfn(base);
> 210
> 211 /*
> 212 * SRAM must be marked as non-cached on OMAP3 since
> the
> 213 * CORE DPLL M2 divider change code (in SRAM) runs
> with the
> 214 * SDRAM controller disabled, and if it is marked
> cached,
> 215 * the ARM may attempt to write cache lines back to
> SDRAM
> 216 * which will cause the system to hang.
> 217 */
> 218 omap_sram_io_desc[0].type = MT_MEMORY_NONCACHED;
> 219 }
>
> Please make sure that the above code is getting executed correctly. You
> can view the code here:
>
> http://android.git.kernel.org/?p=kernel/omap.git;a=blob;f=arch/arm/plat-
> omap/sram.c;h=a2e60e7e3c7c851cdb3896ac26b5ddb3a282194d;hb=refs/heads/and
> roid-omap-2.6.29
>
> Cheers
> Jon
>
> ________________________________
> From: Wang Limei-E12499 [E12499@xxxxxxxxxxxx]
> Sent: Tuesday, June 30, 2009 12:15 AM
> To: Ghongdemath, Girish; Hunter, Jon; Nayak, Rajendra
> Cc: Sripathi Srinivas-A14759; Wang Sawsd-A24013; Falempe Jocelyn-XHP836;
> Wang Limei-E12499
> Subject: Mem freeze at omap3_sram_configure_core_dpll in open source PM
>
>
> Hi Girish and Hon,
>
> I am working on open source PM and trying to enable vdd2 dvfs on Sholes
> HW, but phone freeze when running to
> clock34xx.c/omap3_core_dpll_m2_set_rate(), I have applied the patch from
> Rajendra Nayak, but seems does not resolve my problem. I also checked
> the SDRC settings on both K27 (functional one) and K29 nonfunctional
> one, the SDRC setting and mem parameters passed to
> omap3_configure_core_dpll() are same, do you have any idea about this?
>
> when freeze, below is the log:
>
> DVFS: sdrc_ick.rate=160500000,sdrcrate=80250000
>
> clock: changing CORE DPLL rate from 321000000 to 160500000
>
> clock: SDRC timing params used: 00023f01 61512283 0001120c
> 00000032,new_div=2,unlock_dll=1,c=103
>
> Attached LB right after that, the mem content is abnormal. If attach LB
> first and set breakpoint at clock34xx.c/omap3_core_dpll_m2_set_rate()
> and then step through sram34xx.S/omap3_sram_configure_core_dpll(), when
> hit wait_dll_unlock instructions, LB status bar show "Emuation running",
> can not step any more.
>
> Thanks,
>
> limei
--- End Message ---
