commit 336ae1180df5f69b9e0fb6561bec01c5f64361cf upstream.
There is a small race between when the cycle count is read from
the hardware and when the epoch stabilizes. Consider this
scenario:
CPU0 CPU1
---- ----
cyc = read_sched_clock()
cyc_to_sched_clock()
update_sched_clock()
...
cd.epoch_cyc = cyc;
epoch_cyc = cd.epoch_cyc;
...
epoch_ns + cyc_to_ns((cyc - epoch_cyc)
The cyc on cpu0 was read before the epoch changed. But we
calculate the nanoseconds based on the new epoch by subtracting
the new epoch from the old cycle count. Since epoch is most likely
larger than the old cycle count we calculate a large number that
will be converted to nanoseconds and added to epoch_ns, causing
time to jump forward too much.
Fix this problem by reading the hardware after the epoch has
stabilized.
Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Pull ARM SoC late cleanups from Arnd Bergmann:
"These are cleanups and smaller changes that either depend on earlier
feature branches or came in late during the development cycle. We
normally try to get all cleanups early, so these are the exceptions:
- A follow-up on the clocksource reworks, hopefully the last time we
need to merge clocksource subsystem changes through arm-soc.
A first set of patches was part of the original 3.10 arm-soc
cleanup series because of interdependencies with timer drivers now
moved out of arch/arm.
- Migrating the SPEAr13xx platform away from using auxdata for DMA
channel descriptions towards using information in device tree,
based on the earlier SPEAr multiplatform series
- A few follow-ups on the Atmel SAMA5 support and other changes for
Atmel at91 based on the larger at91 reworks.
- Moving the armada irqchip implementation to drivers/irqchip
- Several OMAP cleanups following up on the larger series already
merged in 3.10."
* tag 'cleanup-for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc: (50 commits)
ARM: OMAP4: change the device names in usb_bind_phy
ARM: OMAP2+: Fix mismerge for timer.c between ff931c82 and da4a686a
ARM: SPEAr: conditionalize SMP code
ARM: arch_timer: Silence debug preempt warnings
ARM: OMAP: remove unused variable
serial: amba-pl011: fix !CONFIG_DMA_ENGINE case
ata: arasan: remove the need for platform_data
ARM: at91/sama5d34ek.dts: remove not needed compatibility string
ARM: at91: dts: add MCI DMA support
ARM: at91: dts: add i2c dma support
ARM: at91: dts: set #dma-cells to the correct value
ARM: at91: suspend both memory controllers on at91sam9263
irqchip: armada-370-xp: slightly cleanup irq controller driver
irqchip: armada-370-xp: move IRQ handler to avoid forward declaration
irqchip: move IRQ driver for Armada 370/XP
ARM: mvebu: move L2 cache initialization in init_early()
devtree: add binding documentation for sp804
ARM: integrator-cp: convert use CLKSRC_OF for timer init
ARM: versatile: use OF init for sp804 timer
ARM: versatile: add versatile dtbs to dtbs target
...
cyc_to_sched_clock() is called by sched_clock() and cyc_to_ns() is called by cyc_to_sched_clock(). I suspect that some compilers inline both of these functions into sched_clock() and so we've been getting away without having a notrace marking. It seems that my compiler isn't inlining cyc_to_sched_clock() though, so I'm hitting a recursion bug when I enable the function graph tracer, causing my system to crash. Marking these functions notrace fixes it. Technically cyc_to_ns() doesn't need the notrace because it's already marked inline, but let's just add it so that if we ever remove inline from that function it doesn't blow up. Signed-off-by: Stephen Boyd <sboyd@codeaurora.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This converts sched_clock to simply a call to a function pointer in order to allow overriding it. This will allow for use with 64-bit counters where overflow handling is not needed. Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Allow multiple calls to setup_sched_clock and switch to the new counter if it is higher frequency. Signed-off-by: Rob Herring <rob.herring@calxeda.com>
If we want load epoch_cyc and epoch_ns atomically,
we should update epoch_cyc_copy first of all.
This notify reader that updating is in progress.
If we update epoch_cyc first like as current implementation,
there is subtle error case.
Look at the below example.
<Initial Condition>
cyc = 9
ns = 900
cyc_copy = 9
== CASE 1 ==
<CPU A = reader> <CPU B = updater>
write cyc = 10
read cyc = 10
read ns = 900
write ns = 1000
write cyc_copy = 10
read cyc_copy = 10
output = (10, 900)
== CASE 2 ==
<CPU A = reader> <CPU B = updater>
read cyc = 9
write cyc = 10
write ns = 1000
read ns = 1000
read cyc_copy = 9
write cyc_copy = 10
output = (9, 1000)
If atomic read is ensured, output should be (9, 900) or (10, 1000).
But, output in example case are not.
So, change updating sequence in order to correct this problem.
Cc: <stable@vger.kernel.org>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
The scheduler imposes a requirement to sched_clock() which is to stop the clock during suspend, if we don't do that any RT thread will be rescheduled in the future which might cause any sort of problems. This became an issue on OMAP when we converted omap-i2c.c to use threaded IRQs, it turned out that depending on how much time we spent on suspend, the I2C IRQ thread would end up being rescheduled so far in the future that I2C transfers would timeout and, because omap_hsmmc depends on an I2C-connected device to detect if an MMC card is inserted in the slot, our rootfs would just vanish. arch/arm/kernel/sched_clock.c already had an optional implementation (sched_clock_needs_suspend()) which would handle scheduler's requirement properly, what this patch does is simply to make that implementation non-optional. Note that this has the side-effect that printk timings won't reflect the actual time spent on suspend so other methods to measure that will have to be used. This has been tested with beagleboard XM (OMAP3630) and pandaboard rev A3 (OMAP4430). Suspend to RAM is now working after this patch. Thanks to Kevin Hilman for helping out with debugging. Acked-by: Kevin Hilman <khilman@ti.com> Acked-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Felipe Balbi <balbi@ti.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Add support for irq time accounting. This commit prepares ARM by adding the call to enable_sched_clock_irqtime() in sched_clock(). We introduce a new kernel parameter - irqtime - which takes an integer. -1 for auto, 0 for disabled, and 1 for enabled. Auto mode selects IRQ accounting if we have a sched_clock() tick rate greater than 1MHz. Frederic Weisbecker is working on a patch set which moves the IRQ_TIME_ACCOUNTING into arch/, so that part is not incorporated into this patch; this facility becomes available on ARM only when both this patch and Frederic's patches are merged. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Many clocks that are used to provide sched_clock will reset during suspend. If read_sched_clock returns 0 after suspend, sched_clock will appear to jump forward. This patch resets cd.epoch_cyc to the current value of read_sched_clock during resume, which causes sched_clock() just after suspend to return the same value as sched_clock() just before suspend. In addition, during the window where epoch_ns has been updated before suspend, but epoch_cyc has not been updated after suspend, it is unknown whether the clock has reset or not, and sched_clock() could return a bogus value. Add a suspended flag, and return the pre-suspend epoch_ns value during this period. The new behavior is triggered by calling setup_sched_clock_needs_suspend instead of setup_sched_clock. Signed-off-by: Colin Cross <ccross@android.com> Reviewed-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>