We don't want GCC optimising our memset_io(), memcpy_fromio() or memcpy_toio() variants, so we must not call one of the standard functions. Provide a separate name for our assembly memcpy() and memset() functions, and use that instead, thereby bypassing GCC's ability to optimise these operations. GCCs optimisation may introduce unaligned accesses which are invalid for device mappings. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Convert the ioremap*() preprocessor macros to real functions, moving them out of line. This allows us to kill off __arm_ioremap(), and __arm_iounmap() helpers, and remove __arm_ioremap_pfn_caller() from global view. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ioremap_wt() was added by aliasing it to ioremap_nocache(), which is a device mapping. Device mappings do not allow unaligned accesses, but it appears that GCC is able to inline its own memcpy() implementation which may use such accesses. The only user of this is pmem, which uses memcpy() on the region. Therefore, this is unsafe. We must implement ioremap_wt() correctly for ARM, or not at all. This patch adds a more correct implementation by re-using ioremap_wc() to provide a normal-memory non-cacheable mapping. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Add documentation of the ARM specific behaviour of the mappings setup by the ioremap() series of macros. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Pull ARM updates from Russell King:
"Bigger items included in this update are:
- A series of updates from Arnd for ARM randconfig build failures
- Updates from Dmitry for StrongARM SA-1100 to move IRQ handling to
drivers/irqchip/
- Move ARMs SP804 timer to drivers/clocksource/
- Perf updates from Mark Rutland in preparation to move the ARM perf
code into drivers/ so it can be shared with ARM64.
- MCPM updates from Nicolas
- Add support for taking platform serial number from DT
- Re-implement Keystone2 physical address space switch to conform to
architecture requirements
- Clean up ARMv7 LPAE code, which goes in hand with the Keystone2
changes.
- L2C cleanups to avoid unlocking caches if we're prevented by the
secure support to unlock.
- Avoid cleaning a potentially dirty cache containing stale data on
CPU initialisation
- Add ARM-only entry point for secondary startup (for machines that
can only call into a Thumb kernel in ARM mode). Same thing is also
done for the resume entry point.
- Provide arch_irqs_disabled via asm-generic
- Enlarge ARMv7M vector table
- Always use BFD linker for VDSO, as gold doesn't accept some of the
options we need.
- Fix an incorrect BSYM (for Thumb symbols) usage, and convert all
BSYM compiler macros to a "badr" (for branch address).
- Shut up compiler warnings provoked by our cmpxchg() implementation.
- Ensure bad xchg sizes fail to link"
* 'for-linus' of git://ftp.arm.linux.org.uk/~rmk/linux-arm: (75 commits)
ARM: Fix build if CLKDEV_LOOKUP is not configured
ARM: fix new BSYM() usage introduced via for-arm-soc branch
ARM: 8383/1: nommu: avoid deprecated source register on mov
ARM: 8391/1: l2c: add options to overwrite prefetching behavior
ARM: 8390/1: irqflags: Get arch_irqs_disabled from asm-generic
ARM: 8387/1: arm/mm/dma-mapping.c: Add arm_coherent_dma_mmap
ARM: 8388/1: tcm: Don't crash when TCM banks are protected by TrustZone
ARM: 8384/1: VDSO: force use of BFD linker
ARM: 8385/1: VDSO: group link options
ARM: cmpxchg: avoid warnings from macro-ized cmpxchg() implementations
ARM: remove __bad_xchg definition
ARM: 8369/1: ARMv7M: define size of vector table for Vybrid
ARM: 8382/1: clocksource: make ARM_TIMER_SP804 depend on GENERIC_SCHED_CLOCK
ARM: 8366/1: move Dual-Timer SP804 driver to drivers/clocksource
ARM: 8365/1: introduce sp804_timer_disable and remove arm_timer.h inclusion
ARM: 8364/1: fix BE32 module loading
ARM: 8360/1: add secondary_startup_arm prototype in header file
ARM: 8359/1: correct secondary_startup_arm mode
ARM: proc-v7: sanitise and document registers around errata
ARM: proc-v7: clean up MIDR access
...
Add ioremap_wt() to all arch-specific asm/io.h headers which define ioremap_wc() locally. These headers do not include <asm-generic/iomap.h>. Some of them include <asm-generic/io.h>, but ioremap_wt() is defined for consistency since they define all ioremap_xxx locally. In all architectures without Write-Through support, ioremap_wt() is defined indentical to ioremap_nocache(). frv and m68k already have ioremap_writethrough(). On those we add ioremap_wt() indetical to ioremap_writethrough() and defines ARCH_HAS_IOREMAP_WT in both architectures. The ioremap_wt() interface is exported to drivers. Signed-off-by: Toshi Kani <toshi.kani@hp.com> Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Elliott@hp.com Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: arnd@arndb.de Cc: hch@lst.de Cc: hmh@hmh.eng.br Cc: jgross@suse.com Cc: konrad.wilk@oracle.com Cc: linux-mm <linux-mm@kvack.org> Cc: linux-nvdimm@lists.01.org Cc: stefan.bader@canonical.com Cc: yigal@plexistor.com Link: http://lkml.kernel.org/r/1433436928-31903-9-git-send-email-bp@alien8.de Signed-off-by: Ingo Molnar <mingo@kernel.org>
If we are building for a LE platform, and we haven't overriden the MMIO ops, then we can optimize the mem*io operations using the standard string functions. Acked-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
commit 195bbcac2e5c12f7fb ("ARM: 7500/1: io: avoid writeback addressing
modes for __raw_ accessors") disables writeback addressing modes for
raw i/o. However, the "+Q" output constraint forces the compiler to
disable load hoist optimizations (because the output constraint informs the
compiler of memory stores which the compiler assumes may alias other memory).
Since the relaxed accessors only guarantee ordering wrt i/o accesses to the
same device and not to main memory, there's never a possibility of an accessor
invalidating a hoisted load (because only non-i/o loads would have been hoisted).
The effect is especially noticable with complex address inputs in loops.
For example, the following code:
#include <linux/kernel.h>
#include <linux/io.h>
static const int *remap;
void wr_loop(void __iomem *base, int c, int val)
{
int i;
for (i = 0; i < c; i++)
writew_relaxed(val, base + remap[c >> 2]);
}
generates
current master | this patch
0: e3510000 cmp r1, #0 | 0: e3510000 cmp r1, #0
4: d12fff1e bxle lr | 4: d12fff1e bxle lr
8: e3003000 movw r3, #0 | 8: e3c1c003 bic ip, r1, #3
c: e3403000 movt r3, #0 | c: e6ff2072 uxth r2, r2
10: e92d4010 push {r4, lr} | 10: e3a03000 mov r3, #0
14: e6ff2072 uxth r2, r2 | 14: e59cc000 ldr ip, [ip]
18: e3c14003 bic r4, r1, #3 | 18: e080000c add r0, r0, ip
1c: e593e000 ldr lr, [r3] |
20: e3a03000 mov r3, #0 | 1c: e1c020b0 strh r2, [r0]
| 20: e2833001 add r3, r3, #1
24: e79ec004 ldr ip, [lr, r4] | 24: e1530001 cmp r3, r1
28: e080c00c add ip, r0, ip | 28: 1afffffb bne 1c
2c: e1cc20b0 strh r2, [ip] | 2c: e12fff1e bx lr
30: e2833001 add r3, r3, #1 |
34: e1530001 cmp r3, r1 |
38: 1afffff9 bne 24 |
|
3c: e8bd8010 pop {r4, pc} |
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Peter Hurley <peter@hurleysoftware.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Include the generic I/O header file so that duplicate implementations can be removed. This will also help to establish consistency across more architectures regarding which accessors they support. Signed-off-by: Thierry Reding <treding@nvidia.com>
This is needed for calls into OF code that parses PCI ranges. It signals support for memory mapped PCI I/O accesses that are described by device trees. Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Arnd Bergmann <arnd@arndb.de> CC: Russell King <linux@arm.linux.org.uk> CC: Rob Herring <robh+dt@kernel.org>