linux.git/arch/arm64/include/asm/cacheflush.h, branch v4.19

arm64: IPI each CPU after invalidating the I-cache for kernel mappings

2018-07-05T16:24:36+00:00

When invalidating the instruction cache for a kernel mapping via
flush_icache_range(), it is also necessary to flush the pipeline for
other CPUs so that instructions fetched into the pipeline before the
I-cache invalidation are discarded. For example, if module 'foo' is
unloaded and then module 'bar' is loaded into the same area of memory,
a CPU could end up executing instructions from 'foo' when branching into
'bar' if these instructions were fetched into the pipeline before 'foo'
was unloaded.

Whilst this is highly unlikely to occur in practice, particularly as
any exception acts as a context-synchronizing operation, following the
letter of the architecture requires us to execute an ISB on each CPU
in order for the new instruction stream to be visible.

Acked-by: Catalin Marinas 
Signed-off-by: Will Deacon

docs: Fix some broken references

2018-06-15T21:10:01+00:00

As we move stuff around, some doc references are broken. Fix some of
them via this script:
	./scripts/documentation-file-ref-check --fix

Manually checked if the produced result is valid, removing a few
false-positives.

Acked-by: Takashi Iwai 
Acked-by: Masami Hiramatsu 
Acked-by: Stephen Boyd 
Acked-by: Charles Keepax 
Acked-by: Mathieu Poirier 
Reviewed-by: Coly Li 
Signed-off-by: Mauro Carvalho Chehab 
Acked-by: Jonathan Corbet

arm64: turn flush_dcache_mmap_lock into a no-op

2018-04-11T17:28:39+00:00

ARM64 doesn't walk the VMA tree in its flush_dcache_page()
implementation, so has no need to take the tree_lock.

Link: http://lkml.kernel.org/r/20180313132639.17387-4-willy@infradead.org
Signed-off-by: Matthew Wilcox 
Reviewed-by: Will Deacon 
Cc: Darrick J. Wong 
Cc: Dave Chinner 
Cc: Jeff Layton 
Cc: Ryusuke Konishi 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

arm64: Add support for new control bits CTR_EL0.DIC and CTR_EL0.IDC

2018-03-09T13:57:57+00:00

The DCache clean & ICache invalidation requirements for instructions
to be data coherence are discoverable through new fields in CTR_EL0.
The following two control bits DIC and IDC were defined for this
purpose. No need to perform point of unification cache maintenance
operations from software on systems where CPU caches are transparent.

This patch optimize the three functions __flush_cache_user_range(),
clean_dcache_area_pou() and invalidate_icache_range() if the hardware
reports CTR_EL0.IDC and/or CTR_EL0.IDC. Basically it skips the two
instructions 'DC CVAU' and 'IC IVAU', and the associated loop logic
in order to avoid the unnecessary overhead.

CTR_EL0.DIC: Instruction cache invalidation requirements for
 instruction to data coherence. The meaning of this bit[29].
  0: Instruction cache invalidation to the point of unification
     is required for instruction to data coherence.
  1: Instruction cache cleaning to the point of unification is
      not required for instruction to data coherence.

CTR_EL0.IDC: Data cache clean requirements for instruction to data
 coherence. The meaning of this bit[28].
  0: Data cache clean to the point of unification is required for
     instruction to data coherence, unless CLIDR_EL1.LoC == 0b000
     or (CLIDR_EL1.LoUIS == 0b000 && CLIDR_EL1.LoUU == 0b000).
  1: Data cache clean to the point of unification is not required
     for instruction to data coherence.

Co-authored-by: Philip Elcan 
Reviewed-by: Mark Rutland 
Signed-off-by: Shanker Donthineni 
Signed-off-by: Will Deacon

arm64: KVM: Add invalidate_icache_range helper

2018-01-08T14:20:43+00:00

We currently tightly couple dcache clean with icache invalidation,
but KVM could do without the initial flush to PoU, as we've
already flushed things to PoC.

Let's introduce invalidate_icache_range which is limited to
invalidating the icache from the linear mapping (and thus
has none of the userspace fault handling complexity), and
wire it in KVM instead of flush_icache_range.

Reviewed-by: Christoffer Dall 
Signed-off-by: Marc Zyngier 
Signed-off-by: Christoffer Dall

arm64: mm: cleanup stale AIVIVT references

2017-11-28T18:13:18+00:00

Since commit:

  155433cb365ee466 ("arm64: cache: Remove support for ASID-tagged VIVT I-caches")

... the kernel no longer cares about AIVIVT I-caches, as these were
removed from the architecture.

This patch removes the stale references to such I-caches.

The comment in flush_context() is also updated to clarify when and where
the TLB invalidation occurs.

Signed-off-by: Mark Rutland 
Cc: Catalin Marinas 
Cc: Will Deacon 
Signed-off-by: Will Deacon

arm64: Implement pmem API support

2017-08-09T11:15:45+00:00

Add a clean-to-point-of-persistence cache maintenance helper, and wire
up the basic architectural support for the pmem driver based on it.

Reviewed-by: Will Deacon 
Signed-off-by: Robin Murphy 
[catalin.marinas@arm.com: move arch_*_pmem() functions to arch/arm64/mm/flush.c]
[catalin.marinas@arm.com: change dmb(sy) to dmb(osh)]
Signed-off-by: Catalin Marinas

arm64: Convert __inval_cache_range() to area-based

2017-08-09T10:00:23+00:00

__inval_cache_range() is already the odd one out among our data cache
maintenance routines as the only remaining range-based one; as we're
going to want an invalidation routine to call from C code for the pmem
API, let's tweak the prototype and name to bring it in line with the
clean operations, and to make its relationship with __dma_inv_area()
neatly mirror that of __clean_dcache_area_poc() and __dma_clean_area().
The loop clearing the early page tables gets mildly massaged in the
process for the sake of consistency.

Reviewed-by: Will Deacon 
Signed-off-by: Robin Murphy 
Signed-off-by: Catalin Marinas

arm64: mm: Fix set_memory_valid() declaration

2017-08-09T09:59:52+00:00

Clearly, set_memory_valid() has never been seen in the same room as its
declaration... Whilst the type mismatch is such that kexec probably
wasn't broken in practice, fix it to match the definition as it should.

Fixes: 9b0aa14e3155 ("arm64: mm: add set_memory_valid()")
Reviewed-by: Will Deacon 
Signed-off-by: Robin Murphy 
Signed-off-by: Catalin Marinas

treewide: decouple cacheflush.h and set_memory.h

2017-05-09T00:15:14+00:00

Now that all call sites, completely decouple cacheflush.h and
set_memory.h

[sfr@canb.auug.org.au: kprobes/x86: merge fix for set_memory.h decoupling]
  Link: http://lkml.kernel.org/r/20170418180903.10300fd3@canb.auug.org.au
Link: http://lkml.kernel.org/r/1488920133-27229-17-git-send-email-labbott@redhat.com
Signed-off-by: Laura Abbott 
Signed-off-by: Stephen Rothwell 
Acked-by: Catalin Marinas 
Acked-by: Mark Rutland 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds