linux-stable.git/arch/arm64/include/asm/pgtable.h, branch linux-4.4.y

arm64: Implement pmdp_set_access_flags() for hardware AF/DBM

2016-06-01T19:15:48+00:00

commit 282aa7051b0169991b34716f0f22d9c2f59c46c4 upstream.

The update to the accessed or dirty states for block mappings must be
done atomically on hardware with support for automatic AF/DBM. The
ptep_set_access_flags() function has been fixed as part of commit
66dbd6e61a52 ("arm64: Implement ptep_set_access_flags() for hardware
AF/DBM"). This patch brings pmdp_set_access_flags() in line with the pte
counterpart.

Fixes: 2f4b829c625e ("arm64: Add support for hardware updates of the access and dirty pte bits")
Reviewed-by: Will Deacon 
Signed-off-by: Catalin Marinas 
Signed-off-by: Will Deacon 
Signed-off-by: Greg Kroah-Hartman

arm64: Implement ptep_set_access_flags() for hardware AF/DBM

2016-06-01T19:15:48+00:00

commit 66dbd6e61a526ae7d11a208238ae2c17e5cacb6b upstream.

When hardware updates of the access and dirty states are enabled, the
default ptep_set_access_flags() implementation based on calling
set_pte_at() directly is potentially racy. This triggers the "racy dirty
state clearing" warning in set_pte_at() because an existing writable PTE
is overridden with a clean entry.

There are two main scenarios for this situation:

1. The CPU getting an access fault does not support hardware updates of
   the access/dirty flags. However, a different agent in the system
   (e.g. SMMU) can do this, therefore overriding a writable entry with a
   clean one could potentially lose the automatically updated dirty
   status

2. A more complex situation is possible when all CPUs support hardware
   AF/DBM:

   a) Initial state: shareable + writable vma and pte_none(pte)
   b) Read fault taken by two threads of the same process on different
      CPUs
   c) CPU0 takes the mmap_sem and proceeds to handling the fault. It
      eventually reaches do_set_pte() which sets a writable + clean pte.
      CPU0 releases the mmap_sem
   d) CPU1 acquires the mmap_sem and proceeds to handle_pte_fault(). The
      pte entry it reads is present, writable and clean and it continues
      to pte_mkyoung()
   e) CPU1 calls ptep_set_access_flags()

   If between (d) and (e) the hardware (another CPU) updates the dirty
   state (clears PTE_RDONLY), CPU1 will override the PTR_RDONLY bit
   marking the entry clean again.

This patch implements an arm64-specific ptep_set_access_flags() function
to perform an atomic update of the PTE flags.

Fixes: 2f4b829c625e ("arm64: Add support for hardware updates of the access and dirty pte bits")
Signed-off-by: Catalin Marinas 
Reported-by: Ming Lei 
Tested-by: Julien Grall 
Cc: Will Deacon 
[will: reworded comment]
Signed-off-by: Will Deacon 
Signed-off-by: Greg Kroah-Hartman

arm64: Ensure pmd_present() returns false after pmd_mknotpresent()

2016-06-01T19:15:48+00:00

commit 5bb1cc0ff9a6b68871970737e6c4c16919928d8b upstream.

Currently, pmd_present() only checks for a non-zero value, returning
true even after pmd_mknotpresent() (which only clears the type bits).
This patch converts pmd_present() to using pte_present(), similar to the
other pmd_*() checks. As a side effect, it will return true for
PROT_NONE mappings, though they are not yet used by the kernel with
transparent huge pages.

For consistency, also change pmd_mknotpresent() to only clear the
PMD_SECT_VALID bit, even though the PMD_TABLE_BIT is already 0 for block
mappings (no functional change). The unused PMD_SECT_PROT_NONE
definition is removed as transparent huge pages use the pte page prot
values.

Fixes: 9c7e535fcc17 ("arm64: mm: Route pmd thp functions through pte equivalents")
Reviewed-by: Will Deacon 
Signed-off-by: Catalin Marinas 
Signed-off-by: Will Deacon 
Signed-off-by: Greg Kroah-Hartman

arm64: Fix typo in the pmdp_huge_get_and_clear() definition

2016-06-01T19:15:48+00:00

commit 911f56eeb87ee378f5e215469268a7a2f68a5a8a upstream.

With hardware AF/DBM support, pmd modifications (transparent huge pages)
should be performed atomically using load/store exclusive. The initial
patches defined the get-and-clear function and __HAVE_ARCH_* macro
without the "huge" word, leaving the pmdp_huge_get_and_clear() to the
default, non-atomic implementation.

Fixes: 2f4b829c625e ("arm64: Add support for hardware updates of the access and dirty pte bits")
Reviewed-by: Will Deacon 
Signed-off-by: Catalin Marinas 
Signed-off-by: Will Deacon 
Signed-off-by: Greg Kroah-Hartman

arm64: Update PTE_RDONLY in set_pte_at() for PROT_NONE permission

2016-05-04T21:48:42+00:00

commit fdc69e7df3cb24f18a93192641786e5b7ecd1dfe upstream.

The set_pte_at() function must update the hardware PTE_RDONLY bit
depending on the state of the PTE_WRITE and PTE_DIRTY bits of the given
entry value. However, it currently only performs this for pte_valid()
entries, ignoring PTE_PROT_NONE. The side-effect is that PROT_NONE
mappings would not have the PTE_RDONLY bit set. Without
CONFIG_ARM64_HW_AFDBM, this is not an issue since such PROT_NONE pages
are not accessible anyway.

With commit 2f4b829c625e ("arm64: Add support for hardware updates of
the access and dirty pte bits"), the ptep_set_wrprotect() function was
re-written to cope with automatic hardware updates of the dirty state.
As an optimisation, only PTE_RDONLY is checked to assess the "dirty"
status. Since set_pte_at() does not set this bit for PROT_NONE mappings,
such pages may be considered "dirty" as a result of
ptep_set_wrprotect().

This patch updates the pte_valid() check to pte_present() in
set_pte_at(). It also adds PTE_PROT_NONE to the swap entry bits comment.

Fixes: 2f4b829c625e ("arm64: Add support for hardware updates of the access and dirty pte bits")
Signed-off-by: Catalin Marinas 
Reported-by: Ganapatrao Kulkarni 
Tested-by: Ganapatrao Kulkarni 
Signed-off-by: Greg Kroah-Hartman

arm64: Honour !PTE_WRITE in set_pte_at() for kernel mappings

2016-05-04T21:48:42+00:00

commit ac15bd63bbb24238f763ec5b24ee175ec301e8cd upstream.

Currently, set_pte_at() only checks the software PTE_WRITE bit for user
mappings when it sets or clears the hardware PTE_RDONLY accordingly. The
kernel ptes are written directly without any modification, relying
solely on the protection bits in macros like PAGE_KERNEL. However,
modifying kernel pte attributes via pte_wrprotect() would be ignored by
set_pte_at(). Since pte_wrprotect() does not set PTE_RDONLY (it only
clears PTE_WRITE), the new permission is not taken into account.

This patch changes set_pte_at() to adjust the read-only permission for
kernel ptes as well. As a side effect, existing PROT_* definitions used
for kernel ioremap*() need to include PTE_DIRTY | PTE_WRITE.

(additionally, white space fix for PTE_KERNEL_ROX)

Acked-by: Andrey Ryabinin 
Tested-by: Ard Biesheuvel 
Signed-off-by: Catalin Marinas 
Reported-by: Ard Biesheuvel 
Signed-off-by: Will Deacon 
Signed-off-by: Greg Kroah-Hartman

arm64: account for sparsemem section alignment when choosing vmemmap offset

2016-03-16T15:42:57+00:00

commit 36e5cd6b897e17d03008f81e075625d8e43e52d0 upstream.

Commit dfd55ad85e4a ("arm64: vmemmap: use virtual projection of linear
region") fixed an issue where the struct page array would overflow into the
adjacent virtual memory region if system RAM was placed so high up in
physical memory that its addresses were not representable in the build time
configured virtual address size.

However, the fix failed to take into account that the vmemmap region needs
to be relatively aligned with respect to the sparsemem section size, so that
a sequence of page structs corresponding with a sparsemem section in the
linear region appears naturally aligned in the vmemmap region.

So round up vmemmap to sparsemem section size. Since this essentially moves
the projection of the linear region up in memory, also revert the reduction
of the size of the vmemmap region.

Fixes: dfd55ad85e4a ("arm64: vmemmap: use virtual projection of linear region")
Tested-by: Mark Langsdorf 
Tested-by: David Daney 
Tested-by: Robert Richter 
Acked-by: Catalin Marinas 
Signed-off-by: Ard Biesheuvel 
Signed-off-by: Will Deacon 
Signed-off-by: Greg Kroah-Hartman

arm64: vmemmap: use virtual projection of linear region

2016-03-09T23:34:52+00:00

commit dfd55ad85e4a7fbaa82df12467515ac3c81e8a3e upstream.

Commit dd006da21646 ("arm64: mm: increase VA range of identity map") made
some changes to the memory mapping code to allow physical memory to reside
at an offset that exceeds the size of the virtual mapping.

However, since the size of the vmemmap area is proportional to the size of
the VA area, but it is populated relative to the physical space, we may
end up with the struct page array being mapped outside of the vmemmap
region. For instance, on my Seattle A0 box, I can see the following output
in the dmesg log.

   vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000   (     8 GB maximum)
             0xffffffbfc0000000 - 0xffffffbfd0000000   (   256 MB actual)

We can fix this by deciding that the vmemmap region is not a projection of
the physical space, but of the virtual space above PAGE_OFFSET, i.e., the
linear region. This way, we are guaranteed that the vmemmap region is of
sufficient size, and we can even reduce the size by half.

Signed-off-by: Ard Biesheuvel 
Signed-off-by: Will Deacon 
Signed-off-by: Greg Kroah-Hartman

arm64: Improve error reporting on set_pte_at() checks

2015-12-11T15:44:24+00:00

Currently the BUG_ON() checks do not give enough information about the
PTEs being set. This patch changes BUG_ON to WARN_ONCE and dumps the
values of the old and new PTEs. In addition, the checks are only made if
the new PTE entry is valid.

Signed-off-by: Catalin Marinas 
Reported-by: Ming Lei 
Cc: Will Deacon

arm64: Fix R/O permissions in mark_rodata_ro

2015-11-18T12:11:36+00:00

The permissions in mark_rodata_ro trigger a build error
with STRICT_MM_TYPECHECKS. Fix this by introducing
PAGE_KERNEL_ROX for the same reasons as PAGE_KERNEL_RO.
From Ard:

"PAGE_KERNEL_EXEC has PTE_WRITE set as well, making the range
writeable under the ARMv8.1 DBM feature, that manages the
dirty bit in hardware (writing to a page with the PTE_RDONLY
and PTE_WRITE bits both set will clear the PTE_RDONLY bit in that case)"

Signed-off-by: Laura Abbott 
Acked-by: Ard Biesheuvel 
Signed-off-by: Catalin Marinas