linux.git/drivers/crypto/talitos.c, branch v4.16 Linux kernel source tree crypto: talitos - fix Kernel Oops on hashing an empty file 2018-02-08T11:37:06+00:00 LEROY Christophe christophe.leroy@c-s.fr 2018-01-26T16:09:59+00:00 87a81dce53b1ea61acaeefa5191a0376a2d1d721 Performing the hash of an empty file leads to a kernel Oops [ 44.504600] Unable to handle kernel paging request for data at address 0x0000000c [ 44.512819] Faulting instruction address: 0xc02d2be8 [ 44.524088] Oops: Kernel access of bad area, sig: 11 [#1] [ 44.529171] BE PREEMPT CMPC885 [ 44.532232] CPU: 0 PID: 491 Comm: md5sum Not tainted 4.15.0-rc8-00211-g3a968610b6ea #81 [ 44.540814] NIP: c02d2be8 LR: c02d2984 CTR: 00000000 [ 44.545812] REGS: c6813c90 TRAP: 0300 Not tainted (4.15.0-rc8-00211-g3a968610b6ea) [ 44.554223] MSR: 00009032 <EE,ME,IR,DR,RI> CR: 48222822 XER: 20000000 [ 44.560855] DAR: 0000000c DSISR: c0000000 [ 44.560855] GPR00: c02d28fc c6813d40 c6828000 c646fa40 00000001 00000001 00000001 00000000 [ 44.560855] GPR08: 0000004c 00000000 c000bfcc 00000000 28222822 100280d4 00000000 10020008 [ 44.560855] GPR16: 00000000 00000020 00000000 00000000 10024008 00000000 c646f9f0 c6179a10 [ 44.560855] GPR24: 00000000 00000001 c62f0018 c6179a10 00000000 c6367a30 c62f0000 c646f9c0 [ 44.598542] NIP [c02d2be8] ahash_process_req+0x448/0x700 [ 44.603751] LR [c02d2984] ahash_process_req+0x1e4/0x700 [ 44.608868] Call Trace: [ 44.611329] [c6813d40] [c02d28fc] ahash_process_req+0x15c/0x700 (unreliable) [ 44.618302] [c6813d90] [c02060c4] hash_recvmsg+0x11c/0x210 [ 44.623716] [c6813db0] [c0331354] ___sys_recvmsg+0x98/0x138 [ 44.629226] [c6813eb0] [c03332c0] __sys_recvmsg+0x40/0x84 [ 44.634562] [c6813f10] [c03336c0] SyS_socketcall+0xb8/0x1d4 [ 44.640073] [c6813f40] [c000d1ac] ret_from_syscall+0x0/0x38 [ 44.645530] Instruction dump: [ 44.648465] 38c00001 7f63db78 4e800421 7c791b78 54690ffe 0f090000 80ff0190 2f870000 [ 44.656122] 40befe50 2f990001 409e0210 813f01bc <8129000c> b39e003a 7d29c214 913e003c This patch fixes that Oops by checking if src is NULL. Fixes: 6a1e8d14156d4 ("crypto: talitos - making mapping helpers more generic") Cc: <stable@vger.kernel.org> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> 
Performing the hash of an empty file leads to a kernel Oops

[   44.504600] Unable to handle kernel paging request for data at address 0x0000000c
[   44.512819] Faulting instruction address: 0xc02d2be8
[   44.524088] Oops: Kernel access of bad area, sig: 11 [#1]
[   44.529171] BE PREEMPT CMPC885
[   44.532232] CPU: 0 PID: 491 Comm: md5sum Not tainted 4.15.0-rc8-00211-g3a968610b6ea #81
[   44.540814] NIP:  c02d2be8 LR: c02d2984 CTR: 00000000
[   44.545812] REGS: c6813c90 TRAP: 0300   Not tainted  (4.15.0-rc8-00211-g3a968610b6ea)
[   44.554223] MSR:  00009032 <EE,ME,IR,DR,RI>  CR: 48222822  XER: 20000000
[   44.560855] DAR: 0000000c DSISR: c0000000
[   44.560855] GPR00: c02d28fc c6813d40 c6828000 c646fa40 00000001 00000001 00000001 00000000
[   44.560855] GPR08: 0000004c 00000000 c000bfcc 00000000 28222822 100280d4 00000000 10020008
[   44.560855] GPR16: 00000000 00000020 00000000 00000000 10024008 00000000 c646f9f0 c6179a10
[   44.560855] GPR24: 00000000 00000001 c62f0018 c6179a10 00000000 c6367a30 c62f0000 c646f9c0
[   44.598542] NIP [c02d2be8] ahash_process_req+0x448/0x700
[   44.603751] LR [c02d2984] ahash_process_req+0x1e4/0x700
[   44.608868] Call Trace:
[   44.611329] [c6813d40] [c02d28fc] ahash_process_req+0x15c/0x700 (unreliable)
[   44.618302] [c6813d90] [c02060c4] hash_recvmsg+0x11c/0x210
[   44.623716] [c6813db0] [c0331354] ___sys_recvmsg+0x98/0x138
[   44.629226] [c6813eb0] [c03332c0] __sys_recvmsg+0x40/0x84
[   44.634562] [c6813f10] [c03336c0] SyS_socketcall+0xb8/0x1d4
[   44.640073] [c6813f40] [c000d1ac] ret_from_syscall+0x0/0x38
[   44.645530] Instruction dump:
[   44.648465] 38c00001 7f63db78 4e800421 7c791b78 54690ffe 0f090000 80ff0190 2f870000
[   44.656122] 40befe50 2f990001 409e0210 813f01bc <8129000c> b39e003a 7d29c214 913e003c

This patch fixes that Oops by checking if src is NULL.

Fixes: 6a1e8d14156d4 ("crypto: talitos - making mapping helpers more generic")
Cc: <stable@vger.kernel.org>
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: talitos - move to generic async completion 2017-11-03T14:11:22+00:00 Gilad Ben-Yossef gilad@benyossef.com 2017-10-18T07:00:49+00:00 f1c90ac3d71324ac7a6beebc33b0d493c4cbbf1a The talitos driver starts several async crypto ops and waits for their completions. Move it over to generic code doing the same. Signed-off-by: Gilad Ben-Yossef <gilad@benyossef.com> Tested-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> 
The talitos driver starts several async crypto ops and  waits for their
completions. Move it over to generic code doing the same.

Signed-off-by: Gilad Ben-Yossef <gilad@benyossef.com>
Tested-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: talitos - avoid useless copy 2017-10-12T14:55:39+00:00 LEROY Christophe christophe.leroy@c-s.fr 2017-10-06T13:05:08+00:00 3c0dd1903b17eb8e21d51195831880bd2ede3416 This patch avoids copy of buffered data to hash from bufnext to buf Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> 
This patch avoids copy of buffered data to hash from bufnext to buf

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: talitos - chain in buffered data for ahash on SEC1 2017-10-12T14:55:38+00:00 LEROY Christophe christophe.leroy@c-s.fr 2017-10-06T13:05:06+00:00 37b5e8897eb588a60dada00c33d2dca8a7fec5fc SEC1 doesn't support S/G in descriptors so for hash operations, the CPU has to build a buffer containing the buffered block and the incoming data. This generates a lot of memory copies which represents more than 50% of CPU time of a md5sum operation as shown below with a 'perf record'. |--86.24%-- kcapi_md_digest | | | |--86.18%-- _kcapi_common_vmsplice_chunk_fd | | | | | |--83.68%-- splice | | | | | | | |--83.59%-- ret_from_syscall | | | | | | | | | |--83.52%-- sys_splice | | | | | | | | | | | |--83.49%-- splice_from_pipe | | | | | | | | | | | | | |--83.04%-- __splice_from_pipe | | | | | | | | | | | | | | | |--80.67%-- pipe_to_sendpage | | | | | | | | | | | | | | | | | |--78.25%-- hash_sendpage | | | | | | | | | | | | | | | | | | | |--60.08%-- ahash_process_req | | | | | | | | | | | | | | | | | | | | | |--56.36%-- sg_copy_buffer | | | | | | | | | | | | | | | | | | | | | | | |--55.29%-- memcpy | | | | | | | | | | | | However, unlike SEC2+, SEC1 offers the possibility to chain descriptors. It is therefore possible to build a first descriptor pointing to the buffered data and a second descriptor pointing to the incoming data, hence avoiding the memory copy to a single buffer. With this patch, the time necessary for a md5sum on a 90Mbytes file is approximately 3 seconds. Without the patch it takes 6 seconds. Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> 
SEC1 doesn't support S/G in descriptors so for hash operations,
the CPU has to build a buffer containing the buffered block and
the incoming data. This generates a lot of memory copies which
represents more than 50% of CPU time of a md5sum operation as
shown below with a 'perf record'.

|--86.24%-- kcapi_md_digest
|          |
|          |--86.18%-- _kcapi_common_vmsplice_chunk_fd
|          |          |
|          |          |--83.68%-- splice
|          |          |          |
|          |          |          |--83.59%-- ret_from_syscall
|          |          |          |          |
|          |          |          |          |--83.52%-- sys_splice
|          |          |          |          |          |
|          |          |          |          |          |--83.49%-- splice_from_pipe
|          |          |          |          |          |          |
|          |          |          |          |          |          |--83.04%-- __splice_from_pipe
|          |          |          |          |          |          |          |
|          |          |          |          |          |          |          |--80.67%-- pipe_to_sendpage
|          |          |          |          |          |          |          |          |
|          |          |          |          |          |          |          |          |--78.25%-- hash_sendpage
|          |          |          |          |          |          |          |          |          |
|          |          |          |          |          |          |          |          |          |--60.08%-- ahash_process_req
|          |          |          |          |          |          |          |          |          |          |
|          |          |          |          |          |          |          |          |          |          |--56.36%-- sg_copy_buffer
|          |          |          |          |          |          |          |          |          |          |          |
|          |          |          |          |          |          |          |          |          |          |          |--55.29%-- memcpy
|          |          |          |          |          |          |          |          |          |          |          |

However, unlike SEC2+, SEC1 offers the possibility to chain
descriptors. It is therefore possible to build a first descriptor
pointing to the buffered data and a second descriptor pointing to
the incoming data, hence avoiding the memory copy to a single
buffer.

With this patch, the time necessary for a md5sum on a 90Mbytes file
is approximately 3 seconds. Without the patch it takes 6 seconds.

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: talitos - do hw_context DMA mapping outside the requests 2017-10-12T14:55:37+00:00 LEROY Christophe christophe.leroy@c-s.fr 2017-10-06T13:05:04+00:00 49f9783b0cea9a4e38fdbfe70ca296e6f20914c4 At every request, we map and unmap the same hash hw_context. This patch moves the dma mapping/unmapping in functions ahash_init() and ahash_import(). Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> 
At every request, we map and unmap the same hash hw_context.

This patch moves the dma mapping/unmapping in functions ahash_init()
and ahash_import().

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: talitos - DMA map key in setkey() 2017-10-12T14:55:34+00:00 LEROY Christophe christophe.leroy@c-s.fr 2017-10-06T13:05:02+00:00 2e13ce0812c624f1ee08ab933ebb89dddffed32c dma_map_single() is an heavy operation which doesn't need to be done at each request as the key doesn't change. Instead of DMA mapping the key at every request, this patch maps it once in setkey() Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> 
dma_map_single() is an heavy operation which doesn't need to
be done at each request as the key doesn't change.

Instead of DMA mapping the key at every request, this patch maps it
once in setkey()

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: talitos - simplify tests in ipsec_esp() 2017-10-12T14:55:33+00:00 LEROY Christophe christophe.leroy@c-s.fr 2017-10-06T13:04:59+00:00 9a655608a468535cd15eafff5ef93e2656e6e2fc Do (desc->hdr & DESC_HDR_TYPE_IPSEC_ESP) only once. Limit number of if/else paths Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> 
Do (desc->hdr & DESC_HDR_TYPE_IPSEC_ESP) only once.
Limit number of if/else paths

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: talitos - remove to_talitos_ptr_len() 2017-10-12T14:55:33+00:00 LEROY Christophe christophe.leroy@c-s.fr 2017-10-06T13:04:57+00:00 da9de146ebd99e162c4724ebce46a2bf2bc06085 to_talitos_ptr() and to_talitos_ptr_len() are always called together in order to fully set a ptr, so lets merge them into a single helper. Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> 
to_talitos_ptr() and to_talitos_ptr_len() are always called together
in order to fully set a ptr, so lets merge them into a single
helper.

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: talitos - don't check the number of channels at each interrupt 2017-10-12T14:55:32+00:00 LEROY Christophe christophe.leroy@c-s.fr 2017-10-06T13:04:55+00:00 9c02e2852fcabf0e1542929e887867d01414dec6 The number of channels is known from the beginning, no need to test it everytime. This patch defines two additional done functions handling only channel 0. Then the probe registers the correct one based on the number of channels. Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> 
The number of channels is known from the beginning, no need to
test it everytime.
This patch defines two additional done functions handling only channel 0.
Then the probe registers the correct one based on the number of channels.

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: talitos - use devm_ioremap() 2017-10-12T14:55:31+00:00 LEROY Christophe christophe.leroy@c-s.fr 2017-10-06T13:04:53+00:00 fd5ea7f011932d121e89e470992aaff8547c761f Use devm_ioremap() Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> 
Use devm_ioremap()

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>