linux.git/fs/bcachefs/btree_write_buffer.c, branch v6.8 Linux kernel source tree bcachefs: Prep work for variable size btree node buffers 2024-01-21T18:27:10+00:00 Kent Overstreet kent.overstreet@linux.dev 2024-01-16T18:29:59+00:00 ec4edd7b9d2038a97e0ba3fad8fc8492b0d12d35 bcachefs btree nodes are big - typically 256k - and btree roots are pinned in memory. As we're now up to 18 btrees, we now have significant memory overhead in mostly empty btree roots. And in the future we're going to start enforcing that certain btree node boundaries exist, to solve lock contention issues - analagous to XFS's AGIs. Thus, we need to start allocating smaller btree node buffers when we can. This patch changes code that refers to the filesystem constant c->opts.btree_node_size to refer to the btree node buffer size - btree_buf_bytes() - where appropriate. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> 
bcachefs btree nodes are big - typically 256k - and btree roots are
pinned in memory. As we're now up to 18 btrees, we now have significant
memory overhead in mostly empty btree roots.

And in the future we're going to start enforcing that certain btree node
boundaries exist, to solve lock contention issues - analagous to XFS's
AGIs.

Thus, we need to start allocating smaller btree node buffers when we
can. This patch changes code that refers to the filesystem constant
c->opts.btree_node_size to refer to the btree node buffer size -
btree_buf_bytes() - where appropriate.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bcachefs: __bch2_journal_key_to_wb -> bch2_journal_key_to_wb_slowpath 2024-01-06T04:24:19+00:00 Kent Overstreet kent.overstreet@linux.dev 2023-12-28T01:26:30+00:00 8feaebb0ae8882d688b4152c5d693e248d17e623 Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bcachefs: wb_key_cmp -> wb_key_ref_cmp 2024-01-06T04:24:19+00:00 Kent Overstreet kent.overstreet@linux.dev 2023-12-27T23:23:34+00:00 371650143d173ccdddc07405d5ee55e390a86921 Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bcachefs: btree_iter -> btree_path_idx_t 2024-01-01T16:47:43+00:00 Kent Overstreet kent.overstreet@linux.dev 2023-12-04T05:39:38+00:00 07f383c71fadc952059ed4ffe37dd465bda4ad3e Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bcachefs: bch2_btree_path_make_mut() -> btree_path_idx_t 2024-01-01T16:47:43+00:00 Kent Overstreet kent.overstreet@linux.dev 2023-12-08T07:24:05+00:00 f6363acaa63a84e7e79bd6e8b926d40a940e7c81 Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bcachefs: darray_for_each() now declares loop iter 2024-01-01T16:47:42+00:00 Kent Overstreet kent.overstreet@linux.dev 2023-12-17T02:40:26+00:00 defd9e39b59961ea0fd774020fb5c05dba199412 Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bcachefs: Inline btree write buffer sort 2024-01-01T16:47:41+00:00 Kent Overstreet kent.overstreet@linux.dev 2023-11-04T04:06:56+00:00 38ced43bb04ac59b1417f520f4d5a5c3bc3d1e57 The sort in the btree write buffer flush path is a very hot path, and it's particularly performance sensitive since it's single threaded and can block every other thread on a multithreaded write workload. It's well worth doing a sort with inlined cmp and swap functions. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> 
The sort in the btree write buffer flush path is a very hot path, and
it's particularly performance sensitive since it's single threaded and
can block every other thread on a multithreaded write workload.

It's well worth doing a sort with inlined cmp and swap functions.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bcachefs: btree write buffer now slurps keys from journal 2024-01-01T16:47:41+00:00 Kent Overstreet kent.overstreet@linux.dev 2023-11-02T22:57:19+00:00 09caeabe1a5daa3b667b797c401d43206d583f23 Previosuly, the transaction commit path would have to add keys to the btree write buffer as a separate operation, requiring additional global synchronization. This patch introduces a new journal entry type, which indicates that the keys need to be copied into the btree write buffer prior to being written out. We switch the journal entry type back to JSET_ENTRY_btree_keys prior to write, so this is not an on disk format change. Flushing the btree write buffer may require pulling keys out of journal entries yet to be written, and quiescing outstanding journal reservations; we previously added journal->buf_lock for synchronization with the journal write path. We also can't put strict bounds on the number of keys in the journal destined for the write buffer, which means we might overflow the size of the preallocated buffer and have to reallocate - this introduces a potentially fatal memory allocation failure. This is something we'll have to watch for, if it becomes an issue in practice we can do additional mitigation. The transaction commit path no longer has to explicitly check if the write buffer is full and wait on flushing; this is another performance optimization. Instead, when the btree write buffer is close to full we change the journal watermark, so that only reservations for journal reclaim are allowed. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> 
Previosuly, the transaction commit path would have to add keys to the
btree write buffer as a separate operation, requiring additional global
synchronization.

This patch introduces a new journal entry type, which indicates that the
keys need to be copied into the btree write buffer prior to being
written out. We switch the journal entry type back to
JSET_ENTRY_btree_keys prior to write, so this is not an on disk format
change.

Flushing the btree write buffer may require pulling keys out of journal
entries yet to be written, and quiescing outstanding journal
reservations; we previously added journal->buf_lock for synchronization
with the journal write path.

We also can't put strict bounds on the number of keys in the journal
destined for the write buffer, which means we might overflow the size of
the preallocated buffer and have to reallocate - this introduces a
potentially fatal memory allocation failure. This is something we'll
have to watch for, if it becomes an issue in practice we can do
additional mitigation.

The transaction commit path no longer has to explicitly check if the
write buffer is full and wait on flushing; this is another performance
optimization. Instead, when the btree write buffer is close to full we
change the journal watermark, so that only reservations for journal
reclaim are allowed.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bcachefs: more write buffer refactoring 2024-01-01T16:47:39+00:00 Kent Overstreet kent.overstreet@linux.dev 2023-11-27T03:06:48+00:00 8a4b4c52c0031f7f40261e41d66922a1049f0407 prep work for big rewrite - no functional changes in this patch. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> 
prep work for big rewrite - no functional changes in this patch.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bcachefs: wb_flush_one_slowpath() 2024-01-01T16:47:39+00:00 Kent Overstreet kent.overstreet@linux.dev 2023-11-27T02:58:11+00:00 ab4fb4b678c37837c6ba4e7a89f1afea4671a3ff A bit of refactoring for better inlining in the main btree write buffer flush path. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> 
A bit of refactoring for better inlining in the main btree write buffer
flush path.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>