Skip to content

Commit

Permalink
GCORE: add necessary Linux-kernel patches
Browse files Browse the repository at this point in the history
  • Loading branch information
vankoven committed Oct 24, 2022
1 parent fd01671 commit 071dae5
Show file tree
Hide file tree
Showing 4 changed files with 318 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
From ce8766d359263571a748e3419874a96bb9eb6d99 Mon Sep 17 00:00:00 2001
From: "[Partner] Sergey Nizovtsev" <sn@tempesta-tech.com>
Date: Sat, 25 Jun 2022 08:50:05 +0300
Subject: [PATCH] GCORE: fpu: disable bottom-halves in FPU context

Signed-off-by: Sergey Nizovtsev <sn@tempesta-tech.com>
---
arch/x86/kernel/fpu/core.c | 8 ++++++++
1 file changed, 8 insertions(+)

diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c
index 98e6c29e17a48..de80a1bfe6bb6 100644
--- a/arch/x86/kernel/fpu/core.c
+++ b/arch/x86/kernel/fpu/core.c
@@ -425,6 +425,13 @@ EXPORT_SYMBOL_GPL(fpu_copy_uabi_to_guest_fpstate);

void kernel_fpu_begin_mask(unsigned int kfpu_mask)
{
+ /*
+ * We don't know in which context the function is called, but we know
+ * preciseely that softirq uses FPU, so we have to disable softirq as
+ * well as task preemption.
+ */
+ local_bh_disable();
+
preempt_disable();

WARN_ON_FPU(!irq_fpu_usable());
@@ -454,6 +461,7 @@ void kernel_fpu_end(void)

this_cpu_write(in_kernel_fpu, false);
preempt_enable();
+ local_bh_enable();
}
EXPORT_SYMBOL_GPL(kernel_fpu_end);

--
2.38.0

Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
From e3f937751fcfa16b809066500e09ec930332877f Mon Sep 17 00:00:00 2001
From: "[Partner] Sergey Nizovtsev" <sn@tempesta-tech.com>
Date: Sat, 25 Jun 2022 09:01:06 +0300
Subject: [PATCH] GCORE: bpf: allow dynamic kfuncs in XDP programs

Tell verifier to search allowed kfunc BTF IDs against dynamic set.

This patch allow modules to register their own helpers for XDP programs:

BTF_SET_START(mod_kfunc_ids)
BTF_ID(func, helper1)
BTF_ID(func, helper2)
BTF_SET_END(mod_kfunc_ids)

register_kfunc_btf_id_set(&prog_test_kfunc_list, &mod_kfunc_btf_set);
unregister_kfunc_btf_id_set(&prog_test_kfunc_list, &mod_kfunc_btf_set);

We hack prog_test_kfunc_list here as it is the only BTF set declared in
vmlinux. Linux-5.18 introduces a more generic way to register kfuncs so
this patch should be removed.

Signed-off-by: Sergey Nizovtsev <sn@tempesta-tech.com>
---
net/core/filter.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/net/core/filter.c b/net/core/filter.c
index f8fbb5fa74f35..4c3a0bac1879f 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -10094,6 +10094,7 @@ const struct bpf_verifier_ops xdp_verifier_ops = {
.is_valid_access = xdp_is_valid_access,
.convert_ctx_access = xdp_convert_ctx_access,
.gen_prologue = bpf_noop_prologue,
+ .check_kfunc_call = bpf_prog_test_check_kfunc_call,
};

const struct bpf_prog_ops xdp_prog_ops = {
--
2.38.0

Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
From aa934184cf90b7fe3a8ce0b481d6a512e0f96e9b Mon Sep 17 00:00:00 2001
From: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Date: Sat, 25 Jun 2022 09:15:40 +0300
Subject: [PATCH] bpf: Introduce mem, size argument pair support for kfunc

BPF helpers can associate two adjacent arguments together to pass memory
of certain size, using ARG_PTR_TO_MEM and ARG_CONST_SIZE arguments.
Since we don't use bpf_func_proto for kfunc, we need to leverage BTF to
implement similar support.

The ARG_CONST_SIZE processing for helpers is refactored into a common
check_mem_size_reg helper that is shared with kfunc as well. kfunc
ptr_to_mem support follows logic similar to global functions, where
verification is done as if pointer is not null, even when it may be
null.

This leads to a simple to follow rule for writing kfunc: always check
the argument pointer for NULL, except when it is PTR_TO_CTX. Also, the
PTR_TO_CTX case is also only safe when the helper expecting pointer to
program ctx is not exposed to other programs where same struct is not
ctx type. In that case, the type check will fall through to other cases
and would permit passing other types of pointers, possibly NULL at
runtime.

Currently, we require the size argument to be suffixed with "__sz" in
the parameter name. This information is then recorded in kernel BTF and
verified during function argument checking. In the future we can use BTF
tagging instead, and modify the kernel function definitions. This will
be a purely kernel-side change.

This allows us to have some form of backwards compatibility for
structures that are passed in to the kernel function with their size,
and allow variable length structures to be passed in if they are
accompanied by a size parameter.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20220114163953.1455836-5-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
(cherry picked from d583691c47dc0424ebe926000339a6d6cd590ff7)
Signed-off-by: Sergey Nizovtsev <sn@tempesta-tech.com>
---
include/linux/bpf_verifier.h | 2 +
kernel/bpf/btf.c | 48 +++++++++++++++++++--
kernel/bpf/verifier.c | 83 ++++++++++++++++++++++++++++++++++++
3 files changed, 130 insertions(+), 3 deletions(-)

diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index e9993172f892e..8cc8633647246 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -521,6 +521,8 @@ bpf_prog_offload_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt);

int check_ptr_off_reg(struct bpf_verifier_env *env,
const struct bpf_reg_state *reg, int regno);
+int check_kfunc_mem_size_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg,
+ u32 regno);
int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg,
u32 regno, u32 mem_size);

diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index ac89e65d1692e..30e72cc2744ed 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -5629,6 +5629,32 @@ static bool __btf_type_is_scalar_struct(struct bpf_verifier_log *log,
return true;
}

+static bool is_kfunc_arg_mem_size(const struct btf *btf,
+ const struct btf_param *arg,
+ const struct bpf_reg_state *reg)
+{
+ int len, sfx_len = sizeof("__sz") - 1;
+ const struct btf_type *t;
+ const char *param_name;
+
+ t = btf_type_skip_modifiers(btf, arg->type, NULL);
+ if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE)
+ return false;
+
+ /* In the future, this can be ported to use BTF tagging */
+ param_name = btf_name_by_offset(btf, arg->name_off);
+ if (str_is_empty(param_name))
+ return false;
+ len = strlen(param_name);
+ if (len < sfx_len)
+ return false;
+ param_name += len - sfx_len;
+ if (strncmp(param_name, "__sz", sfx_len))
+ return false;
+
+ return true;
+}
+
static int btf_check_func_arg_match(struct bpf_verifier_env *env,
const struct btf *btf, u32 func_id,
struct bpf_reg_state *regs,
@@ -5741,17 +5767,33 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,
u32 type_size;

if (is_kfunc) {
+ bool arg_mem_size = i + 1 < nargs && is_kfunc_arg_mem_size(btf, &args[i + 1], &regs[regno + 1]);
+
/* Permit pointer to mem, but only when argument
* type is pointer to scalar, or struct composed
* (recursively) of scalars.
+ * When arg_mem_size is true, the pointer can be
+ * void *.
*/
if (!btf_type_is_scalar(ref_t) &&
- !__btf_type_is_scalar_struct(log, btf, ref_t, 0)) {
+ !__btf_type_is_scalar_struct(log, btf, ref_t, 0) &&
+ (arg_mem_size ? !btf_type_is_void(ref_t) : 1)) {
bpf_log(log,
- "arg#%d pointer type %s %s must point to scalar or struct with scalar\n",
- i, btf_type_str(ref_t), ref_tname);
+ "arg#%d pointer type %s %s must point to %sscalar, or struct with scalar\n",
+ i, btf_type_str(ref_t), ref_tname, arg_mem_size ? "void, " : "");
return -EINVAL;
}
+
+ /* Check for mem, len pair */
+ if (arg_mem_size) {
+ if (check_kfunc_mem_size_reg(env, &regs[regno + 1], regno + 1)) {
+ bpf_log(log, "arg#%d arg#%d memory, len pair leads to invalid memory access\n",
+ i, i + 1);
+ return -EINVAL;
+ }
+ i++;
+ continue;
+ }
}

resolve_ret = btf_resolve_size(btf, ref_t, &type_size);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 984c5e446e570..55826810bf581 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -4892,6 +4892,61 @@ static int check_helper_mem_access(struct bpf_verifier_env *env, int regno,
}
}

+static int check_mem_size_reg(struct bpf_verifier_env *env,
+ struct bpf_reg_state *reg, u32 regno,
+ bool zero_size_allowed,
+ struct bpf_call_arg_meta *meta)
+{
+ int err;
+
+ /* This is used to refine r0 return value bounds for helpers
+ * that enforce this value as an upper bound on return values.
+ * See do_refine_retval_range() for helpers that can refine
+ * the return value. C type of helper is u32 so we pull register
+ * bound from umax_value however, if negative verifier errors
+ * out. Only upper bounds can be learned because retval is an
+ * int type and negative retvals are allowed.
+ */
+ meta->msize_max_value = reg->umax_value;
+
+ /* The register is SCALAR_VALUE; the access check
+ * happens using its boundaries.
+ */
+ if (!tnum_is_const(reg->var_off))
+ /* For unprivileged variable accesses, disable raw
+ * mode so that the program is required to
+ * initialize all the memory that the helper could
+ * just partially fill up.
+ */
+ meta = NULL;
+
+ if (reg->smin_value < 0) {
+ verbose(env, "R%d min value is negative, either use unsigned or 'var &= const'\n",
+ regno);
+ return -EACCES;
+ }
+
+ if (reg->umin_value == 0) {
+ err = check_helper_mem_access(env, regno - 1, 0,
+ zero_size_allowed,
+ meta);
+ if (err)
+ return err;
+ }
+
+ if (reg->umax_value >= BPF_MAX_VAR_SIZ) {
+ verbose(env, "R%d unbounded memory access, use 'var &= const' or 'if (var < const)'\n",
+ regno);
+ return -EACCES;
+ }
+ err = check_helper_mem_access(env, regno - 1,
+ reg->umax_value,
+ zero_size_allowed, meta);
+ if (!err)
+ err = mark_chain_precision(env, regno);
+ return err;
+}
+
int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg,
u32 regno, u32 mem_size)
{
@@ -4915,6 +4970,34 @@ int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg,
return check_helper_mem_access(env, regno, mem_size, true, NULL);
}

+int check_kfunc_mem_size_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg,
+ u32 regno)
+{
+ struct bpf_reg_state *mem_reg = &cur_regs(env)[regno - 1];
+ bool may_be_null = type_may_be_null(mem_reg->type);
+ struct bpf_reg_state saved_reg;
+ struct bpf_call_arg_meta meta;
+ int err;
+
+ WARN_ON_ONCE(regno < BPF_REG_2 || regno > BPF_REG_5);
+
+ memset(&meta, 0, sizeof(meta));
+
+ if (may_be_null) {
+ saved_reg = *mem_reg;
+ mark_ptr_not_null_reg(mem_reg);
+ }
+
+ err = check_mem_size_reg(env, reg, regno, true, &meta);
+ /* Check access for BPF_WRITE */
+ meta.raw_mode = true;
+ err = err ?: check_mem_size_reg(env, reg, regno, true, &meta);
+
+ if (may_be_null)
+ *mem_reg = saved_reg;
+ return err;
+}
+
/* Implementation details:
* bpf_map_lookup returns PTR_TO_MAP_VALUE_OR_NULL
* Two bpf_map_lookups (even with the same key) will have different reg->id.
--
2.38.0

1 change: 1 addition & 0 deletions linux-patches/linux-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Supported version - Linux 5.17

0 comments on commit 071dae5

Please sign in to comment.