The vmtest action is used by several workflows: test, pahole, ondemand.
At the same time, vmtest action requires valid access rights to /dev/kvm
and is the only action that uses it.
This commit moves /dev/kvm permissions setup from test workflow to
vmtest action, in order to make sure that setup logic is shared by all
workflows that run vmtest.
Should fix CI failures like [1].
[1] https://github.com/libbpf/libbpf/actions/runs/7104762048/job/19340484589
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
The following 'sockopt' selftests fail on libbpf CI for kernel 5.5:
- sockopt/getsockopt: read ctx->optlen:FAIL
- sockopt/getsockopt: support smaller ctx->optlen:FAIL
- sockopt/setsockopt: read ctx->level:FAIL
- sockopt/setsockopt: read ctx->optname:FAIL
- sockopt/setsockopt: read ctx->optlen:FAIL
- sockopt/setsockopt: ctx->optlen == -1 is ok:FAIL
Examples of failing CI runs:
- https://github.com/libbpf/libbpf/actions/runs/6961182067
- https://github.com/libbpf/libbpf/actions/runs/6961088131
The failures are strange as all tests were added quite a while ago
(Jun 27 2019) by commit:
9ec8a4c9489d ("selftests/bpf: add sockopt test")
But seem to be unrelated to libbpf.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
All tests disabled in this commit pass on main kernel CI and fail or
flip/flop on libbpf CI. Failures do not seem to be related to libbpf.
It appears that common theme for all failing tests is that hardware
perf events are not delivered as expected on github CI worker
machines.
Examples of failed CI runs:
- https://github.com/libbpf/libbpf/actions/runs/6961182067
- https://github.com/libbpf/libbpf/actions/runs/6961088131
Fails with the following log:
test_send_signal_common:FAIL:incorrect result \
unexpected incorrect result: actual 48 != expected 50
Test mode of operation:
- fork'
- child:
- install handler for SIGUSR1;
- send ready message to parent;
- wait for SIGUSR1 in busy loop;
- send message '2' (50) to parent if SIGUSR1 occured;
- send message '0' (48) to parent if no SIGUSR1 occured.
- parent:
- wait for ready message from child;
- install perf_event or tracepoint bpf program that uses
bpf_send_signal() to send SIGUSR1;
- wait for message '0' or '2' from child, '2' is expected for test
success.
It appears that perf event that should be triggered by parent never
happens, thus message 48 is received by parent and test fails.
Fails with the following log:
test_and_reset_skel:FAIL:found_vm_exec \
unexpected found_vm_exec: actual 0 != expected 1
Such log is printed if variables set from BPF program are not set
after some timeout. The program that should set the variable is
SEC("perf_event") int handle_pe(void), it appears that it is never run.
Fails with the following log:
pe_subtest:FAIL:pe_res1 unexpected pe_res1: actual 0 != expected 1048576
Variable pe_res1 should be triggered by program
SEC("perf_event") int handle_pe(struct pt_regs *ctx),
it appears that it is never run.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
s390 tests are executed on selfhosted runner using root user,
avoid setting /dev/kvm permissions in such case.
This should fix CI failures like [0].
(Still necessary for x86 tests executed on standard github runners).
[0] https://github.com/libbpf/libbpf/actions/runs/6898545987/job/18768732980?pr=752
Fixes: 168630f852 ("ci: give /dev/kvm 0666 permissions inside CI runner")
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Recent kernel commit [0] changed selftests config snippets structure
by extracting VM specific options to the file 'config.vm'. This file
has to be used in .github/actions/vmtest/action.yml at step
'Prepare to build BPF selftests', otherwise drivers necessary for e.g.
root file system access are not compiled into the kernel, leading to
CI failures like [1].
[0] b0cf0dcde8ca ("selftests/bpf: Consolidate VIRTIO/9P configs in config.vm file")
[1] https://github.com/libbpf/libbpf/actions/runs/6830439839/job/18578379328?pr=747
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Starting recently libbpf CI runs started failing with the following
error:
##[group]vm_init - Starting virtual machine...
Starting VM with 4 CPUs...
INFO: /dev/kvm exists
KVM acceleration can be used
Could not access KVM kernel module: Permission denied
qemu-system-x86_64: failed to initialize KVM: Permission denied
##[error]Process completed with exit code 2.
E.g. see here [0]. The error happens because CI user has not enough
rights to access /dev/kvm. On a regular machine the solution would be
to add user to group 'kvm', however that would require a re-login,
which is cumbersome to achieve in CI setting.
Instead, use a recipe described in [1] to make udev set 0666 access
permissions for /dev/kvm.
[0] https://github.com/libbpf/libbpf/actions/runs/6819530119/job/18547589967?pr=746
[1] https://stackoverflow.com/questions/37300811/android-studio-dev-kvm-device-permission-denied/61984745#61984745
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Double-free error in bpf_linker__free() was reported by James Hilliard.
The error is caused by miss-use of realloc() in extend_sec().
The error occurs when two files with empty sections of the same name
are linked:
- when first file is processed:
- extend_sec() calls realloc(dst->raw_data, dst_align_sz)
with dst->raw_data == NULL and dst_align_sz == 0;
- dst->raw_data is set to a special pointer to a memory block of
size zero;
- when second file is processed:
- extend_sec() calls realloc(dst->raw_data, dst_align_sz)
with dst->raw_data == <special pointer> and dst_align_sz == 0;
- realloc() "frees" dst->raw_data special pointer and returns NULL;
- extend_sec() exits with -ENOMEM, and the old dst->raw_data value
is preserved (it is now invalid);
- eventually, bpf_linker__free() attempts to free dst->raw_data again.
This patch fixes the bug by avoiding -ENOMEM exit for dst_align_sz == 0.
The fix was suggested by Andrii Nakryiko <andrii.nakryiko@gmail.com>.
Reported-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Tested-by: James Hilliard <james.hilliard1@gmail.com>
Link: https://lore.kernel.org/bpf/CADvTj4o7ZWUikKwNTwFq0O_AaX+46t_+Ca9gvWMYdWdRtTGeHQ@mail.gmail.com/
Link: https://lore.kernel.org/bpf/20230328004738.381898-3-eddyz87@gmail.com
Signed-off-by: Daniel Müller <deso@posteo.net>
Resolve forward declarations that don't take part in type graphs
comparisons if declaration name is unambiguous. Example:
CU #1:
struct foo; // standalone forward declaration
struct foo *some_global;
CU #2:
struct foo { int x; };
struct foo *another_global;
The `struct foo` from CU #1 is not a part of any definition that is
compared against another definition while `btf_dedup_struct_types`
processes structural types. The the BTF after `btf_dedup_struct_types`
the BTF looks as follows:
[1] STRUCT 'foo' size=4 vlen=1 ...
[2] INT 'int' size=4 ...
[3] PTR '(anon)' type_id=1
[4] FWD 'foo' fwd_kind=struct
[5] PTR '(anon)' type_id=4
This commit adds a new pass `btf_dedup_resolve_fwds`, that maps such
forward declarations to structs or unions with identical name in case
if the name is not ambiguous.
The pass is positioned before `btf_dedup_ref_types` so that types
[3] and [5] could be merged as a same type after [1] and [4] are merged.
The final result for the example above looks as follows:
[1] STRUCT 'foo' size=4 vlen=1
'x' type_id=2 bits_offset=0
[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[3] PTR '(anon)' type_id=1
For defconfig kernel with BTF enabled this removes 63 forward
declarations. Examples of removed declarations: `pt_regs`, `in6_addr`.
The running time of `btf__dedup` function is increased by about 3%.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20221109142611.879983-3-eddyz87@gmail.com
An update for libbpf's hashmap interface from void* -> void* to a
polymorphic one, allowing both long and void* keys and values.
This simplifies many use cases in libbpf as hashmaps there are mostly
integer to integer.
Perf copies hashmap implementation from libbpf and has to be
updated as well.
Changes to libbpf, selftests/bpf and perf are packed as a single
commit to avoid compilation issues with any future bisect.
Polymorphic interface is acheived by hiding hashmap interface
functions behind auxiliary macros that take care of necessary
type casts, for example:
#define hashmap_cast_ptr(p) \
({ \
_Static_assert((p) == NULL || sizeof(*(p)) == sizeof(long),\
#p " pointee should be a long-sized integer or a pointer"); \
(long *)(p); \
})
bool hashmap_find(const struct hashmap *map, long key, long *value);
#define hashmap__find(map, key, value) \
hashmap_find((map), (long)(key), hashmap_cast_ptr(value))
- hashmap__find macro casts key and value parameters to long
and long* respectively
- hashmap_cast_ptr ensures that value pointer points to a memory
of appropriate size.
This hack was suggested by Andrii Nakryiko in [1].
This is a follow up for [2].
[1] https://lore.kernel.org/bpf/CAEf4BzZ8KFneEJxFAaNCCFPGqp20hSpS2aCj76uRk3-qZUH5xg@mail.gmail.com/
[2] https://lore.kernel.org/bpf/af1facf9-7bc8-8a3d-0db4-7b3f333589a2@meta.com/T/#m65b28f1d6d969fcd318b556db6a3ad499a42607d
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20221109142611.879983-2-eddyz87@gmail.com
Changes de-duplication logic for enums in the following way:
- update btf_hash_enum to ignore size and kind fields to get
ENUM and ENUM64 types in a same hash bucket;
- update btf_compat_enum to consider enum fwd to be compatible with
full enum64 (and vice versa);
This allows BTF de-duplication in the following case:
// CU #1
enum foo;
struct s {
enum foo *a;
} *x;
// CU #2
enum foo {
x = 0xfffffffff // big enough to force enum64
};
struct s {
enum foo *a;
} *y;
De-duplicated BTF prior to this commit:
[1] ENUM64 'foo' encoding=UNSIGNED size=8 vlen=1
'x' val=68719476735ULL
[2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64
encoding=(none)
[3] STRUCT 's' size=8 vlen=1
'a' type_id=4 bits_offset=0
[4] PTR '(anon)' type_id=1
[5] PTR '(anon)' type_id=3
[6] STRUCT 's' size=8 vlen=1
'a' type_id=8 bits_offset=0
[7] ENUM 'foo' encoding=UNSIGNED size=4 vlen=0
[8] PTR '(anon)' type_id=7
[9] PTR '(anon)' type_id=6
De-duplicated BTF after this commit:
[1] ENUM64 'foo' encoding=UNSIGNED size=8 vlen=1
'x' val=68719476735ULL
[2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64
encoding=(none)
[3] STRUCT 's' size=8 vlen=1
'a' type_id=4 bits_offset=0
[4] PTR '(anon)' type_id=1
[5] PTR '(anon)' type_id=3
Enum forward declarations in C do not provide information about
enumeration values range. Thus the `btf_type->size` field is
meaningless for forward enum declarations. In fact, GCC does not
encode size in DWARF for forward enum declarations
(but dwarves sets enumeration size to a default value of `sizeof(int) * 8`
when size is not specified see dwarf_loader.c:die__create_new_enumeration).
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20221101235413.1824260-1-eddyz87@gmail.com
btf_dump_emit_struct_def attempts to print empty structures at a
single line, e.g. `struct empty {}`. However, it has to account for a
case when there are no regular but some padding fields in the struct.
In such case `vlen` would be zero, but size would be non-zero.
E.g. here is struct bpf_timer from vmlinux.h before this patch:
struct bpf_timer {
long: 64;
long: 64;};
And after this patch:
struct bpf_dynptr {
long: 64;
long: 64;
};
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20221001104425.415768-1-eddyz87@gmail.com
Eduard Zingerman