Compare commits

...

87 Commits

Author SHA1 Message Date
Andrii Nakryiko
e8547bd4f7 vmtests: fix selftests checkout script
Fix the script.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-08-18 11:37:43 -07:00
Andrii Nakryiko
93959e4e43 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   bfdd5aaa54b0a44d9df550fe4c9db7e1470a11b8
Checkpoint bpf-next commit: 06a4ec1d9dc652e17ee3ac2ceb6c7cf6c2b75cdd
Baseline bpf commit:        929e54a989680c6f134b02293732030b897475dc
Checkpoint bpf commit:      3fb1a96a91120877488071a167d26d76be4be977

Andrii Nakryiko (4):
  libbpf: Fix BTF-defined map-in-map initialization on 32-bit host
    arches
  libbpf: Handle BTF pointer sizes more carefully
  libbpf: Enforce 64-bitness of BTF for BPF object files
  libbpf: Fix build on ppc64le architecture

Jean-Philippe Brucker (1):
  libbpf: Handle GCC built-in types for Arm NEON

Toke Høiland-Jørgensen (1):
  libbpf: Prevent overriding errno when logging errors

Yonghong Song (1):
  libbpf: Do not use __builtin_offsetof for offsetof

 src/bpf_helpers.h |  2 +-
 src/btf.c         | 83 +++++++++++++++++++++++++++++++++++++++++++++--
 src/btf.h         |  2 ++
 src/btf_dump.c    | 39 ++++++++++++++++++++--
 src/libbpf.c      | 32 +++++++++++-------
 src/libbpf.map    |  2 ++
 6 files changed, 143 insertions(+), 17 deletions(-)

--
2.24.1
2020-08-18 11:37:43 -07:00
Andrii Nakryiko
7ee1f12f94 libbpf: Fix build on ppc64le architecture
On ppc64le we get the following warning:

  In file included from btf_dump.c:16:0:
  btf_dump.c: In function ‘btf_dump_emit_struct_def’:
  ../include/linux/kernel.h:20:17: error: comparison of distinct pointer types lacks a cast [-Werror]
    (void) (&_max1 == &_max2);  \
                   ^
  btf_dump.c:882:11: note: in expansion of macro ‘max’
      m_sz = max(0LL, btf__resolve_size(d->btf, m->type));
             ^~~

Fix by explicitly casting to __s64, which is a return type from
btf__resolve_size().

Fixes: 702eddc77a90 ("libbpf: Handle GCC built-in types for Arm NEON")
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200818164456.1181661-1-andriin@fb.com
2020-08-18 11:37:43 -07:00
Andrii Nakryiko
ff09ad9dac libbpf: Enforce 64-bitness of BTF for BPF object files
BPF object files are always targeting 64-bit BPF target architecture, so
enforce that at BTF level as well.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200813204945.1020225-7-andriin@fb.com
2020-08-18 11:37:43 -07:00
Andrii Nakryiko
025fcdc306 libbpf: Handle BTF pointer sizes more carefully
With libbpf and BTF it is pretty common to have libbpf built for one
architecture, while BTF information was generated for a different architecture
(typically, but not always, BPF). In such case, the size of a pointer might
differ betweem architectures. libbpf previously was always making an
assumption that pointer size for BTF is the same as native architecture
pointer size, but that breaks for cases where libbpf is built as 32-bit
library, while BTF is for 64-bit architecture.

To solve this, add heuristic to determine pointer size by searching for `long`
or `unsigned long` integer type and using its size as a pointer size. Also,
allow to override the pointer size with a new API btf__set_pointer_size(), for
cases where application knows which pointer size should be used. User
application can check what libbpf "guessed" by looking at the result of
btf__pointer_size(). If it's not 0, then libbpf successfully determined a
pointer size, otherwise native arch pointer size will be used.

For cases where BTF is parsed from ELF file, use ELF's class (32-bit or
64-bit) to determine pointer size.

Fixes: 8a138aed4a80 ("bpf: btf: Add BTF support to libbpf")
Fixes: 351131b51c7a ("libbpf: add btf_dump API for BTF-to-C conversion")
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200813204945.1020225-5-andriin@fb.com
2020-08-18 11:37:43 -07:00
Andrii Nakryiko
b3405fcb08 libbpf: Fix BTF-defined map-in-map initialization on 32-bit host arches
Libbpf built in 32-bit mode should be careful about not conflating 64-bit BPF
pointers in BPF ELF file and host architecture pointers. This patch fixes
issue of incorrect initializating of map-in-map inner map slots due to such
difference.

Fixes: 646f02ffdd49 ("libbpf: Add BTF-defined map-in-map support")
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200813204945.1020225-4-andriin@fb.com
2020-08-18 11:37:43 -07:00
Toke Høiland-Jørgensen
1194953749 libbpf: Prevent overriding errno when logging errors
Turns out there were a few more instances where libbpf didn't save the
errno before writing an error message, causing errno to be overridden by
the printf() return and the error disappearing if logging is enabled.

Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20200813142905.160381-1-toke@redhat.com
2020-08-18 11:37:43 -07:00
Jean-Philippe Brucker
1d76180057 libbpf: Handle GCC built-in types for Arm NEON
When building Arm NEON (SIMD) code from lib/raid6/neon.uc, GCC emits
DWARF information using a base type "__Poly8_t", which is internal to
GCC and not recognized by Clang. This causes build failures when
building with Clang a vmlinux.h generated from an arm64 kernel that was
built with GCC.

	vmlinux.h:47284:9: error: unknown type name '__Poly8_t'
	typedef __Poly8_t poly8x16_t[16];
	        ^~~~~~~~~

The polyX_t types are defined as unsigned integers in the "Arm C
Language Extension" document (101028_Q220_00_en). Emit typedefs based on
standard integer types for the GCC internal types, similar to those
emitted by Clang.

Including linux/kernel.h to use ARRAY_SIZE() incidentally redefined
max(), causing a build bug due to different types, hence the seemingly
unrelated change.

Reported-by: Jakov Petrina <jakov.petrina@sartura.hr>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20200812143909.3293280-1-jean-philippe@linaro.org
2020-08-18 11:37:43 -07:00
Yonghong Song
048bf21dac libbpf: Do not use __builtin_offsetof for offsetof
Commit 5fbc220862fc ("tools/libpf: Add offsetof/container_of macro
in bpf_helpers.h") added a macro offsetof() to get the offset of a
structure member:

   #define offsetof(TYPE, MEMBER)  ((size_t)&((TYPE *)0)->MEMBER)

In certain use cases, size_t type may not be available so
Commit da7a35062bcc ("libbpf bpf_helpers: Use __builtin_offsetof
for offsetof") changed to use __builtin_offsetof which removed
the dependency on type size_t, which I suggested.

But using __builtin_offsetof will prevent CO-RE relocation
generation in case that, e.g., TYPE is annotated with "preserve_access_info"
where a relocation is desirable in case the member offset is changed
in a different kernel version. So this patch reverted back to
the original macro but using "unsigned long" instead of "site_t".

Fixes: da7a35062bcc ("libbpf bpf_helpers: Use __builtin_offsetof for offsetof")
Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/bpf/20200811030852.3396929-1-yhs@fb.com
2020-08-18 11:37:43 -07:00
Andrii Nakryiko
e954437a76 travis-ci: flatten build stages to gain more speed ups
Do both builds and selftest runs as part of a single build step. This would
allow to complete CI testing faster, as builds will happen in parallel with
"Kernel LATEST + selftests" run.

Also re-enable s390x build.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-08-10 22:38:26 -07:00
Andrii Nakryiko
c57be0b4d6 vmtests: speed up fetching of bpf-next sources
Attempt to first fetch bpf-next tree from a snapshot, falling back to shallow
clone, and if that is not enough, doing a full bpf-next clone. This should
both improve a speed and (because of full clone fallback) improve test
reliability if libbpf wasn't synced in a while.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-08-10 22:31:52 -07:00
Andrii Nakryiko
bf3ab4b0d8 travis-ci: remove s390x build as it fails to be queued by Travis CI
It's been failing for few days. Comment it out for now.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-08-09 13:19:59 -07:00
Andrii Nakryiko
663f66decf vmtests: blacklist problematic tests
Blacklist btf_map_in_map permanently for 5.5. bpf_verif_scale is broken due to
Clang issues on latest. Do not run ALU32 flavor for test_progs on 4.9.0, which
doesn't support ALU32 yet.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-08-07 16:37:09 -07:00
Andrii Nakryiko
ed187d0400 vmtest: bump LLVM_VER to 12
Bump LLVM_VER variable used in selftest build to 12.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-08-07 16:37:09 -07:00
Andrii Nakryiko
80453d4b2d sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   3c4f850e8441ac8b3b6dbaa6107604c4199ef01f
Checkpoint bpf-next commit: bfdd5aaa54b0a44d9df550fe4c9db7e1470a11b8
Baseline bpf commit:        5b801dfb7feb2738975d80223efc2fc193e55573
Checkpoint bpf commit:      929e54a989680c6f134b02293732030b897475dc

Andrii Nakryiko (3):
  libbpf: Make destructors more robust by handling ERR_PTR(err) cases
  libbpf: Add bpf_link detach APIs
  libbpf: Add btf__parse_raw() and generic btf__parse() APIs

Daniel T. Lee (1):
  libbf: Fix uninitialized pointer at btf__parse_raw()

Jerry Crunchtime (1):
  libbpf: Fix register in PT_REGS MIPS macros

Yonghong Song (1):
  tools/bpf: Support new uapi for map element bpf iterator

 include/uapi/linux/bpf.h |  20 ++++---
 src/bpf.c                |  13 +++++
 src/bpf.h                |   7 ++-
 src/bpf_tracing.h        |   4 +-
 src/btf.c                | 118 ++++++++++++++++++++++++++-------------
 src/btf.h                |   5 +-
 src/btf_dump.c           |   2 +-
 src/libbpf.c             |  20 ++++---
 src/libbpf.h             |   6 +-
 src/libbpf.map           |   4 ++
 10 files changed, 137 insertions(+), 62 deletions(-)

--
2.24.1
2020-08-07 16:37:09 -07:00
Daniel T. Lee
7f96c4b1d2 libbf: Fix uninitialized pointer at btf__parse_raw()
Recently, from commit 94a1fedd63ed ("libbpf: Add btf__parse_raw() and
generic btf__parse() APIs"), new API has been added to libbpf that
allows to parse BTF from raw data file (btf__parse_raw()).

The commit derives build failure of samples/bpf due to improper access
of uninitialized pointer at btf_parse_raw().

    btf.c: In function btf__parse_raw:
    btf.c:625:28: error: btf may be used uninitialized in this function
      625 |  return err ? ERR_PTR(err) : btf;
          |         ~~~~~~~~~~~~~~~~~~~^~~~~

This commit fixes the build failure of samples/bpf by adding code of
initializing btf pointer as NULL.

Fixes: 94a1fedd63ed ("libbpf: Add btf__parse_raw() and generic btf__parse() APIs")
Signed-off-by: Daniel T. Lee <danieltimlee@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Link: https://lore.kernel.org/bpf/20200805223359.32109-1-danieltimlee@gmail.com
2020-08-07 16:37:09 -07:00
Yonghong Song
2be293cb4a tools/bpf: Support new uapi for map element bpf iterator
Previous commit adjusted kernel uapi for map
element bpf iterator. This patch adjusted libbpf API
due to uapi change. bpftool and bpf_iter selftests
are also changed accordingly.

Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Link: https://lore.kernel.org/bpf/20200805055058.1457623-1-yhs@fb.com
2020-08-07 16:37:09 -07:00
Andrii Nakryiko
a0334e97aa libbpf: Add btf__parse_raw() and generic btf__parse() APIs
Add public APIs to parse BTF from raw data file (e.g.,
/sys/kernel/btf/vmlinux), as well as generic btf__parse(), which will try to
determine correct format, currently either raw or ELF.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20200802013219.864880-2-andriin@fb.com
2020-08-07 16:37:09 -07:00
Andrii Nakryiko
2d97d4097f libbpf: Add bpf_link detach APIs
Add low-level bpf_link_detach() API. Also add higher-level bpf_link__detach()
one.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Link: https://lore.kernel.org/bpf/20200731182830.286260-3-andriin@fb.com
2020-08-07 16:37:09 -07:00
Jerry Crunchtime
80a52e3252 libbpf: Fix register in PT_REGS MIPS macros
The o32, n32 and n64 calling conventions require the return
value to be stored in $v0 which maps to $2 register, i.e.,
the register 2.

Fixes: c1932cd ("bpf: Add MIPS support to samples/bpf.")
Signed-off-by: Jerry Crunchtime <jerry.c.t@web.de>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/43707d31-0210-e8f0-9226-1af140907641@web.de
2020-08-07 16:37:09 -07:00
Andrii Nakryiko
2dc7cbd893 libbpf: Make destructors more robust by handling ERR_PTR(err) cases
Most of libbpf "constructors" on failure return ERR_PTR(err) result encoded as
a pointer. It's a common mistake to eventually pass such malformed pointers
into xxx__destroy()/xxx__free() "destructors". So instead of fixing up
clean up code in selftests and user programs, handle such error pointers in
destructors themselves. This works beautifully for NULL pointers passed to
destructors, so might as well just work for error pointers.

Suggested-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/20200729232148.896125-1-andriin@fb.com
2020-08-07 16:37:09 -07:00
Thomas Hebb
0466b9833b README: Add Arch to list of downstream distros
Signed-off-by: Thomas Hebb <tommyhebb@gmail.com>
2020-08-06 21:21:02 -07:00
Andrii Nakryiko
ba8d45968b vmtests: specify v12 of clang/llvm for now
Whatever happened, clang-11 and llvm-11, to which clang/llvm packages resolve,
respectively, are not there anymore. Seems like clang-12/llvm-12 are the
latest now, but for whatever reason clang/llvm don't resolve to them yet.
Hard-code version 12 for now.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-08-06 17:32:33 -07:00
Thomas Hebb
734b3f0afe check-reallocarray.sh: Use the same compiler Make does
Currently we hardcode "gcc", which means we get a bogus result any time
a non-default CC is passed to Make. In fact, it's bogus even when CC is
not explicitly set, since Make's default is "cc", which isn't
necessarily the same as "gcc".

Fix the issue by passing the compiler to use to check-reallocarray.sh.

Signed-off-by: Thomas Hebb <tommyhebb@gmail.com>
2020-07-28 14:05:35 -07:00
Andrii Nakryiko
f56874ba8a vmtests: blacklist sk_lookup on LATEST and cg_storage_multi on 5.5
Blacklist two failing tests.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-07-28 14:03:17 -07:00
Andrii Nakryiko
3f26bf1adf sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   9a97c9d2af5ca798377342debf7f0f44281d050e
Checkpoint bpf-next commit: 3c4f850e8441ac8b3b6dbaa6107604c4199ef01f
Baseline bpf commit:        5b801dfb7feb2738975d80223efc2fc193e55573
Checkpoint bpf commit:      5b801dfb7feb2738975d80223efc2fc193e55573

Andrii Nakryiko (1):
  bpf: Fix bpf_ringbuf_output() signature to return long

 include/uapi/linux/bpf.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--
2.24.1
2020-07-28 14:03:17 -07:00
Andrii Nakryiko
ab01213b35 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   5c3320d7fece4612d4a413aa3c8e82cdb5b49fcb
Checkpoint bpf-next commit: 9a97c9d2af5ca798377342debf7f0f44281d050e
Baseline bpf commit:        b2f9f1535bb93ee5fa2ea30ac1c26fa0d676154c
Checkpoint bpf commit:      5b801dfb7feb2738975d80223efc2fc193e55573

Andrii Nakryiko (3):
  libbpf: Support stripping modifiers for btf_dump
  tools/bpftool: Strip away modifiers from global variables
  libbpf: Add support for BPF XDP link

Ciara Loftus (1):
  xsk: Add new statistics

Horatiu Vultur (1):
  net: bridge: Add port attribute IFLA_BRPORT_MRP_IN_OPEN

Ian Rogers (1):
  libbpf bpf_helpers: Use __builtin_offsetof for offsetof

Jakub Sitnicki (2):
  bpf: Sync linux/bpf.h to tools/
  libbpf: Add support for SK_LOOKUP program type

Lorenzo Bianconi (3):
  cpumap: Formalize map value as a named struct
  bpf: cpumap: Add the possibility to attach an eBPF program to cpumap
  libbpf: Add SEC name for xdp programs attached to CPUMAP

Quentin Monnet (1):
  bpf: Fix formatting in documentation for BPF helpers

Randy Dunlap (1):
  bpf: Drop duplicated words in uapi helper comments

Song Liu (1):
  libbpf: Print hint when PERF_EVENT_IOC_SET_BPF returns -EPROTO

Yonghong Song (2):
  bpf: Implement bpf iterator for map elements
  tools/libbpf: Add support for bpf map element iterator

 include/uapi/linux/bpf.h     | 155 +++++++++++++++++++++++++++++------
 include/uapi/linux/if_link.h |   1 +
 include/uapi/linux/if_xdp.h  |   5 +-
 src/bpf.c                    |   1 +
 src/bpf.h                    |   3 +-
 src/bpf_helpers.h            |   2 +-
 src/btf.h                    |   4 +-
 src/btf_dump.c               |  10 ++-
 src/libbpf.c                 |  27 +++++-
 src/libbpf.h                 |   7 +-
 src/libbpf.map               |   3 +
 src/libbpf_probes.c          |   3 +
 12 files changed, 188 insertions(+), 33 deletions(-)

--
2.24.1
2020-07-28 14:03:17 -07:00
Andrii Nakryiko
8af35e73a2 sync: auto-generate latest BPF helpers
Latest changes to BPF helper definitions.
2020-07-28 14:03:17 -07:00
Andrii Nakryiko
a290d45322 libbpf: Add support for BPF XDP link
Sync UAPI header and add support for using bpf_link-based XDP attachment.
Make xdp/ prog type set expected attach type. Kernel didn't enforce
attach_type for XDP programs before, so there is no backwards compatiblity
issues there.

Also fix section_names selftest to recognize that xdp prog types now have
expected attach type.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200722064603.3350758-8-andriin@fb.com
2020-07-28 14:03:17 -07:00
Song Liu
3f6b428909 libbpf: Print hint when PERF_EVENT_IOC_SET_BPF returns -EPROTO
The kernel prevents potential unwinder warnings and crashes by blocking
BPF program with bpf_get_[stack|stackid] on perf_event without
PERF_SAMPLE_CALLCHAIN, or with exclude_callchain_[kernel|user]. Print a
hint message in libbpf to help the user debug such issues.

Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200723180648.1429892-4-songliubraving@fb.com
2020-07-28 14:03:17 -07:00
Yonghong Song
5efd8395ef tools/libbpf: Add support for bpf map element iterator
Add map_fd to bpf_iter_attach_opts and flags to
bpf_link_create_opts. Later on, bpftool or selftest
will be able to create a bpf map element iterator
by passing map_fd to the kernel during link
creation time.

Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200723184117.590673-1-yhs@fb.com
2020-07-28 14:03:17 -07:00
Yonghong Song
b1720407ff bpf: Implement bpf iterator for map elements
The bpf iterator for map elements are implemented.
The bpf program will receive four parameters:
  bpf_iter_meta *meta: the meta data
  bpf_map *map:        the bpf_map whose elements are traversed
  void *key:           the key of one element
  void *value:         the value of the same element

Here, meta and map pointers are always valid, and
key has register type PTR_TO_RDONLY_BUF_OR_NULL and
value has register type PTR_TO_RDWR_BUF_OR_NULL.
The kernel will track the access range of key and value
during verification time. Later, these values will be compared
against the values in the actual map to ensure all accesses
are within range.

A new field iter_seq_info is added to bpf_map_ops which
is used to add map type specific information, i.e., seq_ops,
init/fini seq_file func and seq_file private data size.
Subsequent patches will have actual implementation
for bpf_map_ops->iter_seq_info.

In user space, BPF_ITER_LINK_MAP_FD needs to be
specified in prog attr->link_create.flags, which indicates
that attr->link_create.target_fd is a map_fd.
The reason for such an explicit flag is for possible
future cases where one bpf iterator may allow more than
one possible customization, e.g., pid and cgroup id for
task_file.

Current kernel internal implementation only allows
the target to register at most one required bpf_iter_link_info.
To support the above case, optional bpf_iter_link_info's
are needed, the target can be extended to register such link
infos, and user provided link_info needs to match one of
target supported ones.

Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200723184112.590360-1-yhs@fb.com
2020-07-28 14:03:17 -07:00
Ian Rogers
698820a9d9 libbpf bpf_helpers: Use __builtin_offsetof for offsetof
The non-builtin route for offsetof has a dependency on size_t from
stdlib.h/stdint.h that is undeclared and may break targets.
The offsetof macro in bpf_helpers may disable the same macro in other
headers that have a #ifdef offsetof guard. Rather than add additional
dependencies improve the offsetof macro declared here to use the
builtin that is available since llvm 3.7 (the first with a BPF backend).

Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20200720061741.1514673-1-irogers@google.com
2020-07-28 14:03:17 -07:00
Jakub Sitnicki
6d92249be0 libbpf: Add support for SK_LOOKUP program type
Make libbpf aware of the newly added program type, and assign it a
section name.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20200717103536.397595-13-jakub@cloudflare.com
2020-07-28 14:03:17 -07:00
Jakub Sitnicki
1736996279 bpf: Sync linux/bpf.h to tools/
Newly added program, context type and helper is used by tests in a
subsequent patch. Synchronize the header file.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20200717103536.397595-12-jakub@cloudflare.com
2020-07-28 14:03:17 -07:00
Randy Dunlap
f9f5f054d2 bpf: Drop duplicated words in uapi helper comments
Drop doubled words "will" and "attach".

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/6b9f71ae-4f8e-0259-2c5d-187ddaefe6eb@infradead.org
2020-07-28 14:03:17 -07:00
Lorenzo Bianconi
4a5aecf034 libbpf: Add SEC name for xdp programs attached to CPUMAP
As for DEVMAP, support SEC("xdp_cpumap/") as a short cut for loading
the program with type BPF_PROG_TYPE_XDP and expected attach type
BPF_XDP_CPUMAP.

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/33174c41993a6d860d9c7c1f280a2477ee39ed11.1594734381.git.lorenzo@kernel.org
2020-07-28 14:03:17 -07:00
Lorenzo Bianconi
77f11b3674 bpf: cpumap: Add the possibility to attach an eBPF program to cpumap
Introduce the capability to attach an eBPF program to cpumap entries.
The idea behind this feature is to add the possibility to define on
which CPU run the eBPF program if the underlying hw does not support
RSS. Current supported verdicts are XDP_DROP and XDP_PASS.

This patch has been tested on Marvell ESPRESSObin using xdp_redirect_cpu
sample available in the kernel tree to identify possible performance
regressions. Results show there are no observable differences in
packet-per-second:

$./xdp_redirect_cpu --progname xdp_cpu_map0 --dev eth0 --cpu 1
rx: 354.8 Kpps
rx: 356.0 Kpps
rx: 356.8 Kpps
rx: 356.3 Kpps
rx: 356.6 Kpps
rx: 356.6 Kpps
rx: 356.7 Kpps
rx: 355.8 Kpps
rx: 356.8 Kpps
rx: 356.8 Kpps

Co-developed-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Link: https://lore.kernel.org/bpf/5c9febdf903d810b3415732e5cd98491d7d9067a.1594734381.git.lorenzo@kernel.org
2020-07-28 14:03:17 -07:00
Lorenzo Bianconi
cd46c9d67e cpumap: Formalize map value as a named struct
As it has been already done for devmap, introduce 'struct bpf_cpumap_val'
to formalize the expected values that can be passed in for a CPUMAP.
Update cpumap code to use the struct.

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Link: https://lore.kernel.org/bpf/754f950674665dae6139c061d28c1d982aaf4170.1594734381.git.lorenzo@kernel.org
2020-07-28 14:03:17 -07:00
Horatiu Vultur
41054a32df net: bridge: Add port attribute IFLA_BRPORT_MRP_IN_OPEN
This patch adds a new port attribute, IFLA_BRPORT_MRP_IN_OPEN, which
allows to notify the userspace when the node lost the contiuity of
MRP_InTest frames.

Signed-off-by: Horatiu Vultur <horatiu.vultur@microchip.com>
Acked-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2020-07-28 14:03:17 -07:00
Andrii Nakryiko
852b4c8e73 tools/bpftool: Strip away modifiers from global variables
Reliably remove all the type modifiers from read-only (.rodata) global
variable definitions, including cases of inner field const modifiers and
arrays of const values.

Also modify one of selftests to ensure that const volatile struct doesn't
prevent user-space from modifying .rodata variable.

Fixes: 985ead416df3 ("bpftool: Add skeleton codegen command")
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200713232409.3062144-3-andriin@fb.com
2020-07-28 14:03:17 -07:00
Andrii Nakryiko
de60a31eba libbpf: Support stripping modifiers for btf_dump
One important use case when emitting const/volatile/restrict is undesirable is
BPF skeleton generation of DATASEC layout. These are further memory-mapped and
can be written/read from user-space directly.

For important case of .rodata variables, bpftool strips away first-level
modifiers, to make their use on user-space side simple and not requiring extra
type casts to override compiler complaining about writing to const variables.

This logic works mostly fine, but breaks in some more complicated cases. E.g.:

    const volatile int params[10];

Because in BTF it's a chain of ARRAY -> CONST -> VOLATILE -> INT, bpftool
stops at ARRAY and doesn't strip CONST and VOLATILE. In skeleton this variable
will be emitted as is. So when used from user-space, compiler will complain
about writing to const array. This is problematic, as also mentioned in [0].

To solve this for arrays and other non-trivial cases (e.g., inner
const/volatile fields inside the struct), teach btf_dump to strip away any
modifier, when requested. This is done as an extra option on
btf_dump__emit_type_decl() API.

Reported-by: Anton Protopopov <a.s.protopopov@gmail.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200713232409.3062144-2-andriin@fb.com
2020-07-28 14:03:17 -07:00
Ciara Loftus
8ec7d86efe xsk: Add new statistics
It can be useful for the user to know the reason behind a dropped packet.
Introduce new counters which track drops on the receive path caused by:
1. rx ring being full
2. fill ring being empty

Also, on the tx path introduce a counter which tracks the number of times
we attempt pull from the tx ring when it is empty.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200708072835.4427-2-ciara.loftus@intel.com
2020-07-28 14:03:17 -07:00
Andrii Nakryiko
c3984343bc sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   2977282b63c3b6f112145ecf0bcefff0c65bd3ac
Checkpoint bpf-next commit: 5c3320d7fece4612d4a413aa3c8e82cdb5b49fcb
Baseline bpf commit:        b2f9f1535bb93ee5fa2ea30ac1c26fa0d676154c
Checkpoint bpf commit:      b2f9f1535bb93ee5fa2ea30ac1c26fa0d676154c

Andrii Nakryiko (1):
  libbpf: Fix memory leak and optimize BTF sanitization

 src/btf.c    |  2 +-
 src/btf.h    |  2 +-
 src/libbpf.c | 11 +++--------
 3 files changed, 5 insertions(+), 10 deletions(-)

--
2.24.1
2020-07-10 09:11:41 -07:00
Andrii Nakryiko
5255eb2799 libbpf: Fix memory leak and optimize BTF sanitization
Coverity's static analysis helpfully reported a memory leak introduced by
0f0e55d8247c ("libbpf: Improve BTF sanitization handling"). While fixing it,
I realized that btf__new() already creates a memory copy, so there is no need
to do this. So this patch also fixes misleading btf__new() signature to make
data into a `const void *` input parameter. And it avoids unnecessary memory
allocation and copy in BTF sanitization code altogether.

Fixes: 0f0e55d8247c ("libbpf: Improve BTF sanitization handling")
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20200710011023.1655008-1-andriin@fb.com
2020-07-10 09:11:41 -07:00
Andrii Nakryiko
8b5e81a17a sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   2977282b63c3b6f112145ecf0bcefff0c65bd3ac
Checkpoint bpf-next commit: 2977282b63c3b6f112145ecf0bcefff0c65bd3ac
Baseline bpf commit:        0f57a1e522f413e87852e632f55de4723e511939
Checkpoint bpf commit:      b2f9f1535bb93ee5fa2ea30ac1c26fa0d676154c

Jakub Bogusz (1):
  libbpf: Fix libbpf hashmap on (I)LP32 architectures

 src/hashmap.h | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

--
2.24.1
2020-07-09 22:00:15 -07:00
Jakub Bogusz
cd016d93f7 libbpf: Fix libbpf hashmap on (I)LP32 architectures
On ILP32, 64-bit result was shifted by value calculated for 32-bit long type
and returned value was much outside hashmap capacity.
As advised by Andrii Nakryiko, this patch uses different hashing variant for
architectures with size_t shorter than long long.

Fixes: e3b924224028 ("libbpf: add resizable non-thread safe internal hashmap")
Signed-off-by: Jakub Bogusz <qboosh@pld-linux.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200709225723.1069937-1-andriin@fb.com
2020-07-09 22:00:15 -07:00
Andrii Nakryiko
deaee9541d vmtests: update blacklist for 5.5
Add two tests (sockopt_sk and udp_limit) to blacklist of 5.5 kernel.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-07-08 17:12:53 -07:00
Andrii Nakryiko
daa2c7f851 ci: re-arrange tests to prioritize higher-signal tests
Put selftests in first stage. Put long-running LATEST build & test case first,
so that it can be better parallelized with 4.9 and 5.5.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-07-08 17:12:53 -07:00
Andrii Nakryiko
006904d416 vmtests: whitelist core_retro for 4.9 tests
Add core_retro to whitelist for 4.9, as it is supposed to work on old kernels.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-07-08 17:12:53 -07:00
Andrii Nakryiko
e47ebc895d sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   6b207d66aa9fad0deed13d5f824e1ea193b0a777
Checkpoint bpf-next commit: 2977282b63c3b6f112145ecf0bcefff0c65bd3ac
Baseline bpf commit:        e708e2bd55c921f5bb554fa5837d132a878951cf
Checkpoint bpf commit:      0f57a1e522f413e87852e632f55de4723e511939

Andrii Nakryiko (4):
  libbpf: Make BTF finalization strict
  libbpf: Add btf__set_fd() for more control over loaded BTF FD
  libbpf: Improve BTF sanitization handling
  libbpf: Handle missing BPF_OBJ_GET_INFO_BY_FD gracefully in
    perf_buffer

Stanislav Fomichev (1):
  libbpf: Add support for BPF_CGROUP_INET_SOCK_RELEASE

 include/uapi/linux/bpf.h |   1 +
 src/btf.c                |   7 +-
 src/btf.h                |   1 +
 src/libbpf.c             | 154 ++++++++++++++++++++++-----------------
 src/libbpf.map           |   1 +
 5 files changed, 95 insertions(+), 69 deletions(-)

--
2.24.1
2020-07-08 17:12:53 -07:00
Andrii Nakryiko
3b2837e296 libbpf: Handle missing BPF_OBJ_GET_INFO_BY_FD gracefully in perf_buffer
perf_buffer__new() is relying on BPF_OBJ_GET_INFO_BY_FD availability for few
sanity checks. OBJ_GET_INFO for maps is actually much more recent feature than
perf_buffer support itself, so this causes unnecessary problems on old kernels
before BPF_OBJ_GET_INFO_BY_FD was added.

This patch makes those sanity checks optional and just assumes best if command
is not supported. If user specified something incorrectly (e.g., wrong map
type), kernel will reject it later anyway, except user won't get a nice
explanation as to why it failed. This seems like a good trade off for
supporting perf_buffer on old kernels.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20200708015318.3827358-6-andriin@fb.com
2020-07-08 17:12:53 -07:00
Andrii Nakryiko
90716e9e14 libbpf: Improve BTF sanitization handling
Change sanitization process to preserve original BTF, which might be used by
libbpf itself for Kconfig externs, CO-RE relocs, etc, even if kernel is old
and doesn't support BTF. To achieve that, if libbpf detects the need for BTF
sanitization, it would clone original BTF, sanitize it in-place, attempt to
load it into kernel, and if successful, will preserve loaded BTF FD in
original `struct btf`, while freeing sanitized local copy.

If kernel doesn't support any BTF, original btf and btf_ext will still be
preserved to be used later for CO-RE relocation and other BTF-dependent libbpf
features, which don't dependon kernel BTF support.

Patch takes care to not specify BTF and BTF.ext features when loading BPF
programs and/or maps, if it was detected that kernel doesn't support BTF
features.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20200708015318.3827358-4-andriin@fb.com
2020-07-08 17:12:53 -07:00
Andrii Nakryiko
d5a36e2070 libbpf: Add btf__set_fd() for more control over loaded BTF FD
Add setter for BTF FD to allow application more fine-grained control in more
advanced scenarios. Storing BTF FD inside `struct btf` provides little benefit
and probably would be better done differently (e.g., btf__load() could just
return FD on success), but we are stuck with this due to backwards
compatibility. The main problem is that it's impossible to load BTF and than
free user-space memory, but keep FD intact, because `struct btf` assumes
ownership of that FD upon successful load and will attempt to close it during
btf__free(). To allow callers (e.g., libbpf itself for BTF sanitization) to
have more control over this, add btf__set_fd() to allow to reset FD
arbitrarily, if necessary.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20200708015318.3827358-3-andriin@fb.com
2020-07-08 17:12:53 -07:00
Andrii Nakryiko
133543c202 libbpf: Make BTF finalization strict
With valid ELF and valid BTF, there is no reason (apart from bugs) why BTF
finalization should fail. So make it strict and return error if it fails. This
makes CO-RE relocation more reliable, as they are not going to be just
silently skipped, if BTF finalization failed.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20200708015318.3827358-2-andriin@fb.com
2020-07-08 17:12:53 -07:00
Stanislav Fomichev
abb82202da libbpf: Add support for BPF_CGROUP_INET_SOCK_RELEASE
Add auto-detection for the cgroup/sock_release programs.

Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20200706230128.4073544-3-sdf@google.com
2020-07-08 17:12:53 -07:00
Andrii Nakryiko
5020fdf8fc vmtests: fix 4.9 build
Drop blacklist and instead use a small whitelist of tests that are still
supposed to work on old 4.9 kernel.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-07-07 11:10:16 -07:00
Andrii Nakryiko
a846caca79 vmtests: test no-alu32 variant of test_progs
Add testing of no-alu32 flavor of test_progs.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-07-07 10:41:57 -07:00
Julia Kartseva
1b42b15b5e travis_ci: run tests for 4.9 kernel
Make sure that libbpf sanitizes BTF properly for older kernels.
Add a stage for 4.9.0 kernel in TravisCI.
For now make test failures non-blocking by adding 4.9.0 to `allow_failures`
section.
Blacklist is copy-pasted 5.5.0 kernel blacklist.
2020-07-01 15:38:31 -07:00
Andrii Nakryiko
a2b27a1b62 vmtests: remove custom 5.5 selftest preparetion actions
Now that pre-generated vmlinux.h is used for compilation of non-latest tests,
we don't need custom adjustments for 5.5 kernel selftests. Adjust blacklist
now that those new self-tests are built into test_progs.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-07-01 15:19:18 -07:00
Andrii Nakryiko
7b9d71b21d sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   ca4db6389d611eee2eb7c1dfe710b62d8ea06772
Checkpoint bpf-next commit: 6b207d66aa9fad0deed13d5f824e1ea193b0a777
Baseline bpf commit:        2bdeb3ed547d8822b2566797afa6c2584abdb119
Checkpoint bpf commit:      e708e2bd55c921f5bb554fa5837d132a878951cf

Andrii Nakryiko (1):
  libbpf: Make bpf_endian co-exist with vmlinux.h

Song Liu (1):
  bpf: Introduce helper bpf_get_task_stack()

 include/uapi/linux/bpf.h | 37 +++++++++++++++++++++++++++++++++-
 src/bpf_endian.h         | 43 ++++++++++++++++++++++++++++++++--------
 2 files changed, 71 insertions(+), 9 deletions(-)

--
2.24.1
2020-07-01 14:36:55 -07:00
Andrii Nakryiko
89f7f0796a sync: auto-generate latest BPF helpers
Latest changes to BPF helper definitions.
2020-07-01 14:36:55 -07:00
Song Liu
c054d91247 bpf: Introduce helper bpf_get_task_stack()
Introduce helper bpf_get_task_stack(), which dumps stack trace of given
task. This is different to bpf_get_stack(), which gets stack track of
current task. One potential use case of bpf_get_task_stack() is to call
it from bpf_iter__task and dump all /proc/<pid>/stack to a seq_file.

bpf_get_task_stack() uses stack_trace_save_tsk() instead of
get_perf_callchain() for kernel stack. The benefit of this choice is that
stack_trace_save_tsk() doesn't require changes in arch/. The downside of
using stack_trace_save_tsk() is that stack_trace_save_tsk() dumps the
stack trace to unsigned long array. For 32-bit systems, we need to
translate it to u64 array.

Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20200630062846.664389-3-songliubraving@fb.com
2020-07-01 14:36:55 -07:00
Andrii Nakryiko
9c104b1637 libbpf: Make bpf_endian co-exist with vmlinux.h
Make bpf_endian.h compatible with vmlinux.h. It is a frequent request from
users wanting to use bpf_endian.h in their BPF applications using CO-RE and
vmlinux.h.

To achieve that, re-implement byte swap macros and drop all the header
includes. This way it can be used both with linux header includes, as well as
with a vmlinux.h.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Yonghong Song <yhs@fb.com>
Link: https://lore.kernel.org/bpf/20200630152125.3631920-2-andriin@fb.com
2020-07-01 14:36:55 -07:00
Andrii Nakryiko
d08d57cd91 vmtests: check in vmlinux.h and use it for non-latest builds
Manually generate vmlinux.h based on latest.config to be used for non-latest
selftest build. This will keep bpftool and newest selftests builds succeeding,
while at runtime blacklist will skip them.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-06-30 18:09:33 -07:00
Andrii Nakryiko
803243cc33 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   b3eece09e2e69f528a1ab6104861550dec149083
Checkpoint bpf-next commit: afa12644c877d3f627281bb6493d7ca8f9976e3d
Baseline bpf commit:        4e15507fea70c0c312d79610efa46b6853ccf8e0
Checkpoint bpf commit:      2bdeb3ed547d8822b2566797afa6c2584abdb119

Andrii Nakryiko (4):
  bpf: Switch most helper return values from 32-bit int to 64-bit long
  libbpf: Prevent loading vmlinux BTF twice
  libbpf: Support disabling auto-loading BPF programs
  libbpf: Fix CO-RE relocs against .text section

Colin Ian King (1):
  libbpf: Fix spelling mistake "kallasyms" -> "kallsyms"

Dmitry Yakunin (1):
  bpf: Add SO_KEEPALIVE and related options to bpf_setsockopt

Jesper Dangaard Brouer (1):
  libbpf: Adjust SEC short cut for expected attach type BPF_XDP_DEVMAP

Quentin Monnet (1):
  bpf: Fix formatting in documentation for BPF helpers

Yonghong Song (3):
  bpf: Add bpf_skc_to_tcp6_sock() helper
  bpf: Add bpf_skc_to_{tcp, tcp_timewait, tcp_request}_sock() helpers
  bpf: Add bpf_skc_to_udp6_sock() helper

 include/uapi/linux/bpf.h | 277 ++++++++++++++++++++++-----------------
 src/libbpf.c             |  93 +++++++++----
 src/libbpf.h             |   2 +
 src/libbpf.map           |   2 +
 4 files changed, 233 insertions(+), 141 deletions(-)

--
2.24.1
2020-06-29 13:46:33 -07:00
Andrii Nakryiko
d707f8027b sync: auto-generate latest BPF helpers
Latest changes to BPF helper definitions.
2020-06-29 13:46:33 -07:00
Jesper Dangaard Brouer
652f2c0a40 libbpf: Adjust SEC short cut for expected attach type BPF_XDP_DEVMAP
Adjust the SEC("xdp_devmap/") prog type prefix to contain a
slash "/" for expected attach type BPF_XDP_DEVMAP.  This is consistent
with other prog types like tracing.

Fixes: 2778797037a6 ("libbpf: Add SEC name for xdp programs attached to device map")
Suggested-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/159309521882.821855.6873145686353617509.stgit@firesoul
2020-06-29 13:46:33 -07:00
Quentin Monnet
2fcd394505 bpf: Fix formatting in documentation for BPF helpers
When producing the bpf-helpers.7 man page from the documentation from
the BPF user space header file, rst2man complains:

    <stdin>:2636: (ERROR/3) Unexpected indentation.
    <stdin>:2640: (WARNING/2) Block quote ends without a blank line; unexpected unindent.

Let's fix formatting for the relevant chunk (item list in
bpf_ringbuf_query()'s description), and for a couple other functions.

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20200623153935.6215-1-quentin@isovalent.com
2020-06-29 13:46:33 -07:00
Andrii Nakryiko
af3c9f9fc4 libbpf: Fix CO-RE relocs against .text section
bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't
return .text "BPF program", if it is a function storage for sub-programs.
Because of that, any CO-RE relocation in helper non-inlined functions will
fail. Fix this by searching for .text-corresponding BPF program manually.

Adjust one of bpf_iter selftest to exhibit this pattern.

Fixes: ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")
Reported-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Yonghong Song <yhs@fb.com>
Link: https://lore.kernel.org/bpf/20200619230423.691274-1-andriin@fb.com
2020-06-29 13:46:33 -07:00
Andrii Nakryiko
a62b08dd0c libbpf: Support disabling auto-loading BPF programs
Currently, bpf_object__load() (and by induction skeleton's load), will always
attempt to prepare, relocate, and load into kernel every single BPF program
found inside the BPF object file. This is often convenient and the right thing
to do and what users expect.

But there are plenty of cases (especially with BPF development constantly
picking up the pace), where BPF application is intended to work with old
kernels, with potentially reduced set of features. But on kernels supporting
extra features, it would like to take a full advantage of them, by employing
extra BPF program. This could be a choice of using fentry/fexit over
kprobe/kretprobe, if kernel is recent enough and is built with BTF. Or BPF
program might be providing optimized bpf_iter-based solution that user-space
might want to use, whenever available. And so on.

With libbpf and BPF CO-RE in particular, it's advantageous to not have to
maintain two separate BPF object files to achieve this. So to enable such use
cases, this patch adds ability to request not auto-loading chosen BPF
programs. In such case, libbpf won't attempt to perform relocations (which
might fail due to old kernel), won't try to resolve BTF types for
BTF-aware (tp_btf/fentry/fexit/etc) program types, because BTF might not be
present, and so on. Skeleton will also automatically skip auto-attachment step
for such not loaded BPF programs.

Overall, this feature allows to simplify development and deployment of
real-world BPF applications with complicated compatibility requirements.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200625232629.3444003-2-andriin@fb.com
2020-06-29 13:46:33 -07:00
Yonghong Song
318ed9d544 bpf: Add bpf_skc_to_udp6_sock() helper
The helper is used in tracing programs to cast a socket
pointer to a udp6_sock pointer.
The return value could be NULL if the casting is illegal.

Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Cc: Eric Dumazet <edumazet@google.com>
Link: https://lore.kernel.org/bpf/20200623230815.3988481-1-yhs@fb.com
2020-06-29 13:46:33 -07:00
Yonghong Song
47370741be bpf: Add bpf_skc_to_{tcp, tcp_timewait, tcp_request}_sock() helpers
Three more helpers are added to cast a sock_common pointer to
an tcp_sock, tcp_timewait_sock or a tcp_request_sock for
tracing programs.

Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20200623230811.3988277-1-yhs@fb.com
2020-06-29 13:46:33 -07:00
Yonghong Song
26e5e7dcb0 bpf: Add bpf_skc_to_tcp6_sock() helper
The helper is used in tracing programs to cast a socket
pointer to a tcp6_sock pointer.
The return value could be NULL if the casting is illegal.

A new helper return type RET_PTR_TO_BTF_ID_OR_NULL is added
so the verifier is able to deduce proper return types for the helper.

Different from the previous BTF_ID based helpers,
the bpf_skc_to_tcp6_sock() argument can be several possible
btf_ids. More specifically, all possible socket data structures
with sock_common appearing in the first in the memory layout.
This patch only added socket types related to tcp and udp.

All possible argument btf_id and return value btf_id
for helper bpf_skc_to_tcp6_sock() are pre-calculcated and
cached. In the future, it is even possible to precompute
these btf_id's at kernel build time.

Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20200623230809.3988195-1-yhs@fb.com
2020-06-29 13:46:33 -07:00
Dmitry Yakunin
cd469e21e8 bpf: Add SO_KEEPALIVE and related options to bpf_setsockopt
This patch adds support of SO_KEEPALIVE flag and TCP related options
to bpf_setsockopt() routine. This is helpful if we want to enable or tune
TCP keepalive for applications which don't do it in the userspace code.

v3:
  - update kernel-doc in uapi (Nikita Vetoshkin <nekto0n@yandex-team.ru>)

v4:
  - update kernel-doc in tools too (Alexei Starovoitov)
  - add test to selftests (Alexei Starovoitov)

Signed-off-by: Dmitry Yakunin <zeil@yandex-team.ru>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20200620153052.9439-3-zeil@yandex-team.ru
2020-06-29 13:46:33 -07:00
Andrii Nakryiko
18bfe12dc1 libbpf: Prevent loading vmlinux BTF twice
Prevent loading/parsing vmlinux BTF twice in some cases: for CO-RE relocations
and for BTF-aware hooks (tp_btf, fentry/fexit, etc).

Fixes: a6ed02cac690 ("libbpf: Load btf_vmlinux only once per object.")
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20200624043805.1794620-1-andriin@fb.com
2020-06-29 13:46:33 -07:00
Colin Ian King
fef856084a libbpf: Fix spelling mistake "kallasyms" -> "kallsyms"
There is a spelling mistake in a pr_warn message. Fix it.

Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20200623084207.149253-1-colin.king@canonical.com
2020-06-29 13:46:33 -07:00
Andrii Nakryiko
6f8e021c3c bpf: Switch most helper return values from 32-bit int to 64-bit long
Switch most of BPF helper definitions from returning int to long. These
definitions are coming from comments in BPF UAPI header and are used to
generate bpf_helper_defs.h (under libbpf) to be later included and used from
BPF programs.

In actual in-kernel implementation, all the helpers are defined as returning
u64, but due to some historical reasons, most of them are actually defined as
returning int in UAPI (usually, to return 0 on success, and negative value on
error).

This actually causes Clang to quite often generate sub-optimal code, because
compiler believes that return value is 32-bit, and in a lot of cases has to be
up-converted (usually with a pair of 32-bit bit shifts) to 64-bit values,
before they can be used further in BPF code.

Besides just "polluting" the code, these 32-bit shifts quite often cause
problems for cases in which return value matters. This is especially the case
for the family of bpf_probe_read_str() functions. There are few other similar
helpers (e.g., bpf_read_branch_records()), in which return value is used by
BPF program logic to record variable-length data and process it. For such
cases, BPF program logic carefully manages offsets within some array or map to
read variable-length data. For such uses, it's crucial for BPF verifier to
track possible range of register values to prove that all the accesses happen
within given memory bounds. Those extraneous zero-extending bit shifts,
inserted by Clang (and quite often interleaved with other code, which makes
the issues even more challenging and sometimes requires employing extra
per-variable compiler barriers), throws off verifier logic and makes it mark
registers as having unknown variable offset. We'll study this pattern a bit
later below.

Another common pattern is to check return of BPF helper for non-zero state to
detect error conditions and attempt alternative actions in such case. Even in
this simple and straightforward case, this 32-bit vs BPF's native 64-bit mode
quite often leads to sub-optimal and unnecessary extra code. We'll look at
this pattern as well.

Clang's BPF target supports two modes of code generation: ALU32, in which it
is capable of using lower 32-bit parts of registers, and no-ALU32, in which
only full 64-bit registers are being used. ALU32 mode somewhat mitigates the
above described problems, but not in all cases.

This patch switches all the cases in which BPF helpers return 0 or negative
error from returning int to returning long. It is shown below that such change
in definition leads to equivalent or better code. No-ALU32 mode benefits more,
but ALU32 mode doesn't degrade or still gets improved code generation.

Another class of cases switched from int to long are bpf_probe_read_str()-like
helpers, which encode successful case as non-negative values, while still
returning negative value for errors.

In all of such cases, correctness is preserved due to two's complement
encoding of negative values and the fact that all helpers return values with
32-bit absolute value. Two's complement ensures that for negative values
higher 32 bits are all ones and when truncated, leave valid negative 32-bit
value with the same value. Non-negative values have upper 32 bits set to zero
and similarly preserve value when high 32 bits are truncated. This means that
just casting to int/u32 is correct and efficient (and in ALU32 mode doesn't
require any extra shifts).

To minimize the chances of regressions, two code patterns were investigated,
as mentioned above. For both patterns, BPF assembly was analyzed in
ALU32/NO-ALU32 compiler modes, both with current 32-bit int return type and
new 64-bit long return type.

Case 1. Variable-length data reading and concatenation. This is quite
ubiquitous pattern in tracing/monitoring applications, reading data like
process's environment variables, file path, etc. In such case, many pieces of
string-like variable-length data are read into a single big buffer, and at the
end of the process, only a part of array containing actual data is sent to
user-space for further processing. This case is tested in test_varlen.c
selftest (in the next patch). Code flow is roughly as follows:

  void *payload = &sample->payload;
  u64 len;

  len = bpf_probe_read_kernel_str(payload, MAX_SZ1, &source_data1);
  if (len <= MAX_SZ1) {
      payload += len;
      sample->len1 = len;
  }
  len = bpf_probe_read_kernel_str(payload, MAX_SZ2, &source_data2);
  if (len <= MAX_SZ2) {
      payload += len;
      sample->len2 = len;
  }
  /* and so on */
  sample->total_len = payload - &sample->payload;
  /* send over, e.g., perf buffer */

There could be two variations with slightly different code generated: when len
is 64-bit integer and when it is 32-bit integer. Both variations were analysed.
BPF assembly instructions between two successive invocations of
bpf_probe_read_kernel_str() were used to check code regressions. Results are
below, followed by short analysis. Left side is using helpers with int return
type, the right one is after the switch to long.

ALU32 + INT                                ALU32 + LONG
===========                                ============

64-BIT (13 insns):                         64-BIT (10 insns):
------------------------------------       ------------------------------------
  17:   call 115                             17:   call 115
  18:   if w0 > 256 goto +9 <LBB0_4>         18:   if r0 > 256 goto +6 <LBB0_4>
  19:   w1 = w0                              19:   r1 = 0 ll
  20:   r1 <<= 32                            21:   *(u64 *)(r1 + 0) = r0
  21:   r1 s>>= 32                           22:   r6 = 0 ll
  22:   r2 = 0 ll                            24:   r6 += r0
  24:   *(u64 *)(r2 + 0) = r1              00000000000000c8 <LBB0_4>:
  25:   r6 = 0 ll                            25:   r1 = r6
  27:   r6 += r1                             26:   w2 = 256
00000000000000e0 <LBB0_4>:                   27:   r3 = 0 ll
  28:   r1 = r6                              29:   call 115
  29:   w2 = 256
  30:   r3 = 0 ll
  32:   call 115

32-BIT (11 insns):                         32-BIT (12 insns):
------------------------------------       ------------------------------------
  17:   call 115                             17:   call 115
  18:   if w0 > 256 goto +7 <LBB1_4>         18:   if w0 > 256 goto +8 <LBB1_4>
  19:   r1 = 0 ll                            19:   r1 = 0 ll
  21:   *(u32 *)(r1 + 0) = r0                21:   *(u32 *)(r1 + 0) = r0
  22:   w1 = w0                              22:   r0 <<= 32
  23:   r6 = 0 ll                            23:   r0 >>= 32
  25:   r6 += r1                             24:   r6 = 0 ll
00000000000000d0 <LBB1_4>:                   26:   r6 += r0
  26:   r1 = r6                            00000000000000d8 <LBB1_4>:
  27:   w2 = 256                             27:   r1 = r6
  28:   r3 = 0 ll                            28:   w2 = 256
  30:   call 115                             29:   r3 = 0 ll
                                             31:   call 115

In ALU32 mode, the variant using 64-bit length variable clearly wins and
avoids unnecessary zero-extension bit shifts. In practice, this is even more
important and good, because BPF code won't need to do extra checks to "prove"
that payload/len are within good bounds.

32-bit len is one instruction longer. Clang decided to do 64-to-32 casting
with two bit shifts, instead of equivalent `w1 = w0` assignment. The former
uses extra register. The latter might potentially lose some range information,
but not for 32-bit value. So in this case, verifier infers that r0 is [0, 256]
after check at 18:, and shifting 32 bits left/right keeps that range intact.
We should probably look into Clang's logic and see why it chooses bitshifts
over sub-register assignments for this.

NO-ALU32 + INT                             NO-ALU32 + LONG
==============                             ===============

64-BIT (14 insns):                         64-BIT (10 insns):
------------------------------------       ------------------------------------
  17:   call 115                             17:   call 115
  18:   r0 <<= 32                            18:   if r0 > 256 goto +6 <LBB0_4>
  19:   r1 = r0                              19:   r1 = 0 ll
  20:   r1 >>= 32                            21:   *(u64 *)(r1 + 0) = r0
  21:   if r1 > 256 goto +7 <LBB0_4>         22:   r6 = 0 ll
  22:   r0 s>>= 32                           24:   r6 += r0
  23:   r1 = 0 ll                          00000000000000c8 <LBB0_4>:
  25:   *(u64 *)(r1 + 0) = r0                25:   r1 = r6
  26:   r6 = 0 ll                            26:   r2 = 256
  28:   r6 += r0                             27:   r3 = 0 ll
00000000000000e8 <LBB0_4>:                   29:   call 115
  29:   r1 = r6
  30:   r2 = 256
  31:   r3 = 0 ll
  33:   call 115

32-BIT (13 insns):                         32-BIT (13 insns):
------------------------------------       ------------------------------------
  17:   call 115                             17:   call 115
  18:   r1 = r0                              18:   r1 = r0
  19:   r1 <<= 32                            19:   r1 <<= 32
  20:   r1 >>= 32                            20:   r1 >>= 32
  21:   if r1 > 256 goto +6 <LBB1_4>         21:   if r1 > 256 goto +6 <LBB1_4>
  22:   r2 = 0 ll                            22:   r2 = 0 ll
  24:   *(u32 *)(r2 + 0) = r0                24:   *(u32 *)(r2 + 0) = r0
  25:   r6 = 0 ll                            25:   r6 = 0 ll
  27:   r6 += r1                             27:   r6 += r1
00000000000000e0 <LBB1_4>:                 00000000000000e0 <LBB1_4>:
  28:   r1 = r6                              28:   r1 = r6
  29:   r2 = 256                             29:   r2 = 256
  30:   r3 = 0 ll                            30:   r3 = 0 ll
  32:   call 115                             32:   call 115

In NO-ALU32 mode, for the case of 64-bit len variable, Clang generates much
superior code, as expected, eliminating unnecessary bit shifts. For 32-bit
len, code is identical.

So overall, only ALU-32 32-bit len case is more-or-less equivalent and the
difference stems from internal Clang decision, rather than compiler lacking
enough information about types.

Case 2. Let's look at the simpler case of checking return result of BPF helper
for errors. The code is very simple:

  long bla;
  if (bpf_probe_read_kenerl(&bla, sizeof(bla), 0))
      return 1;
  else
      return 0;

ALU32 + CHECK (9 insns)                    ALU32 + CHECK (9 insns)
====================================       ====================================
  0:    r1 = r10                             0:    r1 = r10
  1:    r1 += -8                             1:    r1 += -8
  2:    w2 = 8                               2:    w2 = 8
  3:    r3 = 0                               3:    r3 = 0
  4:    call 113                             4:    call 113
  5:    w1 = w0                              5:    r1 = r0
  6:    w0 = 1                               6:    w0 = 1
  7:    if w1 != 0 goto +1 <LBB2_2>          7:    if r1 != 0 goto +1 <LBB2_2>
  8:    w0 = 0                               8:    w0 = 0
0000000000000048 <LBB2_2>:                 0000000000000048 <LBB2_2>:
  9:    exit                                 9:    exit

Almost identical code, the only difference is the use of full register
assignment (r1 = r0) vs half-registers (w1 = w0) in instruction #5. On 32-bit
architectures, new BPF assembly might be slightly less optimal, in theory. But
one can argue that's not a big issue, given that use of full registers is
still prevalent (e.g., for parameter passing).

NO-ALU32 + CHECK (11 insns)                NO-ALU32 + CHECK (9 insns)
====================================       ====================================
  0:    r1 = r10                             0:    r1 = r10
  1:    r1 += -8                             1:    r1 += -8
  2:    r2 = 8                               2:    r2 = 8
  3:    r3 = 0                               3:    r3 = 0
  4:    call 113                             4:    call 113
  5:    r1 = r0                              5:    r1 = r0
  6:    r1 <<= 32                            6:    r0 = 1
  7:    r1 >>= 32                            7:    if r1 != 0 goto +1 <LBB2_2>
  8:    r0 = 1                               8:    r0 = 0
  9:    if r1 != 0 goto +1 <LBB2_2>        0000000000000048 <LBB2_2>:
 10:    r0 = 0                               9:    exit
0000000000000058 <LBB2_2>:
 11:    exit

NO-ALU32 is a clear improvement, getting rid of unnecessary zero-extension bit
shifts.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20200623032224.4020118-1-andriin@fb.com
2020-06-29 13:46:33 -07:00
Andrii Nakryiko
143213eb82 README: info on routing general BPF/libbpf quesions
We keep getting more and more questions about BPF/libbpf usage.
This repo is not the right place to ask them, as not that many people
monitor it. Re-route folks to bpf@vger.kernel.org

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-06-22 20:52:42 -07:00
Andrii Nakryiko
ac74ee188d sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   1bdb6c9a1c43fdf9b83b2331dfc6229bd2e71d9b
Checkpoint bpf-next commit: b3eece09e2e69f528a1ab6104861550dec149083
Baseline bpf commit:        4e15507fea70c0c312d79610efa46b6853ccf8e0
Checkpoint bpf commit:      4e15507fea70c0c312d79610efa46b6853ccf8e0

Andrii Nakryiko (3):
  libbpf: Generalize libbpf externs support
  libbpf: Add support for extracting kernel symbol addresses
  libbpf: Wrap source argument of BPF_CORE_READ macro in parentheses

 src/bpf_core_read.h |   8 +-
 src/bpf_helpers.h   |   1 +
 src/btf.h           |   5 +
 src/libbpf.c        | 482 +++++++++++++++++++++++++++++++-------------
 4 files changed, 350 insertions(+), 146 deletions(-)

--
2.24.1
2020-06-22 20:31:52 -07:00
Andrii Nakryiko
15943906dc libbpf: Wrap source argument of BPF_CORE_READ macro in parentheses
Wrap source argument of BPF_CORE_READ family of macros into parentheses to
allow uses like this:

BPF_CORE_READ((struct cast_struct *)src, a, b, c);

Fixes: 7db3822ab991 ("libbpf: Add BPF_CORE_READ/BPF_CORE_READ_INTO helpers")
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200619231703.738941-8-andriin@fb.com
2020-06-22 20:31:52 -07:00
Andrii Nakryiko
85749135a6 libbpf: Add support for extracting kernel symbol addresses
Add support for another (in addition to existing Kconfig) special kind of
externs in BPF code, kernel symbol externs. Such externs allow BPF code to
"know" kernel symbol address and either use it for comparisons with kernel
data structures (e.g., struct file's f_op pointer, to distinguish different
kinds of file), or, with the help of bpf_probe_user_kernel(), to follow
pointers and read data from global variables. Kernel symbol addresses are
found through /proc/kallsyms, which should be present in the system.

Currently, such kernel symbol variables are typeless: they have to be defined
as `extern const void <symbol>` and the only operation you can do (in C code)
with them is to take its address. Such extern should reside in a special
section '.ksyms'. bpf_helpers.h header provides __ksym macro for this. Strong
vs weak semantics stays the same as with Kconfig externs. If symbol is not
found in /proc/kallsyms, this will be a failure for strong (non-weak) extern,
but will be defaulted to 0 for weak externs.

If the same symbol is defined multiple times in /proc/kallsyms, then it will
be error if any of the associated addresses differs. In that case, address is
ambiguous, so libbpf falls on the side of caution, rather than confusing user
with randomly chosen address.

In the future, once kernel is extended with variables BTF information, such
ksym externs will be supported in a typed version, which will allow BPF
program to read variable's contents directly, similarly to how it's done for
fentry/fexit input arguments.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Hao Luo <haoluo@google.com>
Link: https://lore.kernel.org/bpf/20200619231703.738941-3-andriin@fb.com
2020-06-22 20:31:52 -07:00
Andrii Nakryiko
3b320677cd libbpf: Generalize libbpf externs support
Switch existing Kconfig externs to be just one of few possible kinds of more
generic externs. This refactoring is in preparation for ksymbol extern
support, added in the follow up patch. There are no functional changes
intended.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Hao Luo <haoluo@google.com>
Link: https://lore.kernel.org/bpf/20200619231703.738941-2-andriin@fb.com
2020-06-22 20:31:52 -07:00
Andrii Nakryiko
15fee53503 vmtests: blacklist test using RINGBUF
Test was updated to use BPF_MAP_TYPE_RINGBUF, which is only available starting
from 5.8 version.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2020-06-22 17:11:04 -07:00
Andrii Nakryiko
169d35c746 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   69119673bd50b176ded34032fadd41530fb5af21
Checkpoint bpf-next commit: 1bdb6c9a1c43fdf9b83b2331dfc6229bd2e71d9b
Baseline bpf commit:        4e15507fea70c0c312d79610efa46b6853ccf8e0
Checkpoint bpf commit:      4e15507fea70c0c312d79610efa46b6853ccf8e0

Andrii Nakryiko (2):
  libbpf: Bump version to 0.1.0
  libbpf: Add a bunch of attribute getters/setters for map definitions

 src/libbpf.c   | 100 +++++++++++++++++++++++++++++++++++++++++++++----
 src/libbpf.h   |  30 +++++++++++++--
 src/libbpf.map |  17 +++++++++
 3 files changed, 137 insertions(+), 10 deletions(-)

--
2.24.1
2020-06-22 17:11:04 -07:00
Andrii Nakryiko
d8d4713476 libbpf: Add a bunch of attribute getters/setters for map definitions
Add a bunch of getter for various aspects of BPF map. Some of these attribute
(e.g., key_size, value_size, type, etc) are available right now in struct
bpf_map_def, but this patch adds getter allowing to fetch them individually.
bpf_map_def approach isn't very scalable, when ABI stability requirements are
taken into account. It's much easier to extend libbpf and add support for new
features, when each aspect of BPF map has separate getter/setter.

Getters follow the common naming convention of not explicitly having "get" in
its name: bpf_map__type() returns map type, bpf_map__key_size() returns
key_size. Setters, though, explicitly have set in their name:
bpf_map__set_type(), bpf_map__set_key_size().

This patch ensures we now have a getter and a setter for the following
map attributes:
  - type;
  - max_entries;
  - map_flags;
  - numa_node;
  - key_size;
  - value_size;
  - ifindex.

bpf_map__resize() enforces unnecessary restriction of max_entries > 0. It is
unnecessary, because libbpf actually supports zero max_entries for some cases
(e.g., for PERF_EVENT_ARRAY map) and treats it specially during map creation
time. To allow setting max_entries=0, new bpf_map__set_max_entries() setter is
added. bpf_map__resize()'s behavior is preserved for backwards compatibility
reasons.

Map ifindex getter is added as well. There is a setter already, but no
corresponding getter. Fix this assymetry as well. bpf_map__set_ifindex()
itself is converted from void function into error-returning one, similar to
other setters. The only error returned right now is -EBUSY, if BPF map is
already loaded and has corresponding FD.

One lacking attribute with no ability to get/set or even specify it
declaratively is numa_node. This patch fixes this gap and both adds
programmatic getter/setter, as well as adds support for numa_node field in
BTF-defined map.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Toke Høiland-Jørgensen <toke@redhat.com>
Link: https://lore.kernel.org/bpf/20200621062112.3006313-1-andriin@fb.com
2020-06-22 17:11:04 -07:00
Andrii Nakryiko
ef26b4c37f libbpf: Bump version to 0.1.0
Bump libbpf version to 0.1.0, as new development cycle starts.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200617183132.1970836-1-andriin@fb.com
2020-06-22 17:11:04 -07:00
34 changed files with 83711 additions and 645 deletions

View File

@@ -35,8 +35,23 @@ stages:
jobs:
include:
- stage: Build & Test
name: Debian Build
- stage: Builds & Tests
name: Kernel LATEST + selftests
language: bash
env: KERNEL=LATEST
script: $CI_ROOT/vmtest/run_vmtest.sh || travis_terminate 1
- name: Kernel 4.9.0 + selftests
language: bash
env: KERNEL=4.9.0
script: $CI_ROOT/vmtest/run_vmtest.sh || travis_terminate 1
- name: Kernel 5.5.0 + selftests
language: bash
env: KERNEL=5.5.0
script: $CI_ROOT/vmtest/run_vmtest.sh || travis_terminate 1
- name: Debian Build
language: bash
install: $CI_ROOT/managers/debian.sh SETUP
script: $CI_ROOT/managers/debian.sh RUN || travis_terminate 1
@@ -91,16 +106,6 @@ jobs:
language: bash
script: sudo $CI_ROOT/managers/ubuntu.sh || travis_terminate 1
- name: Kernel 5.5.0 + selftests
language: bash
env: KERNEL=5.5.0
script: $CI_ROOT/vmtest/run_vmtest.sh || travis_terminate 1
- name: Kernel LATEST + selftests
language: bash
env: KERNEL=LATEST
script: $CI_ROOT/vmtest/run_vmtest.sh || travis_terminate 1
- stage: Coverity
language: bash
env:
@@ -121,3 +126,5 @@ jobs:
- sudo apt-get install -y libelf-dev pkg-config
script:
- scripts/coverity.sh || travis_terminate 1
allow_failures:
- env: KERNEL=x.x.x

View File

@@ -1 +1 @@
4e15507fea70c0c312d79610efa46b6853ccf8e0
3fb1a96a91120877488071a167d26d76be4be977

View File

@@ -1 +1 @@
69119673bd50b176ded34032fadd41530fb5af21
06a4ec1d9dc652e17ee3ac2ceb6c7cf6c2b75cdd

View File

@@ -1,20 +1,29 @@
This is a mirror of [bpf-next linux tree](https://kernel.googlesource.com/pub/scm/linux/kernel/git/bpf/bpf-next)'s
This is a mirror of [bpf-next Linux source
tree](https://kernel.googlesource.com/pub/scm/linux/kernel/git/bpf/bpf-next)'s
`tools/lib/bpf` directory plus its supporting header files.
The following files will by sync'ed with bpf-next repo:
- `src/` <-> `bpf-next/tools/lib/bpf/`
- `include/uapi/linux/bpf_common.h` <-> `bpf-next/tools/include/uapi/linux/bpf_common.h`
- `include/uapi/linux/bpf.h` <-> `bpf-next/tools/include/uapi/linux/bpf.h`
- `include/uapi/linux/btf.h` <-> `bpf-next/tools/include/uapi/linux/btf.h`
- `include/uapi/linux/if_link.h` <-> `bpf-next/tools/include/uapi/linux/if_link.h`
- `include/uapi/linux/if_xdp.h` <-> `bpf-next/tools/include/uapi/linux/if_xdp.h`
- `include/uapi/linux/netlink.h` <-> `bpf-next/tools/include/uapi/linux/netlink.h`
- `include/tools/libc_compat.h` <-> `bpf-next/tools/include/tools/libc_compat.h`
All the gory details of syncing can be found in `scripts/sync-kernel.sh`
script.
Other header files at this repo (`include/linux/*.h`) are reduced versions of
their counterpart files at bpf-next's `tools/include/linux/*.h` to make compilation
successful.
Some header files in this repo (`include/linux/*.h`) are reduced versions of
their counterpart files at
[bpf-next](https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/)'s
`tools/include/linux/*.h` to make compilation successful.
BPF questions
=============
All general BPF questions, including kernel functionality, libbpf APIs and
their application, should be sent to bpf@vger.kernel.org mailing list. You can
subscribe to it [here](http://vger.kernel.org/vger-lists.html#bpf) and search
its archive [here](https://lore.kernel.org/bpf/). Please search the archive
before asking new questions. It very well might be that this was already
addressed or answered before.
bpf@vger.kernel.org is monitored by many more people and they will happily try
to help you with whatever issue you have. This repository's PRs and issues
should be opened only for dealing with issues pertaining to specific way this
libbpf mirror repo is set up and organized.
Build
[![Build Status](https://travis-ci.org/libbpf/libbpf.svg?branch=master)](https://travis-ci.org/libbpf/libbpf)
@@ -25,8 +34,9 @@ libelf is an internal dependency of libbpf and thus it is required to link
against and must be installed on the system for applications to work.
pkg-config is used by default to find libelf, and the program called can be
overridden with `PKG_CONFIG`.
If using `pkg-config` at build time is not desired, it can be disabled by setting
`NO_PKG_CONFIG=1` when calling make.
If using `pkg-config` at build time is not desired, it can be disabled by
setting `NO_PKG_CONFIG=1` when calling make.
To build both static libbpf.a and shared libbpf.so:
```bash
@@ -58,17 +68,20 @@ Distributions packaging libbpf from this mirror:
- [Fedora](https://src.fedoraproject.org/rpms/libbpf)
- [Gentoo](https://packages.gentoo.org/packages/dev-libs/libbpf)
- [Debian](https://packages.debian.org/sid/libbpf-dev)
- [Arch](https://www.archlinux.org/packages/extra/x86_64/libbpf/)
Benefits of packaging from the mirror over packaging from kernel sources:
- Consistent versioning across distributions.
- No ties to any specific kernel, transparent handling of older kernels.
Libbpf is designed to be kernel-agnostic and work across multitude of kernel
versions. It has built-in mechanisms to gracefully handle older kernels,
that are missing some of the features, by working around or gracefully
degrading functionality. Thus libbpf is not tied to a specific kernel
version and can/should be packaged and versioned independently.
- Continuous integration testing via [TravisCI](https://travis-ci.org/libbpf/libbpf).
- Static code analysis via [LGTM](https://lgtm.com/projects/g/libbpf/libbpf) and [Coverity](https://scan.coverity.com/projects/libbpf).
Libbpf is designed to be kernel-agnostic and work across multitude of
kernel versions. It has built-in mechanisms to gracefully handle older
kernels, that are missing some of the features, by working around or
gracefully degrading functionality. Thus libbpf is not tied to a specific
kernel version and can/should be packaged and versioned independently.
- Continuous integration testing via
[TravisCI](https://travis-ci.org/libbpf/libbpf).
- Static code analysis via [LGTM](https://lgtm.com/projects/g/libbpf/libbpf)
and [Coverity](https://scan.coverity.com/projects/libbpf).
Package dependencies of libbpf, package names may vary across distros:
- zlib

File diff suppressed because it is too large Load Diff

View File

@@ -344,6 +344,7 @@ enum {
IFLA_BRPORT_ISOLATED,
IFLA_BRPORT_BACKUP_PORT,
IFLA_BRPORT_MRP_RING_OPEN,
IFLA_BRPORT_MRP_IN_OPEN,
__IFLA_BRPORT_MAX
};
#define IFLA_BRPORT_MAX (__IFLA_BRPORT_MAX - 1)

View File

@@ -73,9 +73,12 @@ struct xdp_umem_reg {
};
struct xdp_statistics {
__u64 rx_dropped; /* Dropped for reasons other than invalid desc */
__u64 rx_dropped; /* Dropped for other reasons */
__u64 rx_invalid_descs; /* Dropped due to invalid descriptor */
__u64 tx_invalid_descs; /* Dropped due to invalid descriptor */
__u64 rx_ring_full; /* Dropped due to rx ring being full */
__u64 rx_fill_ring_empty_descs; /* Failed to retrieve item from fill ring */
__u64 tx_ring_empty_descs; /* Failed to retrieve item from tx ring */
};
struct xdp_options {

View File

@@ -1,4 +1,5 @@
#!/bin/sh
# Usage: check-reallocarray.sh cc_path [cc_args...]
tfile=$(mktemp /tmp/test_reallocarray_XXXXXXXX.c)
ofile=${tfile%.c}.o
@@ -13,6 +14,6 @@ int main(void)
}
EOL
gcc $tfile -o $ofile >/dev/null 2>&1
"$@" $tfile -o $ofile >/dev/null 2>&1
if [ $? -ne 0 ]; then echo "FAIL"; fi
/bin/rm -f $tfile $ofile

View File

@@ -10,7 +10,7 @@ TOPDIR = ..
INCLUDES := -I. -I$(TOPDIR)/include -I$(TOPDIR)/include/uapi
ALL_CFLAGS := $(INCLUDES)
FEATURE_REALLOCARRAY := $(shell $(TOPDIR)/scripts/check-reallocarray.sh)
FEATURE_REALLOCARRAY := $(shell $(TOPDIR)/scripts/check-reallocarray.sh $(CC))
ifneq ($(FEATURE_REALLOCARRAY),)
ALL_CFLAGS += -DCOMPAT_NEED_REALLOCARRAY
endif

View File

@@ -598,10 +598,24 @@ int bpf_link_create(int prog_fd, int target_fd,
attr.link_create.prog_fd = prog_fd;
attr.link_create.target_fd = target_fd;
attr.link_create.attach_type = attach_type;
attr.link_create.flags = OPTS_GET(opts, flags, 0);
attr.link_create.iter_info =
ptr_to_u64(OPTS_GET(opts, iter_info, (void *)0));
attr.link_create.iter_info_len = OPTS_GET(opts, iter_info_len, 0);
return sys_bpf(BPF_LINK_CREATE, &attr, sizeof(attr));
}
int bpf_link_detach(int link_fd)
{
union bpf_attr attr;
memset(&attr, 0, sizeof(attr));
attr.link_detach.link_fd = link_fd;
return sys_bpf(BPF_LINK_DETACH, &attr, sizeof(attr));
}
int bpf_link_update(int link_fd, int new_prog_fd,
const struct bpf_link_update_opts *opts)
{

View File

@@ -168,15 +168,21 @@ LIBBPF_API int bpf_prog_detach(int attachable_fd, enum bpf_attach_type type);
LIBBPF_API int bpf_prog_detach2(int prog_fd, int attachable_fd,
enum bpf_attach_type type);
union bpf_iter_link_info; /* defined in up-to-date linux/bpf.h */
struct bpf_link_create_opts {
size_t sz; /* size of this struct for forward/backward compatibility */
__u32 flags;
union bpf_iter_link_info *iter_info;
__u32 iter_info_len;
};
#define bpf_link_create_opts__last_field sz
#define bpf_link_create_opts__last_field iter_info_len
LIBBPF_API int bpf_link_create(int prog_fd, int target_fd,
enum bpf_attach_type attach_type,
const struct bpf_link_create_opts *opts);
LIBBPF_API int bpf_link_detach(int link_fd);
struct bpf_link_update_opts {
size_t sz; /* size of this struct for forward/backward compatibility */
__u32 flags; /* extra flags */

View File

@@ -217,7 +217,7 @@ enum bpf_field_info_kind {
*/
#define BPF_CORE_READ_INTO(dst, src, a, ...) \
({ \
___core_read(bpf_core_read, dst, src, a, ##__VA_ARGS__) \
___core_read(bpf_core_read, dst, (src), a, ##__VA_ARGS__) \
})
/*
@@ -227,7 +227,7 @@ enum bpf_field_info_kind {
*/
#define BPF_CORE_READ_STR_INTO(dst, src, a, ...) \
({ \
___core_read(bpf_core_read_str, dst, src, a, ##__VA_ARGS__) \
___core_read(bpf_core_read_str, dst, (src), a, ##__VA_ARGS__)\
})
/*
@@ -254,8 +254,8 @@ enum bpf_field_info_kind {
*/
#define BPF_CORE_READ(src, a, ...) \
({ \
___type(src, a, ##__VA_ARGS__) __r; \
BPF_CORE_READ_INTO(&__r, src, a, ##__VA_ARGS__); \
___type((src), a, ##__VA_ARGS__) __r; \
BPF_CORE_READ_INTO(&__r, (src), a, ##__VA_ARGS__); \
__r; \
})

View File

@@ -2,8 +2,35 @@
#ifndef __BPF_ENDIAN__
#define __BPF_ENDIAN__
#include <linux/stddef.h>
#include <linux/swab.h>
/*
* Isolate byte #n and put it into byte #m, for __u##b type.
* E.g., moving byte #6 (nnnnnnnn) into byte #1 (mmmmmmmm) for __u64:
* 1) xxxxxxxx nnnnnnnn xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx mmmmmmmm xxxxxxxx
* 2) nnnnnnnn xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx mmmmmmmm xxxxxxxx 00000000
* 3) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 nnnnnnnn
* 4) 00000000 00000000 00000000 00000000 00000000 00000000 nnnnnnnn 00000000
*/
#define ___bpf_mvb(x, b, n, m) ((__u##b)(x) << (b-(n+1)*8) >> (b-8) << (m*8))
#define ___bpf_swab16(x) ((__u16)( \
___bpf_mvb(x, 16, 0, 1) | \
___bpf_mvb(x, 16, 1, 0)))
#define ___bpf_swab32(x) ((__u32)( \
___bpf_mvb(x, 32, 0, 3) | \
___bpf_mvb(x, 32, 1, 2) | \
___bpf_mvb(x, 32, 2, 1) | \
___bpf_mvb(x, 32, 3, 0)))
#define ___bpf_swab64(x) ((__u64)( \
___bpf_mvb(x, 64, 0, 7) | \
___bpf_mvb(x, 64, 1, 6) | \
___bpf_mvb(x, 64, 2, 5) | \
___bpf_mvb(x, 64, 3, 4) | \
___bpf_mvb(x, 64, 4, 3) | \
___bpf_mvb(x, 64, 5, 2) | \
___bpf_mvb(x, 64, 6, 1) | \
___bpf_mvb(x, 64, 7, 0)))
/* LLVM's BPF target selects the endianness of the CPU
* it compiles on, or the user specifies (bpfel/bpfeb),
@@ -23,16 +50,16 @@
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
# define __bpf_ntohs(x) __builtin_bswap16(x)
# define __bpf_htons(x) __builtin_bswap16(x)
# define __bpf_constant_ntohs(x) ___constant_swab16(x)
# define __bpf_constant_htons(x) ___constant_swab16(x)
# define __bpf_constant_ntohs(x) ___bpf_swab16(x)
# define __bpf_constant_htons(x) ___bpf_swab16(x)
# define __bpf_ntohl(x) __builtin_bswap32(x)
# define __bpf_htonl(x) __builtin_bswap32(x)
# define __bpf_constant_ntohl(x) ___constant_swab32(x)
# define __bpf_constant_htonl(x) ___constant_swab32(x)
# define __bpf_constant_ntohl(x) ___bpf_swab32(x)
# define __bpf_constant_htonl(x) ___bpf_swab32(x)
# define __bpf_be64_to_cpu(x) __builtin_bswap64(x)
# define __bpf_cpu_to_be64(x) __builtin_bswap64(x)
# define __bpf_constant_be64_to_cpu(x) ___constant_swab64(x)
# define __bpf_constant_cpu_to_be64(x) ___constant_swab64(x)
# define __bpf_constant_be64_to_cpu(x) ___bpf_swab64(x)
# define __bpf_constant_cpu_to_be64(x) ___bpf_swab64(x)
#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
# define __bpf_ntohs(x) (x)
# define __bpf_htons(x) (x)

View File

@@ -2,6 +2,7 @@
/* Forward declarations of BPF structs */
struct bpf_fib_lookup;
struct bpf_sk_lookup;
struct bpf_perf_event_data;
struct bpf_perf_event_value;
struct bpf_pidns_info;
@@ -19,6 +20,12 @@ struct sk_reuseport_md;
struct sockaddr;
struct tcphdr;
struct seq_file;
struct tcp6_sock;
struct tcp_sock;
struct tcp_timewait_sock;
struct tcp_request_sock;
struct udp6_sock;
struct task_struct;
struct __sk_buff;
struct sk_msg_md;
struct xdp_md;
@@ -54,7 +61,7 @@ static void *(*bpf_map_lookup_elem)(void *map, const void *key) = (void *) 1;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_map_update_elem)(void *map, const void *key, const void *value, __u64 flags) = (void *) 2;
static long (*bpf_map_update_elem)(void *map, const void *key, const void *value, __u64 flags) = (void *) 2;
/*
* bpf_map_delete_elem
@@ -64,7 +71,7 @@ static int (*bpf_map_update_elem)(void *map, const void *key, const void *value,
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_map_delete_elem)(void *map, const void *key) = (void *) 3;
static long (*bpf_map_delete_elem)(void *map, const void *key) = (void *) 3;
/*
* bpf_probe_read
@@ -78,7 +85,7 @@ static int (*bpf_map_delete_elem)(void *map, const void *key) = (void *) 3;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_probe_read)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 4;
static long (*bpf_probe_read)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 4;
/*
* bpf_ktime_get_ns
@@ -153,7 +160,7 @@ static __u64 (*bpf_ktime_get_ns)(void) = (void *) 5;
* The number of bytes written to the buffer, or a negative error
* in case of failure.
*/
static int (*bpf_trace_printk)(const char *fmt, __u32 fmt_size, ...) = (void *) 6;
static long (*bpf_trace_printk)(const char *fmt, __u32 fmt_size, ...) = (void *) 6;
/*
* bpf_get_prandom_u32
@@ -203,7 +210,7 @@ static __u32 (*bpf_get_smp_processor_id)(void) = (void *) 8;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_store_bytes)(struct __sk_buff *skb, __u32 offset, const void *from, __u32 len, __u64 flags) = (void *) 9;
static long (*bpf_skb_store_bytes)(struct __sk_buff *skb, __u32 offset, const void *from, __u32 len, __u64 flags) = (void *) 9;
/*
* bpf_l3_csum_replace
@@ -232,7 +239,7 @@ static int (*bpf_skb_store_bytes)(struct __sk_buff *skb, __u32 offset, const voi
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_l3_csum_replace)(struct __sk_buff *skb, __u32 offset, __u64 from, __u64 to, __u64 size) = (void *) 10;
static long (*bpf_l3_csum_replace)(struct __sk_buff *skb, __u32 offset, __u64 from, __u64 to, __u64 size) = (void *) 10;
/*
* bpf_l4_csum_replace
@@ -268,7 +275,7 @@ static int (*bpf_l3_csum_replace)(struct __sk_buff *skb, __u32 offset, __u64 fro
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_l4_csum_replace)(struct __sk_buff *skb, __u32 offset, __u64 from, __u64 to, __u64 flags) = (void *) 11;
static long (*bpf_l4_csum_replace)(struct __sk_buff *skb, __u32 offset, __u64 from, __u64 to, __u64 flags) = (void *) 11;
/*
* bpf_tail_call
@@ -303,7 +310,7 @@ static int (*bpf_l4_csum_replace)(struct __sk_buff *skb, __u32 offset, __u64 fro
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_tail_call)(void *ctx, void *prog_array_map, __u32 index) = (void *) 12;
static long (*bpf_tail_call)(void *ctx, void *prog_array_map, __u32 index) = (void *) 12;
/*
* bpf_clone_redirect
@@ -331,7 +338,7 @@ static int (*bpf_tail_call)(void *ctx, void *prog_array_map, __u32 index) = (voi
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_clone_redirect)(struct __sk_buff *skb, __u32 ifindex, __u64 flags) = (void *) 13;
static long (*bpf_clone_redirect)(struct __sk_buff *skb, __u32 ifindex, __u64 flags) = (void *) 13;
/*
* bpf_get_current_pid_tgid
@@ -368,7 +375,7 @@ static __u64 (*bpf_get_current_uid_gid)(void) = (void *) 15;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_get_current_comm)(void *buf, __u32 size_of_buf) = (void *) 16;
static long (*bpf_get_current_comm)(void *buf, __u32 size_of_buf) = (void *) 16;
/*
* bpf_get_cgroup_classid
@@ -418,7 +425,7 @@ static __u32 (*bpf_get_cgroup_classid)(struct __sk_buff *skb) = (void *) 17;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_vlan_push)(struct __sk_buff *skb, __be16 vlan_proto, __u16 vlan_tci) = (void *) 18;
static long (*bpf_skb_vlan_push)(struct __sk_buff *skb, __be16 vlan_proto, __u16 vlan_tci) = (void *) 18;
/*
* bpf_skb_vlan_pop
@@ -434,7 +441,7 @@ static int (*bpf_skb_vlan_push)(struct __sk_buff *skb, __be16 vlan_proto, __u16
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_vlan_pop)(struct __sk_buff *skb) = (void *) 19;
static long (*bpf_skb_vlan_pop)(struct __sk_buff *skb) = (void *) 19;
/*
* bpf_skb_get_tunnel_key
@@ -489,7 +496,7 @@ static int (*bpf_skb_vlan_pop)(struct __sk_buff *skb) = (void *) 19;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_get_tunnel_key)(struct __sk_buff *skb, struct bpf_tunnel_key *key, __u32 size, __u64 flags) = (void *) 20;
static long (*bpf_skb_get_tunnel_key)(struct __sk_buff *skb, struct bpf_tunnel_key *key, __u32 size, __u64 flags) = (void *) 20;
/*
* bpf_skb_set_tunnel_key
@@ -530,7 +537,7 @@ static int (*bpf_skb_get_tunnel_key)(struct __sk_buff *skb, struct bpf_tunnel_ke
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_set_tunnel_key)(struct __sk_buff *skb, struct bpf_tunnel_key *key, __u32 size, __u64 flags) = (void *) 21;
static long (*bpf_skb_set_tunnel_key)(struct __sk_buff *skb, struct bpf_tunnel_key *key, __u32 size, __u64 flags) = (void *) 21;
/*
* bpf_perf_event_read
@@ -590,7 +597,7 @@ static __u64 (*bpf_perf_event_read)(void *map, __u64 flags) = (void *) 22;
* are **TC_ACT_REDIRECT** on success or **TC_ACT_SHOT** on
* error.
*/
static int (*bpf_redirect)(__u32 ifindex, __u64 flags) = (void *) 23;
static long (*bpf_redirect)(__u32 ifindex, __u64 flags) = (void *) 23;
/*
* bpf_get_route_realm
@@ -667,7 +674,7 @@ static __u32 (*bpf_get_route_realm)(struct __sk_buff *skb) = (void *) 24;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_perf_event_output)(void *ctx, void *map, __u64 flags, void *data, __u64 size) = (void *) 25;
static long (*bpf_perf_event_output)(void *ctx, void *map, __u64 flags, void *data, __u64 size) = (void *) 25;
/*
* bpf_skb_load_bytes
@@ -688,7 +695,7 @@ static int (*bpf_perf_event_output)(void *ctx, void *map, __u64 flags, void *dat
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_load_bytes)(const void *skb, __u32 offset, void *to, __u32 len) = (void *) 26;
static long (*bpf_skb_load_bytes)(const void *skb, __u32 offset, void *to, __u32 len) = (void *) 26;
/*
* bpf_get_stackid
@@ -734,7 +741,7 @@ static int (*bpf_skb_load_bytes)(const void *skb, __u32 offset, void *to, __u32
* The positive or null stack id on success, or a negative error
* in case of failure.
*/
static int (*bpf_get_stackid)(void *ctx, void *map, __u64 flags) = (void *) 27;
static long (*bpf_get_stackid)(void *ctx, void *map, __u64 flags) = (void *) 27;
/*
* bpf_csum_diff
@@ -787,7 +794,7 @@ static __s64 (*bpf_csum_diff)(__be32 *from, __u32 from_size, __be32 *to, __u32 t
* Returns
* The size of the option data retrieved.
*/
static int (*bpf_skb_get_tunnel_opt)(struct __sk_buff *skb, void *opt, __u32 size) = (void *) 29;
static long (*bpf_skb_get_tunnel_opt)(struct __sk_buff *skb, void *opt, __u32 size) = (void *) 29;
/*
* bpf_skb_set_tunnel_opt
@@ -801,7 +808,7 @@ static int (*bpf_skb_get_tunnel_opt)(struct __sk_buff *skb, void *opt, __u32 siz
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_set_tunnel_opt)(struct __sk_buff *skb, void *opt, __u32 size) = (void *) 30;
static long (*bpf_skb_set_tunnel_opt)(struct __sk_buff *skb, void *opt, __u32 size) = (void *) 30;
/*
* bpf_skb_change_proto
@@ -832,7 +839,7 @@ static int (*bpf_skb_set_tunnel_opt)(struct __sk_buff *skb, void *opt, __u32 siz
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_change_proto)(struct __sk_buff *skb, __be16 proto, __u64 flags) = (void *) 31;
static long (*bpf_skb_change_proto)(struct __sk_buff *skb, __be16 proto, __u64 flags) = (void *) 31;
/*
* bpf_skb_change_type
@@ -863,7 +870,7 @@ static int (*bpf_skb_change_proto)(struct __sk_buff *skb, __be16 proto, __u64 fl
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_change_type)(struct __sk_buff *skb, __u32 type) = (void *) 32;
static long (*bpf_skb_change_type)(struct __sk_buff *skb, __u32 type) = (void *) 32;
/*
* bpf_skb_under_cgroup
@@ -878,7 +885,7 @@ static int (*bpf_skb_change_type)(struct __sk_buff *skb, __u32 type) = (void *)
* * 1, if the *skb* succeeded the cgroup2 descendant test.
* * A negative error code, if an error occurred.
*/
static int (*bpf_skb_under_cgroup)(struct __sk_buff *skb, void *map, __u32 index) = (void *) 33;
static long (*bpf_skb_under_cgroup)(struct __sk_buff *skb, void *map, __u32 index) = (void *) 33;
/*
* bpf_get_hash_recalc
@@ -930,7 +937,7 @@ static __u64 (*bpf_get_current_task)(void) = (void *) 35;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_probe_write_user)(void *dst, const void *src, __u32 len) = (void *) 36;
static long (*bpf_probe_write_user)(void *dst, const void *src, __u32 len) = (void *) 36;
/*
* bpf_current_task_under_cgroup
@@ -946,7 +953,7 @@ static int (*bpf_probe_write_user)(void *dst, const void *src, __u32 len) = (voi
* * 1, if the *skb* task does not belong to the cgroup2.
* * A negative error code, if an error occurred.
*/
static int (*bpf_current_task_under_cgroup)(void *map, __u32 index) = (void *) 37;
static long (*bpf_current_task_under_cgroup)(void *map, __u32 index) = (void *) 37;
/*
* bpf_skb_change_tail
@@ -974,7 +981,7 @@ static int (*bpf_current_task_under_cgroup)(void *map, __u32 index) = (void *) 3
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_change_tail)(struct __sk_buff *skb, __u32 len, __u64 flags) = (void *) 38;
static long (*bpf_skb_change_tail)(struct __sk_buff *skb, __u32 len, __u64 flags) = (void *) 38;
/*
* bpf_skb_pull_data
@@ -1014,7 +1021,7 @@ static int (*bpf_skb_change_tail)(struct __sk_buff *skb, __u32 len, __u64 flags)
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_pull_data)(struct __sk_buff *skb, __u32 len) = (void *) 39;
static long (*bpf_skb_pull_data)(struct __sk_buff *skb, __u32 len) = (void *) 39;
/*
* bpf_csum_update
@@ -1057,7 +1064,7 @@ static void (*bpf_set_hash_invalid)(struct __sk_buff *skb) = (void *) 41;
* Returns
* The id of current NUMA node.
*/
static int (*bpf_get_numa_node_id)(void) = (void *) 42;
static long (*bpf_get_numa_node_id)(void) = (void *) 42;
/*
* bpf_skb_change_head
@@ -1082,7 +1089,7 @@ static int (*bpf_get_numa_node_id)(void) = (void *) 42;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_change_head)(struct __sk_buff *skb, __u32 len, __u64 flags) = (void *) 43;
static long (*bpf_skb_change_head)(struct __sk_buff *skb, __u32 len, __u64 flags) = (void *) 43;
/*
* bpf_xdp_adjust_head
@@ -1101,7 +1108,7 @@ static int (*bpf_skb_change_head)(struct __sk_buff *skb, __u32 len, __u64 flags)
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_xdp_adjust_head)(struct xdp_md *xdp_md, int delta) = (void *) 44;
static long (*bpf_xdp_adjust_head)(struct xdp_md *xdp_md, int delta) = (void *) 44;
/*
* bpf_probe_read_str
@@ -1118,7 +1125,7 @@ static int (*bpf_xdp_adjust_head)(struct xdp_md *xdp_md, int delta) = (void *) 4
* including the trailing NUL character. On error, a negative
* value.
*/
static int (*bpf_probe_read_str)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 45;
static long (*bpf_probe_read_str)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 45;
/*
* bpf_get_socket_cookie
@@ -1159,7 +1166,7 @@ static __u32 (*bpf_get_socket_uid)(struct __sk_buff *skb) = (void *) 47;
* Returns
* 0
*/
static __u32 (*bpf_set_hash)(struct __sk_buff *skb, __u32 hash) = (void *) 48;
static long (*bpf_set_hash)(struct __sk_buff *skb, __u32 hash) = (void *) 48;
/*
* bpf_setsockopt
@@ -1181,17 +1188,20 @@ static __u32 (*bpf_set_hash)(struct __sk_buff *skb, __u32 hash) = (void *) 48;
*
* * **SOL_SOCKET**, which supports the following *optname*\ s:
* **SO_RCVBUF**, **SO_SNDBUF**, **SO_MAX_PACING_RATE**,
* **SO_PRIORITY**, **SO_RCVLOWAT**, **SO_MARK**.
* **SO_PRIORITY**, **SO_RCVLOWAT**, **SO_MARK**,
* **SO_BINDTODEVICE**, **SO_KEEPALIVE**.
* * **IPPROTO_TCP**, which supports the following *optname*\ s:
* **TCP_CONGESTION**, **TCP_BPF_IW**,
* **TCP_BPF_SNDCWND_CLAMP**.
* **TCP_BPF_SNDCWND_CLAMP**, **TCP_SAVE_SYN**,
* **TCP_KEEPIDLE**, **TCP_KEEPINTVL**, **TCP_KEEPCNT**,
* **TCP_SYNCNT**, **TCP_USER_TIMEOUT**.
* * **IPPROTO_IP**, which supports *optname* **IP_TOS**.
* * **IPPROTO_IPV6**, which supports *optname* **IPV6_TCLASS**.
*
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_setsockopt)(void *bpf_socket, int level, int optname, void *optval, int optlen) = (void *) 49;
static long (*bpf_setsockopt)(void *bpf_socket, int level, int optname, void *optval, int optlen) = (void *) 49;
/*
* bpf_skb_adjust_room
@@ -1241,7 +1251,7 @@ static int (*bpf_setsockopt)(void *bpf_socket, int level, int optname, void *opt
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_adjust_room)(struct __sk_buff *skb, __s32 len_diff, __u32 mode, __u64 flags) = (void *) 50;
static long (*bpf_skb_adjust_room)(struct __sk_buff *skb, __s32 len_diff, __u32 mode, __u64 flags) = (void *) 50;
/*
* bpf_redirect_map
@@ -1266,7 +1276,7 @@ static int (*bpf_skb_adjust_room)(struct __sk_buff *skb, __s32 len_diff, __u32 m
* **XDP_REDIRECT** on success, or the value of the two lower bits
* of the *flags* argument on error.
*/
static int (*bpf_redirect_map)(void *map, __u32 key, __u64 flags) = (void *) 51;
static long (*bpf_redirect_map)(void *map, __u32 key, __u64 flags) = (void *) 51;
/*
* bpf_sk_redirect_map
@@ -1281,7 +1291,7 @@ static int (*bpf_redirect_map)(void *map, __u32 key, __u64 flags) = (void *) 51;
* Returns
* **SK_PASS** on success, or **SK_DROP** on error.
*/
static int (*bpf_sk_redirect_map)(struct __sk_buff *skb, void *map, __u32 key, __u64 flags) = (void *) 52;
static long (*bpf_sk_redirect_map)(struct __sk_buff *skb, void *map, __u32 key, __u64 flags) = (void *) 52;
/*
* bpf_sock_map_update
@@ -1304,7 +1314,7 @@ static int (*bpf_sk_redirect_map)(struct __sk_buff *skb, void *map, __u32 key, _
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_sock_map_update)(struct bpf_sock_ops *skops, void *map, void *key, __u64 flags) = (void *) 53;
static long (*bpf_sock_map_update)(struct bpf_sock_ops *skops, void *map, void *key, __u64 flags) = (void *) 53;
/*
* bpf_xdp_adjust_meta
@@ -1337,7 +1347,7 @@ static int (*bpf_sock_map_update)(struct bpf_sock_ops *skops, void *map, void *k
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_xdp_adjust_meta)(struct xdp_md *xdp_md, int delta) = (void *) 54;
static long (*bpf_xdp_adjust_meta)(struct xdp_md *xdp_md, int delta) = (void *) 54;
/*
* bpf_perf_event_read_value
@@ -1391,7 +1401,7 @@ static int (*bpf_xdp_adjust_meta)(struct xdp_md *xdp_md, int delta) = (void *) 5
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_perf_event_read_value)(void *map, __u64 flags, struct bpf_perf_event_value *buf, __u32 buf_size) = (void *) 55;
static long (*bpf_perf_event_read_value)(void *map, __u64 flags, struct bpf_perf_event_value *buf, __u32 buf_size) = (void *) 55;
/*
* bpf_perf_prog_read_value
@@ -1406,7 +1416,7 @@ static int (*bpf_perf_event_read_value)(void *map, __u64 flags, struct bpf_perf_
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_perf_prog_read_value)(struct bpf_perf_event_data *ctx, struct bpf_perf_event_value *buf, __u32 buf_size) = (void *) 56;
static long (*bpf_perf_prog_read_value)(struct bpf_perf_event_data *ctx, struct bpf_perf_event_value *buf, __u32 buf_size) = (void *) 56;
/*
* bpf_getsockopt
@@ -1435,7 +1445,7 @@ static int (*bpf_perf_prog_read_value)(struct bpf_perf_event_data *ctx, struct b
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_getsockopt)(void *bpf_socket, int level, int optname, void *optval, int optlen) = (void *) 57;
static long (*bpf_getsockopt)(void *bpf_socket, int level, int optname, void *optval, int optlen) = (void *) 57;
/*
* bpf_override_return
@@ -1464,7 +1474,7 @@ static int (*bpf_getsockopt)(void *bpf_socket, int level, int optname, void *opt
* Returns
* 0
*/
static int (*bpf_override_return)(struct pt_regs *regs, __u64 rc) = (void *) 58;
static long (*bpf_override_return)(struct pt_regs *regs, __u64 rc) = (void *) 58;
/*
* bpf_sock_ops_cb_flags_set
@@ -1512,7 +1522,7 @@ static int (*bpf_override_return)(struct pt_regs *regs, __u64 rc) = (void *) 58;
* be set is returned (which comes down to 0 if all bits were set
* as required).
*/
static int (*bpf_sock_ops_cb_flags_set)(struct bpf_sock_ops *bpf_sock, int argval) = (void *) 59;
static long (*bpf_sock_ops_cb_flags_set)(struct bpf_sock_ops *bpf_sock, int argval) = (void *) 59;
/*
* bpf_msg_redirect_map
@@ -1530,7 +1540,7 @@ static int (*bpf_sock_ops_cb_flags_set)(struct bpf_sock_ops *bpf_sock, int argva
* Returns
* **SK_PASS** on success, or **SK_DROP** on error.
*/
static int (*bpf_msg_redirect_map)(struct sk_msg_md *msg, void *map, __u32 key, __u64 flags) = (void *) 60;
static long (*bpf_msg_redirect_map)(struct sk_msg_md *msg, void *map, __u32 key, __u64 flags) = (void *) 60;
/*
* bpf_msg_apply_bytes
@@ -1568,7 +1578,7 @@ static int (*bpf_msg_redirect_map)(struct sk_msg_md *msg, void *map, __u32 key,
* Returns
* 0
*/
static int (*bpf_msg_apply_bytes)(struct sk_msg_md *msg, __u32 bytes) = (void *) 61;
static long (*bpf_msg_apply_bytes)(struct sk_msg_md *msg, __u32 bytes) = (void *) 61;
/*
* bpf_msg_cork_bytes
@@ -1590,7 +1600,7 @@ static int (*bpf_msg_apply_bytes)(struct sk_msg_md *msg, __u32 bytes) = (void *)
* Returns
* 0
*/
static int (*bpf_msg_cork_bytes)(struct sk_msg_md *msg, __u32 bytes) = (void *) 62;
static long (*bpf_msg_cork_bytes)(struct sk_msg_md *msg, __u32 bytes) = (void *) 62;
/*
* bpf_msg_pull_data
@@ -1625,7 +1635,7 @@ static int (*bpf_msg_cork_bytes)(struct sk_msg_md *msg, __u32 bytes) = (void *)
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_msg_pull_data)(struct sk_msg_md *msg, __u32 start, __u32 end, __u64 flags) = (void *) 63;
static long (*bpf_msg_pull_data)(struct sk_msg_md *msg, __u32 start, __u32 end, __u64 flags) = (void *) 63;
/*
* bpf_bind
@@ -1647,7 +1657,7 @@ static int (*bpf_msg_pull_data)(struct sk_msg_md *msg, __u32 start, __u32 end, _
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_bind)(struct bpf_sock_addr *ctx, struct sockaddr *addr, int addr_len) = (void *) 64;
static long (*bpf_bind)(struct bpf_sock_addr *ctx, struct sockaddr *addr, int addr_len) = (void *) 64;
/*
* bpf_xdp_adjust_tail
@@ -1665,7 +1675,7 @@ static int (*bpf_bind)(struct bpf_sock_addr *ctx, struct sockaddr *addr, int add
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_xdp_adjust_tail)(struct xdp_md *xdp_md, int delta) = (void *) 65;
static long (*bpf_xdp_adjust_tail)(struct xdp_md *xdp_md, int delta) = (void *) 65;
/*
* bpf_skb_get_xfrm_state
@@ -1685,7 +1695,7 @@ static int (*bpf_xdp_adjust_tail)(struct xdp_md *xdp_md, int delta) = (void *) 6
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_get_xfrm_state)(struct __sk_buff *skb, __u32 index, struct bpf_xfrm_state *xfrm_state, __u32 size, __u64 flags) = (void *) 66;
static long (*bpf_skb_get_xfrm_state)(struct __sk_buff *skb, __u32 index, struct bpf_xfrm_state *xfrm_state, __u32 size, __u64 flags) = (void *) 66;
/*
* bpf_get_stack
@@ -1722,7 +1732,7 @@ static int (*bpf_skb_get_xfrm_state)(struct __sk_buff *skb, __u32 index, struct
* A non-negative value equal to or less than *size* on success,
* or a negative error in case of failure.
*/
static int (*bpf_get_stack)(void *ctx, void *buf, __u32 size, __u64 flags) = (void *) 67;
static long (*bpf_get_stack)(void *ctx, void *buf, __u32 size, __u64 flags) = (void *) 67;
/*
* bpf_skb_load_bytes_relative
@@ -1748,7 +1758,7 @@ static int (*bpf_get_stack)(void *ctx, void *buf, __u32 size, __u64 flags) = (vo
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_load_bytes_relative)(const void *skb, __u32 offset, void *to, __u32 len, __u32 start_header) = (void *) 68;
static long (*bpf_skb_load_bytes_relative)(const void *skb, __u32 offset, void *to, __u32 len, __u32 start_header) = (void *) 68;
/*
* bpf_fib_lookup
@@ -1783,7 +1793,7 @@ static int (*bpf_skb_load_bytes_relative)(const void *skb, __u32 offset, void *t
* * > 0 one of **BPF_FIB_LKUP_RET_** codes explaining why the
* packet is not forwarded or needs assist from full stack
*/
static int (*bpf_fib_lookup)(void *ctx, struct bpf_fib_lookup *params, int plen, __u32 flags) = (void *) 69;
static long (*bpf_fib_lookup)(void *ctx, struct bpf_fib_lookup *params, int plen, __u32 flags) = (void *) 69;
/*
* bpf_sock_hash_update
@@ -1806,7 +1816,7 @@ static int (*bpf_fib_lookup)(void *ctx, struct bpf_fib_lookup *params, int plen,
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_sock_hash_update)(struct bpf_sock_ops *skops, void *map, void *key, __u64 flags) = (void *) 70;
static long (*bpf_sock_hash_update)(struct bpf_sock_ops *skops, void *map, void *key, __u64 flags) = (void *) 70;
/*
* bpf_msg_redirect_hash
@@ -1824,7 +1834,7 @@ static int (*bpf_sock_hash_update)(struct bpf_sock_ops *skops, void *map, void *
* Returns
* **SK_PASS** on success, or **SK_DROP** on error.
*/
static int (*bpf_msg_redirect_hash)(struct sk_msg_md *msg, void *map, void *key, __u64 flags) = (void *) 71;
static long (*bpf_msg_redirect_hash)(struct sk_msg_md *msg, void *map, void *key, __u64 flags) = (void *) 71;
/*
* bpf_sk_redirect_hash
@@ -1842,7 +1852,7 @@ static int (*bpf_msg_redirect_hash)(struct sk_msg_md *msg, void *map, void *key,
* Returns
* **SK_PASS** on success, or **SK_DROP** on error.
*/
static int (*bpf_sk_redirect_hash)(struct __sk_buff *skb, void *map, void *key, __u64 flags) = (void *) 72;
static long (*bpf_sk_redirect_hash)(struct __sk_buff *skb, void *map, void *key, __u64 flags) = (void *) 72;
/*
* bpf_lwt_push_encap
@@ -1883,7 +1893,7 @@ static int (*bpf_sk_redirect_hash)(struct __sk_buff *skb, void *map, void *key,
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_lwt_push_encap)(struct __sk_buff *skb, __u32 type, void *hdr, __u32 len) = (void *) 73;
static long (*bpf_lwt_push_encap)(struct __sk_buff *skb, __u32 type, void *hdr, __u32 len) = (void *) 73;
/*
* bpf_lwt_seg6_store_bytes
@@ -1902,7 +1912,7 @@ static int (*bpf_lwt_push_encap)(struct __sk_buff *skb, __u32 type, void *hdr, _
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_lwt_seg6_store_bytes)(struct __sk_buff *skb, __u32 offset, const void *from, __u32 len) = (void *) 74;
static long (*bpf_lwt_seg6_store_bytes)(struct __sk_buff *skb, __u32 offset, const void *from, __u32 len) = (void *) 74;
/*
* bpf_lwt_seg6_adjust_srh
@@ -1922,7 +1932,7 @@ static int (*bpf_lwt_seg6_store_bytes)(struct __sk_buff *skb, __u32 offset, cons
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_lwt_seg6_adjust_srh)(struct __sk_buff *skb, __u32 offset, __s32 delta) = (void *) 75;
static long (*bpf_lwt_seg6_adjust_srh)(struct __sk_buff *skb, __u32 offset, __s32 delta) = (void *) 75;
/*
* bpf_lwt_seg6_action
@@ -1955,7 +1965,7 @@ static int (*bpf_lwt_seg6_adjust_srh)(struct __sk_buff *skb, __u32 offset, __s32
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_lwt_seg6_action)(struct __sk_buff *skb, __u32 action, void *param, __u32 param_len) = (void *) 76;
static long (*bpf_lwt_seg6_action)(struct __sk_buff *skb, __u32 action, void *param, __u32 param_len) = (void *) 76;
/*
* bpf_rc_repeat
@@ -1978,7 +1988,7 @@ static int (*bpf_lwt_seg6_action)(struct __sk_buff *skb, __u32 action, void *par
* Returns
* 0
*/
static int (*bpf_rc_repeat)(void *ctx) = (void *) 77;
static long (*bpf_rc_repeat)(void *ctx) = (void *) 77;
/*
* bpf_rc_keydown
@@ -2008,7 +2018,7 @@ static int (*bpf_rc_repeat)(void *ctx) = (void *) 77;
* Returns
* 0
*/
static int (*bpf_rc_keydown)(void *ctx, __u32 protocol, __u64 scancode, __u32 toggle) = (void *) 78;
static long (*bpf_rc_keydown)(void *ctx, __u32 protocol, __u64 scancode, __u32 toggle) = (void *) 78;
/*
* bpf_skb_cgroup_id
@@ -2073,7 +2083,7 @@ static void *(*bpf_get_local_storage)(void *map, __u64 flags) = (void *) 81;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_sk_select_reuseport)(struct sk_reuseport_md *reuse, void *map, void *key, __u64 flags) = (void *) 82;
static long (*bpf_sk_select_reuseport)(struct sk_reuseport_md *reuse, void *map, void *key, __u64 flags) = (void *) 82;
/*
* bpf_skb_ancestor_cgroup_id
@@ -2116,7 +2126,7 @@ static __u64 (*bpf_skb_ancestor_cgroup_id)(struct __sk_buff *skb, int ancestor_l
* Look for an IPv6 socket.
*
* If the *netns* is a negative signed 32-bit integer, then the
* socket lookup table in the netns associated with the *ctx* will
* socket lookup table in the netns associated with the *ctx*
* will be used. For the TC hooks, this is the netns of the device
* in the skb. For socket hooks, this is the netns of the socket.
* If *netns* is any other signed 32-bit value greater than or
@@ -2157,7 +2167,7 @@ static struct bpf_sock *(*bpf_sk_lookup_tcp)(void *ctx, struct bpf_sock_tuple *t
* Look for an IPv6 socket.
*
* If the *netns* is a negative signed 32-bit integer, then the
* socket lookup table in the netns associated with the *ctx* will
* socket lookup table in the netns associated with the *ctx*
* will be used. For the TC hooks, this is the netns of the device
* in the skb. For socket hooks, this is the netns of the socket.
* If *netns* is any other signed 32-bit value greater than or
@@ -2189,7 +2199,7 @@ static struct bpf_sock *(*bpf_sk_lookup_udp)(void *ctx, struct bpf_sock_tuple *t
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_sk_release)(struct bpf_sock *sock) = (void *) 86;
static long (*bpf_sk_release)(struct bpf_sock *sock) = (void *) 86;
/*
* bpf_map_push_elem
@@ -2203,7 +2213,7 @@ static int (*bpf_sk_release)(struct bpf_sock *sock) = (void *) 86;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_map_push_elem)(void *map, const void *value, __u64 flags) = (void *) 87;
static long (*bpf_map_push_elem)(void *map, const void *value, __u64 flags) = (void *) 87;
/*
* bpf_map_pop_elem
@@ -2213,7 +2223,7 @@ static int (*bpf_map_push_elem)(void *map, const void *value, __u64 flags) = (vo
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_map_pop_elem)(void *map, void *value) = (void *) 88;
static long (*bpf_map_pop_elem)(void *map, void *value) = (void *) 88;
/*
* bpf_map_peek_elem
@@ -2223,7 +2233,7 @@ static int (*bpf_map_pop_elem)(void *map, void *value) = (void *) 88;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_map_peek_elem)(void *map, void *value) = (void *) 89;
static long (*bpf_map_peek_elem)(void *map, void *value) = (void *) 89;
/*
* bpf_msg_push_data
@@ -2243,7 +2253,7 @@ static int (*bpf_map_peek_elem)(void *map, void *value) = (void *) 89;
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_msg_push_data)(struct sk_msg_md *msg, __u32 start, __u32 len, __u64 flags) = (void *) 90;
static long (*bpf_msg_push_data)(struct sk_msg_md *msg, __u32 start, __u32 len, __u64 flags) = (void *) 90;
/*
* bpf_msg_pop_data
@@ -2259,7 +2269,7 @@ static int (*bpf_msg_push_data)(struct sk_msg_md *msg, __u32 start, __u32 len, _
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_msg_pop_data)(struct sk_msg_md *msg, __u32 start, __u32 len, __u64 flags) = (void *) 91;
static long (*bpf_msg_pop_data)(struct sk_msg_md *msg, __u32 start, __u32 len, __u64 flags) = (void *) 91;
/*
* bpf_rc_pointer_rel
@@ -2277,7 +2287,7 @@ static int (*bpf_msg_pop_data)(struct sk_msg_md *msg, __u32 start, __u32 len, __
* Returns
* 0
*/
static int (*bpf_rc_pointer_rel)(void *ctx, __s32 rel_x, __s32 rel_y) = (void *) 92;
static long (*bpf_rc_pointer_rel)(void *ctx, __s32 rel_x, __s32 rel_y) = (void *) 92;
/*
* bpf_spin_lock
@@ -2329,7 +2339,7 @@ static int (*bpf_rc_pointer_rel)(void *ctx, __s32 rel_x, __s32 rel_y) = (void *)
* Returns
* 0
*/
static int (*bpf_spin_lock)(struct bpf_spin_lock *lock) = (void *) 93;
static long (*bpf_spin_lock)(struct bpf_spin_lock *lock) = (void *) 93;
/*
* bpf_spin_unlock
@@ -2340,7 +2350,7 @@ static int (*bpf_spin_lock)(struct bpf_spin_lock *lock) = (void *) 93;
* Returns
* 0
*/
static int (*bpf_spin_unlock)(struct bpf_spin_lock *lock) = (void *) 94;
static long (*bpf_spin_unlock)(struct bpf_spin_lock *lock) = (void *) 94;
/*
* bpf_sk_fullsock
@@ -2378,7 +2388,7 @@ static struct bpf_tcp_sock *(*bpf_tcp_sock)(struct bpf_sock *sk) = (void *) 96;
* 1 if the **CE** flag is set (either by the current helper call
* or because it was already present), 0 if it is not set.
*/
static int (*bpf_skb_ecn_set_ce)(struct __sk_buff *skb) = (void *) 97;
static long (*bpf_skb_ecn_set_ce)(struct __sk_buff *skb) = (void *) 97;
/*
* bpf_get_listener_sock
@@ -2432,7 +2442,7 @@ static struct bpf_sock *(*bpf_skc_lookup_tcp)(void *ctx, struct bpf_sock_tuple *
* 0 if *iph* and *th* are a valid SYN cookie ACK, or a negative
* error otherwise.
*/
static int (*bpf_tcp_check_syncookie)(struct bpf_sock *sk, void *iph, __u32 iph_len, struct tcphdr *th, __u32 th_len) = (void *) 100;
static long (*bpf_tcp_check_syncookie)(struct bpf_sock *sk, void *iph, __u32 iph_len, struct tcphdr *th, __u32 th_len) = (void *) 100;
/*
* bpf_sysctl_get_name
@@ -2452,7 +2462,7 @@ static int (*bpf_tcp_check_syncookie)(struct bpf_sock *sk, void *iph, __u32 iph_
* **-E2BIG** if the buffer wasn't big enough (*buf* will contain
* truncated name in this case).
*/
static int (*bpf_sysctl_get_name)(struct bpf_sysctl *ctx, char *buf, unsigned long buf_len, __u64 flags) = (void *) 101;
static long (*bpf_sysctl_get_name)(struct bpf_sysctl *ctx, char *buf, unsigned long buf_len, __u64 flags) = (void *) 101;
/*
* bpf_sysctl_get_current_value
@@ -2475,7 +2485,7 @@ static int (*bpf_sysctl_get_name)(struct bpf_sysctl *ctx, char *buf, unsigned lo
* **-EINVAL** if current value was unavailable, e.g. because
* sysctl is uninitialized and read returns -EIO for it.
*/
static int (*bpf_sysctl_get_current_value)(struct bpf_sysctl *ctx, char *buf, unsigned long buf_len) = (void *) 102;
static long (*bpf_sysctl_get_current_value)(struct bpf_sysctl *ctx, char *buf, unsigned long buf_len) = (void *) 102;
/*
* bpf_sysctl_get_new_value
@@ -2496,7 +2506,7 @@ static int (*bpf_sysctl_get_current_value)(struct bpf_sysctl *ctx, char *buf, un
*
* **-EINVAL** if sysctl is being read.
*/
static int (*bpf_sysctl_get_new_value)(struct bpf_sysctl *ctx, char *buf, unsigned long buf_len) = (void *) 103;
static long (*bpf_sysctl_get_new_value)(struct bpf_sysctl *ctx, char *buf, unsigned long buf_len) = (void *) 103;
/*
* bpf_sysctl_set_new_value
@@ -2517,7 +2527,7 @@ static int (*bpf_sysctl_get_new_value)(struct bpf_sysctl *ctx, char *buf, unsign
*
* **-EINVAL** if sysctl is being read.
*/
static int (*bpf_sysctl_set_new_value)(struct bpf_sysctl *ctx, const char *buf, unsigned long buf_len) = (void *) 104;
static long (*bpf_sysctl_set_new_value)(struct bpf_sysctl *ctx, const char *buf, unsigned long buf_len) = (void *) 104;
/*
* bpf_strtol
@@ -2545,7 +2555,7 @@ static int (*bpf_sysctl_set_new_value)(struct bpf_sysctl *ctx, const char *buf,
*
* **-ERANGE** if resulting value was out of range.
*/
static int (*bpf_strtol)(const char *buf, unsigned long buf_len, __u64 flags, long *res) = (void *) 105;
static long (*bpf_strtol)(const char *buf, unsigned long buf_len, __u64 flags, long *res) = (void *) 105;
/*
* bpf_strtoul
@@ -2572,7 +2582,7 @@ static int (*bpf_strtol)(const char *buf, unsigned long buf_len, __u64 flags, lo
*
* **-ERANGE** if resulting value was out of range.
*/
static int (*bpf_strtoul)(const char *buf, unsigned long buf_len, __u64 flags, unsigned long *res) = (void *) 106;
static long (*bpf_strtoul)(const char *buf, unsigned long buf_len, __u64 flags, unsigned long *res) = (void *) 106;
/*
* bpf_sk_storage_get
@@ -2616,7 +2626,7 @@ static void *(*bpf_sk_storage_get)(void *map, struct bpf_sock *sk, void *value,
*
* **-ENOENT** if the bpf-local-storage cannot be found.
*/
static int (*bpf_sk_storage_delete)(void *map, struct bpf_sock *sk) = (void *) 108;
static long (*bpf_sk_storage_delete)(void *map, struct bpf_sock *sk) = (void *) 108;
/*
* bpf_send_signal
@@ -2635,7 +2645,7 @@ static int (*bpf_sk_storage_delete)(void *map, struct bpf_sock *sk) = (void *) 1
*
* **-EAGAIN** if bpf program can try again.
*/
static int (*bpf_send_signal)(__u32 sig) = (void *) 109;
static long (*bpf_send_signal)(__u32 sig) = (void *) 109;
/*
* bpf_tcp_gen_syncookie
@@ -2693,7 +2703,7 @@ static __s64 (*bpf_tcp_gen_syncookie)(struct bpf_sock *sk, void *iph, __u32 iph_
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_skb_output)(void *ctx, void *map, __u64 flags, void *data, __u64 size) = (void *) 111;
static long (*bpf_skb_output)(void *ctx, void *map, __u64 flags, void *data, __u64 size) = (void *) 111;
/*
* bpf_probe_read_user
@@ -2704,7 +2714,7 @@ static int (*bpf_skb_output)(void *ctx, void *map, __u64 flags, void *data, __u6
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_probe_read_user)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 112;
static long (*bpf_probe_read_user)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 112;
/*
* bpf_probe_read_kernel
@@ -2715,7 +2725,7 @@ static int (*bpf_probe_read_user)(void *dst, __u32 size, const void *unsafe_ptr)
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_probe_read_kernel)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 113;
static long (*bpf_probe_read_kernel)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 113;
/*
* bpf_probe_read_user_str
@@ -2763,7 +2773,7 @@ static int (*bpf_probe_read_kernel)(void *dst, __u32 size, const void *unsafe_pt
* including the trailing NUL character. On error, a negative
* value.
*/
static int (*bpf_probe_read_user_str)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 114;
static long (*bpf_probe_read_user_str)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 114;
/*
* bpf_probe_read_kernel_str
@@ -2775,7 +2785,7 @@ static int (*bpf_probe_read_user_str)(void *dst, __u32 size, const void *unsafe_
* On success, the strictly positive length of the string, including
* the trailing NUL character. On error, a negative value.
*/
static int (*bpf_probe_read_kernel_str)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 115;
static long (*bpf_probe_read_kernel_str)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 115;
/*
* bpf_tcp_send_ack
@@ -2786,7 +2796,7 @@ static int (*bpf_probe_read_kernel_str)(void *dst, __u32 size, const void *unsaf
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_tcp_send_ack)(void *tp, __u32 rcv_nxt) = (void *) 116;
static long (*bpf_tcp_send_ack)(void *tp, __u32 rcv_nxt) = (void *) 116;
/*
* bpf_send_signal_thread
@@ -2804,7 +2814,7 @@ static int (*bpf_tcp_send_ack)(void *tp, __u32 rcv_nxt) = (void *) 116;
*
* **-EAGAIN** if bpf program can try again.
*/
static int (*bpf_send_signal_thread)(__u32 sig) = (void *) 117;
static long (*bpf_send_signal_thread)(__u32 sig) = (void *) 117;
/*
* bpf_jiffies64
@@ -2837,7 +2847,7 @@ static __u64 (*bpf_jiffies64)(void) = (void *) 118;
*
* **-ENOENT** if architecture does not support branch records.
*/
static int (*bpf_read_branch_records)(struct bpf_perf_event_data *ctx, void *buf, __u32 size, __u64 flags) = (void *) 119;
static long (*bpf_read_branch_records)(struct bpf_perf_event_data *ctx, void *buf, __u32 size, __u64 flags) = (void *) 119;
/*
* bpf_get_ns_current_pid_tgid
@@ -2853,7 +2863,7 @@ static int (*bpf_read_branch_records)(struct bpf_perf_event_data *ctx, void *buf
*
* **-ENOENT** if pidns does not exists for the current task.
*/
static int (*bpf_get_ns_current_pid_tgid)(__u64 dev, __u64 ino, struct bpf_pidns_info *nsdata, __u32 size) = (void *) 120;
static long (*bpf_get_ns_current_pid_tgid)(__u64 dev, __u64 ino, struct bpf_pidns_info *nsdata, __u32 size) = (void *) 120;
/*
* bpf_xdp_output
@@ -2881,7 +2891,7 @@ static int (*bpf_get_ns_current_pid_tgid)(__u64 dev, __u64 ino, struct bpf_pidns
* Returns
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_xdp_output)(void *ctx, void *map, __u64 flags, void *data, __u64 size) = (void *) 121;
static long (*bpf_xdp_output)(void *ctx, void *map, __u64 flags, void *data, __u64 size) = (void *) 121;
/*
* bpf_get_netns_cookie
@@ -2925,6 +2935,10 @@ static __u64 (*bpf_get_current_ancestor_cgroup_id)(int ancestor_level) = (void *
/*
* bpf_sk_assign
*
* Helper is overloaded depending on BPF program type. This
* description applies to **BPF_PROG_TYPE_SCHED_CLS** and
* **BPF_PROG_TYPE_SCHED_ACT** programs.
*
* Assign the *sk* to the *skb*. When combined with appropriate
* routing configuration to receive the packet towards the socket,
* will cause *skb* to be delivered to the specified socket.
@@ -2951,7 +2965,7 @@ static __u64 (*bpf_get_current_ancestor_cgroup_id)(int ancestor_level) = (void *
* **-ESOCKTNOSUPPORT** if the socket type is not supported
* (reuseport).
*/
static int (*bpf_sk_assign)(struct __sk_buff *skb, struct bpf_sock *sk, __u64 flags) = (void *) 124;
static long (*bpf_sk_assign)(void *ctx, struct bpf_sock *sk, __u64 flags) = (void *) 124;
/*
* bpf_ktime_get_boot_ns
@@ -2996,7 +3010,7 @@ static __u64 (*bpf_ktime_get_boot_ns)(void) = (void *) 125;
*
* **-EOVERFLOW** if an overflow happened: The same object will be tried again.
*/
static int (*bpf_seq_printf)(struct seq_file *m, const char *fmt, __u32 fmt_size, const void *data, __u32 data_len) = (void *) 126;
static long (*bpf_seq_printf)(struct seq_file *m, const char *fmt, __u32 fmt_size, const void *data, __u32 data_len) = (void *) 126;
/*
* bpf_seq_write
@@ -3010,7 +3024,7 @@ static int (*bpf_seq_printf)(struct seq_file *m, const char *fmt, __u32 fmt_size
*
* **-EOVERFLOW** if an overflow happened: The same object will be tried again.
*/
static int (*bpf_seq_write)(struct seq_file *m, const void *data, __u32 len) = (void *) 127;
static long (*bpf_seq_write)(struct seq_file *m, const void *data, __u32 len) = (void *) 127;
/*
* bpf_sk_cgroup_id
@@ -3056,16 +3070,15 @@ static __u64 (*bpf_sk_ancestor_cgroup_id)(struct bpf_sock *sk, int ancestor_leve
* bpf_ringbuf_output
*
* Copy *size* bytes from *data* into a ring buffer *ringbuf*.
* If BPF_RB_NO_WAKEUP is specified in *flags*, no notification of
* new data availability is sent.
* IF BPF_RB_FORCE_WAKEUP is specified in *flags*, notification of
* new data availability is sent unconditionally.
* If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
* of new data availability is sent.
* If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
* of new data availability is sent unconditionally.
*
* Returns
* 0, on success;
* < 0, on error.
* 0 on success, or a negative error in case of failure.
*/
static int (*bpf_ringbuf_output)(void *ringbuf, void *data, __u64 size, __u64 flags) = (void *) 130;
static long (*bpf_ringbuf_output)(void *ringbuf, void *data, __u64 size, __u64 flags) = (void *) 130;
/*
* bpf_ringbuf_reserve
@@ -3082,10 +3095,10 @@ static void *(*bpf_ringbuf_reserve)(void *ringbuf, __u64 size, __u64 flags) = (v
* bpf_ringbuf_submit
*
* Submit reserved ring buffer sample, pointed to by *data*.
* If BPF_RB_NO_WAKEUP is specified in *flags*, no notification of
* new data availability is sent.
* IF BPF_RB_FORCE_WAKEUP is specified in *flags*, notification of
* new data availability is sent unconditionally.
* If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
* of new data availability is sent.
* If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
* of new data availability is sent unconditionally.
*
* Returns
* Nothing. Always succeeds.
@@ -3096,10 +3109,10 @@ static void (*bpf_ringbuf_submit)(void *data, __u64 flags) = (void *) 132;
* bpf_ringbuf_discard
*
* Discard reserved ring buffer sample, pointed to by *data*.
* If BPF_RB_NO_WAKEUP is specified in *flags*, no notification of
* new data availability is sent.
* IF BPF_RB_FORCE_WAKEUP is specified in *flags*, notification of
* new data availability is sent unconditionally.
* If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
* of new data availability is sent.
* If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
* of new data availability is sent unconditionally.
*
* Returns
* Nothing. Always succeeds.
@@ -3111,17 +3124,19 @@ static void (*bpf_ringbuf_discard)(void *data, __u64 flags) = (void *) 133;
*
* Query various characteristics of provided ring buffer. What
* exactly is queries is determined by *flags*:
* - BPF_RB_AVAIL_DATA - amount of data not yet consumed;
* - BPF_RB_RING_SIZE - the size of ring buffer;
* - BPF_RB_CONS_POS - consumer position (can wrap around);
* - BPF_RB_PROD_POS - producer(s) position (can wrap around);
* Data returned is just a momentary snapshots of actual values
*
* * **BPF_RB_AVAIL_DATA**: Amount of data not yet consumed.
* * **BPF_RB_RING_SIZE**: The size of ring buffer.
* * **BPF_RB_CONS_POS**: Consumer position (can wrap around).
* * **BPF_RB_PROD_POS**: Producer(s) position (can wrap around).
*
* Data returned is just a momentary snapshot of actual values
* and could be inaccurate, so this facility should be used to
* power heuristics and for reporting, not to make 100% correct
* calculation.
*
* Returns
* Requested value, or 0, if flags are not recognized.
* Requested value, or 0, if *flags* are not recognized.
*/
static __u64 (*bpf_ringbuf_query)(void *ringbuf, __u64 flags) = (void *) 134;
@@ -3159,6 +3174,92 @@ static __u64 (*bpf_ringbuf_query)(void *ringbuf, __u64 flags) = (void *) 134;
* is returned or the error code -EACCES in case the skb is not
* subject to CHECKSUM_UNNECESSARY.
*/
static int (*bpf_csum_level)(struct __sk_buff *skb, __u64 level) = (void *) 135;
static long (*bpf_csum_level)(struct __sk_buff *skb, __u64 level) = (void *) 135;
/*
* bpf_skc_to_tcp6_sock
*
* Dynamically cast a *sk* pointer to a *tcp6_sock* pointer.
*
* Returns
* *sk* if casting is valid, or NULL otherwise.
*/
static struct tcp6_sock *(*bpf_skc_to_tcp6_sock)(void *sk) = (void *) 136;
/*
* bpf_skc_to_tcp_sock
*
* Dynamically cast a *sk* pointer to a *tcp_sock* pointer.
*
* Returns
* *sk* if casting is valid, or NULL otherwise.
*/
static struct tcp_sock *(*bpf_skc_to_tcp_sock)(void *sk) = (void *) 137;
/*
* bpf_skc_to_tcp_timewait_sock
*
* Dynamically cast a *sk* pointer to a *tcp_timewait_sock* pointer.
*
* Returns
* *sk* if casting is valid, or NULL otherwise.
*/
static struct tcp_timewait_sock *(*bpf_skc_to_tcp_timewait_sock)(void *sk) = (void *) 138;
/*
* bpf_skc_to_tcp_request_sock
*
* Dynamically cast a *sk* pointer to a *tcp_request_sock* pointer.
*
* Returns
* *sk* if casting is valid, or NULL otherwise.
*/
static struct tcp_request_sock *(*bpf_skc_to_tcp_request_sock)(void *sk) = (void *) 139;
/*
* bpf_skc_to_udp6_sock
*
* Dynamically cast a *sk* pointer to a *udp6_sock* pointer.
*
* Returns
* *sk* if casting is valid, or NULL otherwise.
*/
static struct udp6_sock *(*bpf_skc_to_udp6_sock)(void *sk) = (void *) 140;
/*
* bpf_get_task_stack
*
* Return a user or a kernel stack in bpf program provided buffer.
* To achieve this, the helper needs *task*, which is a valid
* pointer to struct task_struct. To store the stacktrace, the
* bpf program provides *buf* with a nonnegative *size*.
*
* The last argument, *flags*, holds the number of stack frames to
* skip (from 0 to 255), masked with
* **BPF_F_SKIP_FIELD_MASK**. The next bits can be used to set
* the following flags:
*
* **BPF_F_USER_STACK**
* Collect a user space stack instead of a kernel stack.
* **BPF_F_USER_BUILD_ID**
* Collect buildid+offset instead of ips for user stack,
* only valid if **BPF_F_USER_STACK** is also specified.
*
* **bpf_get_task_stack**\ () can collect up to
* **PERF_MAX_STACK_DEPTH** both kernel and user frames, subject
* to sufficient large buffer size. Note that
* this limit can be controlled with the **sysctl** program, and
* that it should be manually increased in order to profile long
* user stacks (such as stacks for Java programs). To do so, use:
*
* ::
*
* # sysctl kernel.perf_event_max_stack=<new value>
*
* Returns
* A non-negative value equal to or less than *size* on success,
* or a negative error in case of failure.
*/
static long (*bpf_get_task_stack)(struct task_struct *task, void *buf, __u32 size, __u64 flags) = (void *) 141;

View File

@@ -40,7 +40,7 @@
* Helper macro to manipulate data structures
*/
#ifndef offsetof
#define offsetof(TYPE, MEMBER) ((size_t)&((TYPE *)0)->MEMBER)
#define offsetof(TYPE, MEMBER) ((unsigned long)&((TYPE *)0)->MEMBER)
#endif
#ifndef container_of
#define container_of(ptr, type, member) \
@@ -75,5 +75,6 @@ enum libbpf_tristate {
};
#define __kconfig __attribute__((section(".kconfig")))
#define __ksym __attribute__((section(".ksyms")))
#endif

View File

@@ -215,7 +215,7 @@ struct pt_regs;
#define PT_REGS_PARM5(x) ((x)->regs[8])
#define PT_REGS_RET(x) ((x)->regs[31])
#define PT_REGS_FP(x) ((x)->regs[30]) /* Works only with CONFIG_FRAME_POINTER */
#define PT_REGS_RC(x) ((x)->regs[1])
#define PT_REGS_RC(x) ((x)->regs[2])
#define PT_REGS_SP(x) ((x)->regs[29])
#define PT_REGS_IP(x) ((x)->cp0_epc)
@@ -226,7 +226,7 @@ struct pt_regs;
#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ((x), regs[8])
#define PT_REGS_RET_CORE(x) BPF_CORE_READ((x), regs[31])
#define PT_REGS_FP_CORE(x) BPF_CORE_READ((x), regs[30])
#define PT_REGS_RC_CORE(x) BPF_CORE_READ((x), regs[1])
#define PT_REGS_RC_CORE(x) BPF_CORE_READ((x), regs[2])
#define PT_REGS_SP_CORE(x) BPF_CORE_READ((x), regs[29])
#define PT_REGS_IP_CORE(x) BPF_CORE_READ((x), cp0_epc)

210
src/btf.c
View File

@@ -41,6 +41,7 @@ struct btf {
__u32 types_size;
__u32 data_size;
int fd;
int ptr_sz;
};
static inline __u64 ptr_to_u64(const void *ptr)
@@ -221,6 +222,70 @@ const struct btf_type *btf__type_by_id(const struct btf *btf, __u32 type_id)
return btf->types[type_id];
}
static int determine_ptr_size(const struct btf *btf)
{
const struct btf_type *t;
const char *name;
int i;
for (i = 1; i <= btf->nr_types; i++) {
t = btf__type_by_id(btf, i);
if (!btf_is_int(t))
continue;
name = btf__name_by_offset(btf, t->name_off);
if (!name)
continue;
if (strcmp(name, "long int") == 0 ||
strcmp(name, "long unsigned int") == 0) {
if (t->size != 4 && t->size != 8)
continue;
return t->size;
}
}
return -1;
}
static size_t btf_ptr_sz(const struct btf *btf)
{
if (!btf->ptr_sz)
((struct btf *)btf)->ptr_sz = determine_ptr_size(btf);
return btf->ptr_sz < 0 ? sizeof(void *) : btf->ptr_sz;
}
/* Return pointer size this BTF instance assumes. The size is heuristically
* determined by looking for 'long' or 'unsigned long' integer type and
* recording its size in bytes. If BTF type information doesn't have any such
* type, this function returns 0. In the latter case, native architecture's
* pointer size is assumed, so will be either 4 or 8, depending on
* architecture that libbpf was compiled for. It's possible to override
* guessed value by using btf__set_pointer_size() API.
*/
size_t btf__pointer_size(const struct btf *btf)
{
if (!btf->ptr_sz)
((struct btf *)btf)->ptr_sz = determine_ptr_size(btf);
if (btf->ptr_sz < 0)
/* not enough BTF type info to guess */
return 0;
return btf->ptr_sz;
}
/* Override or set pointer size in bytes. Only values of 4 and 8 are
* supported.
*/
int btf__set_pointer_size(struct btf *btf, size_t ptr_sz)
{
if (ptr_sz != 4 && ptr_sz != 8)
return -EINVAL;
btf->ptr_sz = ptr_sz;
return 0;
}
static bool btf_type_is_void(const struct btf_type *t)
{
return t == &btf_void || btf_is_fwd(t);
@@ -253,7 +318,7 @@ __s64 btf__resolve_size(const struct btf *btf, __u32 type_id)
size = t->size;
goto done;
case BTF_KIND_PTR:
size = sizeof(void *);
size = btf_ptr_sz(btf);
goto done;
case BTF_KIND_TYPEDEF:
case BTF_KIND_VOLATILE:
@@ -293,9 +358,9 @@ int btf__align_of(const struct btf *btf, __u32 id)
switch (kind) {
case BTF_KIND_INT:
case BTF_KIND_ENUM:
return min(sizeof(void *), (size_t)t->size);
return min(btf_ptr_sz(btf), (size_t)t->size);
case BTF_KIND_PTR:
return sizeof(void *);
return btf_ptr_sz(btf);
case BTF_KIND_TYPEDEF:
case BTF_KIND_VOLATILE:
case BTF_KIND_CONST:
@@ -386,10 +451,10 @@ __s32 btf__find_by_name_kind(const struct btf *btf, const char *type_name,
void btf__free(struct btf *btf)
{
if (!btf)
if (IS_ERR_OR_NULL(btf))
return;
if (btf->fd != -1)
if (btf->fd >= 0)
close(btf->fd);
free(btf->data);
@@ -397,7 +462,7 @@ void btf__free(struct btf *btf)
free(btf);
}
struct btf *btf__new(__u8 *data, __u32 size)
struct btf *btf__new(const void *data, __u32 size)
{
struct btf *btf;
int err;
@@ -533,6 +598,18 @@ struct btf *btf__parse_elf(const char *path, struct btf_ext **btf_ext)
if (IS_ERR(btf))
goto done;
switch (gelf_getclass(elf)) {
case ELFCLASS32:
btf__set_pointer_size(btf, 4);
break;
case ELFCLASS64:
btf__set_pointer_size(btf, 8);
break;
default:
pr_warn("failed to get ELF class (bitness) for %s\n", path);
break;
}
if (btf_ext && btf_ext_data) {
*btf_ext = btf_ext__new(btf_ext_data->d_buf,
btf_ext_data->d_size);
@@ -562,6 +639,83 @@ done:
return btf;
}
struct btf *btf__parse_raw(const char *path)
{
struct btf *btf = NULL;
void *data = NULL;
FILE *f = NULL;
__u16 magic;
int err = 0;
long sz;
f = fopen(path, "rb");
if (!f) {
err = -errno;
goto err_out;
}
/* check BTF magic */
if (fread(&magic, 1, sizeof(magic), f) < sizeof(magic)) {
err = -EIO;
goto err_out;
}
if (magic != BTF_MAGIC) {
/* definitely not a raw BTF */
err = -EPROTO;
goto err_out;
}
/* get file size */
if (fseek(f, 0, SEEK_END)) {
err = -errno;
goto err_out;
}
sz = ftell(f);
if (sz < 0) {
err = -errno;
goto err_out;
}
/* rewind to the start */
if (fseek(f, 0, SEEK_SET)) {
err = -errno;
goto err_out;
}
/* pre-alloc memory and read all of BTF data */
data = malloc(sz);
if (!data) {
err = -ENOMEM;
goto err_out;
}
if (fread(data, 1, sz, f) < sz) {
err = -EIO;
goto err_out;
}
/* finally parse BTF data */
btf = btf__new(data, sz);
err_out:
free(data);
if (f)
fclose(f);
return err ? ERR_PTR(err) : btf;
}
struct btf *btf__parse(const char *path, struct btf_ext **btf_ext)
{
struct btf *btf;
if (btf_ext)
*btf_ext = NULL;
btf = btf__parse_raw(path);
if (!IS_ERR(btf) || PTR_ERR(btf) != -EPROTO)
return btf;
return btf__parse_elf(path, btf_ext);
}
static int compare_vsi_off(const void *_a, const void *_b)
{
const struct btf_var_secinfo *a = _a;
@@ -700,6 +854,11 @@ int btf__fd(const struct btf *btf)
return btf->fd;
}
void btf__set_fd(struct btf *btf, int fd)
{
btf->fd = fd;
}
const void *btf__get_raw_data(const struct btf *btf, __u32 *size)
{
*size = btf->data_size;
@@ -1020,7 +1179,7 @@ static int btf_ext_parse_hdr(__u8 *data, __u32 data_size)
void btf_ext__free(struct btf_ext *btf_ext)
{
if (!btf_ext)
if (IS_ERR_OR_NULL(btf_ext))
return;
free(btf_ext->data);
free(btf_ext);
@@ -2946,41 +3105,6 @@ static int btf_dedup_remap_types(struct btf_dedup *d)
return 0;
}
static struct btf *btf_load_raw(const char *path)
{
struct btf *btf;
size_t read_cnt;
struct stat st;
void *data;
FILE *f;
if (stat(path, &st))
return ERR_PTR(-errno);
data = malloc(st.st_size);
if (!data)
return ERR_PTR(-ENOMEM);
f = fopen(path, "rb");
if (!f) {
btf = ERR_PTR(-errno);
goto cleanup;
}
read_cnt = fread(data, 1, st.st_size, f);
fclose(f);
if (read_cnt < st.st_size) {
btf = ERR_PTR(-EBADF);
goto cleanup;
}
btf = btf__new(data, read_cnt);
cleanup:
free(data);
return btf;
}
/*
* Probe few well-known locations for vmlinux kernel image and try to load BTF
* data out of it to use for target BTF.
@@ -3016,7 +3140,7 @@ struct btf *libbpf_find_kernel_btf(void)
continue;
if (locations[i].raw_btf)
btf = btf_load_raw(path);
btf = btf__parse_raw(path);
else
btf = btf__parse_elf(path, NULL);

View File

@@ -63,9 +63,10 @@ struct btf_ext_header {
};
LIBBPF_API void btf__free(struct btf *btf);
LIBBPF_API struct btf *btf__new(__u8 *data, __u32 size);
LIBBPF_API struct btf *btf__parse_elf(const char *path,
struct btf_ext **btf_ext);
LIBBPF_API struct btf *btf__new(const void *data, __u32 size);
LIBBPF_API struct btf *btf__parse(const char *path, struct btf_ext **btf_ext);
LIBBPF_API struct btf *btf__parse_elf(const char *path, struct btf_ext **btf_ext);
LIBBPF_API struct btf *btf__parse_raw(const char *path);
LIBBPF_API int btf__finalize_data(struct bpf_object *obj, struct btf *btf);
LIBBPF_API int btf__load(struct btf *btf);
LIBBPF_API __s32 btf__find_by_name(const struct btf *btf,
@@ -75,10 +76,13 @@ LIBBPF_API __s32 btf__find_by_name_kind(const struct btf *btf,
LIBBPF_API __u32 btf__get_nr_types(const struct btf *btf);
LIBBPF_API const struct btf_type *btf__type_by_id(const struct btf *btf,
__u32 id);
LIBBPF_API size_t btf__pointer_size(const struct btf *btf);
LIBBPF_API int btf__set_pointer_size(struct btf *btf, size_t ptr_sz);
LIBBPF_API __s64 btf__resolve_size(const struct btf *btf, __u32 type_id);
LIBBPF_API int btf__resolve_type(const struct btf *btf, __u32 type_id);
LIBBPF_API int btf__align_of(const struct btf *btf, __u32 id);
LIBBPF_API int btf__fd(const struct btf *btf);
LIBBPF_API void btf__set_fd(struct btf *btf, int fd);
LIBBPF_API const void *btf__get_raw_data(const struct btf *btf, __u32 *size);
LIBBPF_API const char *btf__name_by_offset(const struct btf *btf, __u32 offset);
LIBBPF_API int btf__get_from_id(__u32 id, struct btf **btf);
@@ -143,8 +147,10 @@ struct btf_dump_emit_type_decl_opts {
* necessary indentation already
*/
int indent_level;
/* strip all the const/volatile/restrict mods */
bool strip_mods;
};
#define btf_dump_emit_type_decl_opts__last_field indent_level
#define btf_dump_emit_type_decl_opts__last_field strip_mods
LIBBPF_API int
btf_dump__emit_type_decl(struct btf_dump *d, __u32 id,
@@ -168,6 +174,11 @@ static inline bool btf_kflag(const struct btf_type *t)
return BTF_INFO_KFLAG(t->info);
}
static inline bool btf_is_void(const struct btf_type *t)
{
return btf_kind(t) == BTF_KIND_UNKN;
}
static inline bool btf_is_int(const struct btf_type *t)
{
return btf_kind(t) == BTF_KIND_INT;

View File

@@ -13,6 +13,7 @@
#include <errno.h>
#include <linux/err.h>
#include <linux/btf.h>
#include <linux/kernel.h>
#include "btf.h"
#include "hashmap.h"
#include "libbpf.h"
@@ -60,6 +61,8 @@ struct btf_dump {
const struct btf_ext *btf_ext;
btf_dump_printf_fn_t printf_fn;
struct btf_dump_opts opts;
int ptr_sz;
bool strip_mods;
/* per-type auxiliary state */
struct btf_dump_type_aux_state *type_states;
@@ -137,6 +140,7 @@ struct btf_dump *btf_dump__new(const struct btf *btf,
d->btf_ext = btf_ext;
d->printf_fn = printf_fn;
d->opts.ctx = opts ? opts->ctx : NULL;
d->ptr_sz = btf__pointer_size(btf) ? : sizeof(void *);
d->type_names = hashmap__new(str_hash_fn, str_equal_fn, NULL);
if (IS_ERR(d->type_names)) {
@@ -182,7 +186,7 @@ void btf_dump__free(struct btf_dump *d)
{
int i, cnt;
if (!d)
if (IS_ERR_OR_NULL(d))
return;
free(d->type_states);
@@ -548,6 +552,9 @@ static int btf_dump_order_type(struct btf_dump *d, __u32 id, bool through_ptr)
}
}
static void btf_dump_emit_missing_aliases(struct btf_dump *d, __u32 id,
const struct btf_type *t);
static void btf_dump_emit_struct_fwd(struct btf_dump *d, __u32 id,
const struct btf_type *t);
static void btf_dump_emit_struct_def(struct btf_dump *d, __u32 id,
@@ -670,6 +677,9 @@ static void btf_dump_emit_type(struct btf_dump *d, __u32 id, __u32 cont_id)
switch (kind) {
case BTF_KIND_INT:
/* Emit type alias definitions if necessary */
btf_dump_emit_missing_aliases(d, id, t);
tstate->emit_state = EMITTED;
break;
case BTF_KIND_ENUM:
@@ -796,7 +806,7 @@ static void btf_dump_emit_bit_padding(const struct btf_dump *d,
int align, int lvl)
{
int off_diff = m_off - cur_off;
int ptr_bits = sizeof(void *) * 8;
int ptr_bits = d->ptr_sz * 8;
if (off_diff <= 0)
/* no gap */
@@ -869,7 +879,7 @@ static void btf_dump_emit_struct_def(struct btf_dump *d,
btf_dump_printf(d, ": %d", m_sz);
off = m_off + m_sz;
} else {
m_sz = max(0, btf__resolve_size(d->btf, m->type));
m_sz = max((__s64)0, btf__resolve_size(d->btf, m->type));
off = m_off + m_sz * 8;
}
btf_dump_printf(d, ";");
@@ -889,6 +899,32 @@ static void btf_dump_emit_struct_def(struct btf_dump *d,
btf_dump_printf(d, " __attribute__((packed))");
}
static const char *missing_base_types[][2] = {
/*
* GCC emits typedefs to its internal __PolyX_t types when compiling Arm
* SIMD intrinsics. Alias them to standard base types.
*/
{ "__Poly8_t", "unsigned char" },
{ "__Poly16_t", "unsigned short" },
{ "__Poly64_t", "unsigned long long" },
{ "__Poly128_t", "unsigned __int128" },
};
static void btf_dump_emit_missing_aliases(struct btf_dump *d, __u32 id,
const struct btf_type *t)
{
const char *name = btf_dump_type_name(d, id);
int i;
for (i = 0; i < ARRAY_SIZE(missing_base_types); i++) {
if (strcmp(name, missing_base_types[i][0]) == 0) {
btf_dump_printf(d, "typedef %s %s;\n\n",
missing_base_types[i][1], name);
break;
}
}
}
static void btf_dump_emit_enum_fwd(struct btf_dump *d, __u32 id,
const struct btf_type *t)
{
@@ -1032,7 +1068,9 @@ int btf_dump__emit_type_decl(struct btf_dump *d, __u32 id,
fname = OPTS_GET(opts, field_name, "");
lvl = OPTS_GET(opts, indent_level, 0);
d->strip_mods = OPTS_GET(opts, strip_mods, false);
btf_dump_emit_type_decl(d, id, fname, lvl);
d->strip_mods = false;
return 0;
}
@@ -1045,6 +1083,10 @@ static void btf_dump_emit_type_decl(struct btf_dump *d, __u32 id,
stack_start = d->decl_stack_cnt;
for (;;) {
t = btf__type_by_id(d->btf, id);
if (d->strip_mods && btf_is_mod(t))
goto skip_mod;
err = btf_dump_push_decl_stack_id(d, id);
if (err < 0) {
/*
@@ -1056,12 +1098,11 @@ static void btf_dump_emit_type_decl(struct btf_dump *d, __u32 id,
d->decl_stack_cnt = stack_start;
return;
}
skip_mod:
/* VOID */
if (id == 0)
break;
t = btf__type_by_id(d->btf, id);
switch (btf_kind(t)) {
case BTF_KIND_PTR:
case BTF_KIND_VOLATILE:

View File

@@ -11,14 +11,18 @@
#include <stdbool.h>
#include <stddef.h>
#include <limits.h>
#ifndef __WORDSIZE
#define __WORDSIZE (__SIZEOF_LONG__ * 8)
#endif
static inline size_t hash_bits(size_t h, int bits)
{
/* shuffle bits and return requested number of upper bits */
return (h * 11400714819323198485llu) >> (__WORDSIZE - bits);
#if (__SIZEOF_SIZE_T__ == __SIZEOF_LONG_LONG__)
/* LP64 case */
return (h * 11400714819323198485llu) >> (__SIZEOF_LONG_LONG__ * 8 - bits);
#elif (__SIZEOF_SIZE_T__ <= __SIZEOF_LONG__)
return (h * 2654435769lu) >> (__SIZEOF_LONG__ * 8 - bits);
#else
# error "Unsupported size_t size"
#endif
}
typedef size_t (*hashmap_hash_fn)(const void *key, void *ctx);

File diff suppressed because it is too large Load Diff

View File

@@ -200,6 +200,8 @@ LIBBPF_API void bpf_program__set_ifindex(struct bpf_program *prog,
LIBBPF_API const char *bpf_program__name(const struct bpf_program *prog);
LIBBPF_API const char *bpf_program__title(const struct bpf_program *prog,
bool needs_copy);
LIBBPF_API bool bpf_program__autoload(const struct bpf_program *prog);
LIBBPF_API int bpf_program__set_autoload(struct bpf_program *prog, bool autoload);
/* returns program size in bytes */
LIBBPF_API size_t bpf_program__size(const struct bpf_program *prog);
@@ -227,6 +229,7 @@ LIBBPF_API int bpf_link__unpin(struct bpf_link *link);
LIBBPF_API int bpf_link__update_program(struct bpf_link *link,
struct bpf_program *prog);
LIBBPF_API void bpf_link__disconnect(struct bpf_link *link);
LIBBPF_API int bpf_link__detach(struct bpf_link *link);
LIBBPF_API int bpf_link__destroy(struct bpf_link *link);
LIBBPF_API struct bpf_link *
@@ -255,6 +258,8 @@ LIBBPF_API struct bpf_link *
bpf_program__attach_cgroup(struct bpf_program *prog, int cgroup_fd);
LIBBPF_API struct bpf_link *
bpf_program__attach_netns(struct bpf_program *prog, int netns_fd);
LIBBPF_API struct bpf_link *
bpf_program__attach_xdp(struct bpf_program *prog, int ifindex);
struct bpf_map;
@@ -262,8 +267,10 @@ LIBBPF_API struct bpf_link *bpf_map__attach_struct_ops(struct bpf_map *map);
struct bpf_iter_attach_opts {
size_t sz; /* size of this struct for forward/backward compatibility */
union bpf_iter_link_info *link_info;
__u32 link_info_len;
};
#define bpf_iter_attach_opts__last_field sz
#define bpf_iter_attach_opts__last_field link_info_len
LIBBPF_API struct bpf_link *
bpf_program__attach_iter(struct bpf_program *prog,
@@ -348,6 +355,7 @@ LIBBPF_API int bpf_program__set_perf_event(struct bpf_program *prog);
LIBBPF_API int bpf_program__set_tracing(struct bpf_program *prog);
LIBBPF_API int bpf_program__set_struct_ops(struct bpf_program *prog);
LIBBPF_API int bpf_program__set_extension(struct bpf_program *prog);
LIBBPF_API int bpf_program__set_sk_lookup(struct bpf_program *prog);
LIBBPF_API enum bpf_prog_type bpf_program__get_type(struct bpf_program *prog);
LIBBPF_API void bpf_program__set_type(struct bpf_program *prog,
@@ -375,6 +383,7 @@ LIBBPF_API bool bpf_program__is_perf_event(const struct bpf_program *prog);
LIBBPF_API bool bpf_program__is_tracing(const struct bpf_program *prog);
LIBBPF_API bool bpf_program__is_struct_ops(const struct bpf_program *prog);
LIBBPF_API bool bpf_program__is_extension(const struct bpf_program *prog);
LIBBPF_API bool bpf_program__is_sk_lookup(const struct bpf_program *prog);
/*
* No need for __attribute__((packed)), all members of 'bpf_map_def'
@@ -418,11 +427,38 @@ bpf_map__next(const struct bpf_map *map, const struct bpf_object *obj);
LIBBPF_API struct bpf_map *
bpf_map__prev(const struct bpf_map *map, const struct bpf_object *obj);
/* get/set map FD */
LIBBPF_API int bpf_map__fd(const struct bpf_map *map);
LIBBPF_API int bpf_map__reuse_fd(struct bpf_map *map, int fd);
/* get map definition */
LIBBPF_API const struct bpf_map_def *bpf_map__def(const struct bpf_map *map);
/* get map name */
LIBBPF_API const char *bpf_map__name(const struct bpf_map *map);
/* get/set map type */
LIBBPF_API enum bpf_map_type bpf_map__type(const struct bpf_map *map);
LIBBPF_API int bpf_map__set_type(struct bpf_map *map, enum bpf_map_type type);
/* get/set map size (max_entries) */
LIBBPF_API __u32 bpf_map__max_entries(const struct bpf_map *map);
LIBBPF_API int bpf_map__set_max_entries(struct bpf_map *map, __u32 max_entries);
LIBBPF_API int bpf_map__resize(struct bpf_map *map, __u32 max_entries);
/* get/set map flags */
LIBBPF_API __u32 bpf_map__map_flags(const struct bpf_map *map);
LIBBPF_API int bpf_map__set_map_flags(struct bpf_map *map, __u32 flags);
/* get/set map NUMA node */
LIBBPF_API __u32 bpf_map__numa_node(const struct bpf_map *map);
LIBBPF_API int bpf_map__set_numa_node(struct bpf_map *map, __u32 numa_node);
/* get/set map key size */
LIBBPF_API __u32 bpf_map__key_size(const struct bpf_map *map);
LIBBPF_API int bpf_map__set_key_size(struct bpf_map *map, __u32 size);
/* get/set map value size */
LIBBPF_API __u32 bpf_map__value_size(const struct bpf_map *map);
LIBBPF_API int bpf_map__set_value_size(struct bpf_map *map, __u32 size);
/* get map key/value BTF type IDs */
LIBBPF_API __u32 bpf_map__btf_key_type_id(const struct bpf_map *map);
LIBBPF_API __u32 bpf_map__btf_value_type_id(const struct bpf_map *map);
/* get/set map if_index */
LIBBPF_API __u32 bpf_map__ifindex(const struct bpf_map *map);
LIBBPF_API int bpf_map__set_ifindex(struct bpf_map *map, __u32 ifindex);
typedef void (*bpf_map_clear_priv_t)(struct bpf_map *, void *);
LIBBPF_API int bpf_map__set_priv(struct bpf_map *map, void *priv,
@@ -430,11 +466,8 @@ LIBBPF_API int bpf_map__set_priv(struct bpf_map *map, void *priv,
LIBBPF_API void *bpf_map__priv(const struct bpf_map *map);
LIBBPF_API int bpf_map__set_initial_value(struct bpf_map *map,
const void *data, size_t size);
LIBBPF_API int bpf_map__reuse_fd(struct bpf_map *map, int fd);
LIBBPF_API int bpf_map__resize(struct bpf_map *map, __u32 max_entries);
LIBBPF_API bool bpf_map__is_offload_neutral(const struct bpf_map *map);
LIBBPF_API bool bpf_map__is_internal(const struct bpf_map *map);
LIBBPF_API void bpf_map__set_ifindex(struct bpf_map *map, __u32 ifindex);
LIBBPF_API int bpf_map__set_pin_path(struct bpf_map *map, const char *path);
LIBBPF_API const char *bpf_map__get_pin_path(const struct bpf_map *map);
LIBBPF_API bool bpf_map__is_pinned(const struct bpf_map *map);

View File

@@ -270,3 +270,32 @@ LIBBPF_0.0.9 {
ring_buffer__new;
ring_buffer__poll;
} LIBBPF_0.0.8;
LIBBPF_0.1.0 {
global:
bpf_link__detach;
bpf_link_detach;
bpf_map__ifindex;
bpf_map__key_size;
bpf_map__map_flags;
bpf_map__max_entries;
bpf_map__numa_node;
bpf_map__set_key_size;
bpf_map__set_map_flags;
bpf_map__set_max_entries;
bpf_map__set_numa_node;
bpf_map__set_type;
bpf_map__set_value_size;
bpf_map__type;
bpf_map__value_size;
bpf_program__attach_xdp;
bpf_program__autoload;
bpf_program__is_sk_lookup;
bpf_program__set_autoload;
bpf_program__set_sk_lookup;
btf__parse;
btf__parse_raw;
btf__pointer_size;
btf__set_fd;
btf__set_pointer_size;
} LIBBPF_0.0.9;

View File

@@ -78,6 +78,9 @@ probe_load(enum bpf_prog_type prog_type, const struct bpf_insn *insns,
case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
xattr.expected_attach_type = BPF_CGROUP_INET4_CONNECT;
break;
case BPF_PROG_TYPE_SK_LOOKUP:
xattr.expected_attach_type = BPF_SK_LOOKUP;
break;
case BPF_PROG_TYPE_KPROBE:
xattr.kern_version = get_kernel_version();
break;

View File

@@ -2,7 +2,7 @@
set -euxo pipefail
LLVM_VER=11
LLVM_VER=12
LIBBPF_PATH="${REPO_ROOT}"
REPO_PATH="travis-ci/vmtest/bpf-next"
@@ -11,13 +11,20 @@ if [ -f "${PREPARE_SELFTESTS_SCRIPT}" ]; then
(cd "${REPO_ROOT}/${REPO_PATH}/tools/testing/selftests/bpf" && ${PREPARE_SELFTESTS_SCRIPT})
fi
if [[ "${KERNEL}" = 'LATEST' ]]; then
VMLINUX_H=
else
VMLINUX_H=${VMTEST_ROOT}/vmlinux.h
fi
make \
CLANG=clang-${LLVM_VER} \
LLC=llc-${LLVM_VER} \
LLVM_STRIP=llvm-strip-${LLVM_VER} \
VMLINUX_BTF="${VMLINUX_BTF}" \
VMLINUX_H=${VMLINUX_H} \
-C "${REPO_ROOT}/${REPO_PATH}/tools/testing/selftests/bpf" \
-j $((4*$(nproc)))
-j $((2*$(nproc)))
mkdir ${LIBBPF_PATH}/selftests
cp -R "${REPO_ROOT}/${REPO_PATH}/tools/testing/selftests/bpf" \
${LIBBPF_PATH}/selftests

View File

@@ -8,17 +8,31 @@ REPO_PATH=$1
BPF_NEXT_ORIGIN=https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
LINUX_SHA=$(cat ${LIBBPF_PATH}/CHECKPOINT-COMMIT)
SNAPSHOT_URL=https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/snapshot/bpf-next-${LINUX_SHA}.tar.gz
echo REPO_PATH = ${REPO_PATH}
echo LINUX_SHA = ${LINUX_SHA}
if [ ! -d "${REPO_PATH}" ]; then
mkdir -p ${REPO_PATH}
cd ${REPO_PATH}
git init
git remote add bpf-next ${BPF_NEXT_ORIGIN}
for depth in 32 64 128; do
git fetch --depth ${depth} bpf-next
git reset --hard ${LINUX_SHA} && break
done
mkdir -p $(dirname "${REPO_PATH}")
cd $(dirname "${REPO_PATH}")
# attempt to fetch desired bpf-next repo snapshot
if wget ${SNAPSHOT_URL} ; then
tar xf bpf-next-${LINUX_SHA}.tar.gz
mv bpf-next-${LINUX_SHA} $(basename ${REPO_PATH})
else
# but fallback to git fetch approach if that fails
mkdir -p $(basename ${REPO_PATH})
cd $(basename ${REPO_PATH})
git init
git remote add bpf-next ${BPF_NEXT_ORIGIN}
# try shallow clone first
git fetch --depth 32 bpf-next
# check if desired SHA exists
if ! git cat-file -e ${LINUX_SHA}^{commit} ; then
# if not, fetch all of bpf-next; slow and painful
git fetch bpf-next
fi
git reset --hard ${LINUX_SHA}
fi
fi

View File

@@ -1,6 +1,8 @@
INDEX https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/INDEX
libbpf-vmtest-rootfs-2020.03.11.tar.zst https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/libbpf-vmtest-rootfs-2020.03.11.tar.zst
vmlinux-4.9.0.zst https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/vmlinux-4.9.0.zst
vmlinux-5.5.0-rc6.zst https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/vmlinux-5.5.0-rc6.zst
vmlinux-5.5.0.zst https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/vmlinux-5.5.0.zst
vmlinuz-5.5.0-rc6 https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/vmlinuz-5.5.0-rc6
vmlinuz-5.5.0 https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/vmlinuz-5.5.0
vmlinuz-4.9.0 https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/vmlinuz-4.9.0

View File

@@ -9,7 +9,9 @@ bpf_verif_scale
#bpf_verif_scale/strobemeta.o
#bpf_verif_scale/strobemeta_nounroll1.o
#bpf_verif_scale/strobemeta_nounroll2.o
btf_map_in_map # inner map leak fixed in 5.8
cg_storage_multi # v5.9+ functionality
cgroup_attach_multi # BPF_F_REPLACE_PROG missing
cgroup_link # LINK_CREATE is missing
cgroup_skb_sk_lookup # bpf_sk_lookup_tcp() helper is missing
@@ -22,8 +24,10 @@ fexit_test # bpf_prog_test_tracing missing
flow_dissector # bpf_link-based flow dissector is in 5.8+
flow_dissector_reattach
get_stack_raw_tp # exercising BPF verifier bug causing infinite loop
ksyms # __start_BTF has different name
link_pinning # bpf_link is missing
load_bytes_relative # new functionality in 5.8
map_ptr # test uses BPF_MAP_TYPE_RINGBUF, added in 5.8
mmap # 5.5 kernel is too permissive with re-mmaping
modify_return # fmod_ret support is missing
ns_current_pid_tgid # bpf_get_ns_current_pid_tgid() helper is missing
@@ -37,15 +41,23 @@ reference_tracking
select_reuseport # UDP support is missing
sk_assign # bpf_sk_assign helper missing
skb_helpers # helpers added in 5.8+
sockmap_basic # uses new socket fields, 5.8+
sockmap_listen # no listen socket supportin SOCKMAP
sockopt_sk
sk_lookup # v5.9+
skb_ctx # ctx_{size, }_{in, out} in BPF_PROG_TEST_RUN is missing
test_global_funcs # kernel doesn't support BTF linkage=global on FUNCs
test_lsm # no BPF_LSM support
test_overhead # no fmod_ret support
udp_limit # no cgroup/sock_release BPF program type (5.9+)
varlen # verifier bug fixed in later kernels
vmlinux # hrtimer_nanosleep() signature changed incompatibly
xdp_adjust_tail # new XDP functionality added in 5.8
xdp_attach # IFLA_XDP_EXPECTED_FD support is missing
xdp_bpf2bpf # freplace is missing
xdp_cpumap_attach # v5.9+
xdp_devmap_attach # new feature in 5.8
xdp_link # v5.9+
# TEMPORARILY DISABLED

View File

@@ -2,4 +2,6 @@
send_signal # flaky
test_lsm # semi-working
sk_assign # needs better setup in Travis CI
sk_lookup
core_reloc # temporary test breakage
bpf_verif_scale # clang regression

View File

@@ -0,0 +1,7 @@
btf_dump
core_retro
cpu_mask
hashmap
perf_buffer
section_names

View File

@@ -1,12 +0,0 @@
#!/bin/bash
set -euxo pipefail
# these tests expect vmlinux.h to have latest defiition of bpf_devmap_val xdp_md->egress_ifindex
rm progs/test_xdp_with_devmap_helpers.c
rm progs/test_xdp_devmap_helpers.c
rm prog_tests/xdp_devmap_attach.c
# no BPF_F_NO_PREALLOC in BTF and no sk_msg_md->sk field
rm progs/test_skmsg_load_helpers.c
rm prog_tests/sockmap_basic.c

View File

@@ -3,13 +3,17 @@
set -euxo pipefail
test_progs() {
echo TEST_PROGS
./test_progs ${BLACKLIST:+-b$BLACKLIST} ${WHITELIST:+-t$WHITELIST}
if [[ "${KERNEL}" != '4.9.0' ]]; then
echo TEST_PROGS
./test_progs ${BLACKLIST:+-b$BLACKLIST} ${WHITELIST:+-t$WHITELIST}
fi
echo TEST_PROGS-NO_ALU32
./test_progs-no_alu32 ${BLACKLIST:+-b$BLACKLIST} ${WHITELIST:+-t$WHITELIST}
}
test_maps() {
echo TEST_MAPS
# Allow failing on older kernels.
./test_maps
}

View File

@@ -12,8 +12,8 @@ ${VMTEST_ROOT}/build_pahole.sh travis-ci/vmtest/pahole
# Install required packages
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
echo "deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic main" | sudo tee -a /etc/apt/sources.list
sudo apt-get -qq update
sudo apt-get -qq -y install clang lld llvm
sudo apt-get update
sudo apt-get -y install clang-12 lld-12 llvm-12
# Build selftests (and latest kernel, if necessary)
KERNEL="${KERNEL}" ${VMTEST_ROOT}/prepare_selftests.sh travis-ci/vmtest/bpf-next

82043
travis-ci/vmtest/vmlinux.h Normal file

File diff suppressed because it is too large Load Diff