Commit Graph

90 Commits

Author SHA1 Message Date
Magnus Karlsson
b0404eeba8 libbpf: remove dependency on barrier.h in xsk.h
The use of smp_rmb() and smp_wmb() creates a Linux header dependency
on barrier.h that is unnecessary in most parts. This patch implements
the two small defines that are needed from barrier.h. As a bonus, the
new implementations are faster than the default ones as they default
to sfence and lfence for x86, while we only need a compiler barrier in
our case. Just as it is when the same ring access code is compiled in
the kernel.

Fixes: 1cad07884239 ("libbpf: add support for using AF_XDP sockets")
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-19 09:52:37 -07:00
Magnus Karlsson
b0abd1bc24 libbpf: remove likely/unlikely in xsk.h
This patch removes the use of likely and unlikely in xsk.h since they
create a dependency on Linux headers as reported by several
users. There have also been reports that the use of these decreases
performance as the compiler puts the code on two different cache lines
instead of on a single one. All in all, I think we are better off
without them.

Fixes: 1cad07884239 ("libbpf: add support for using AF_XDP sockets")
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-19 09:52:37 -07:00
Magnus Karlsson
01d7b7ebbf libbpf: fix XDP socket ring buffer memory ordering
The ring buffer code of	XDP sockets is missing a memory	barrier	on the
consumer side between the load of the data and the write that signals
that it is ok for the producer to put new data into the buffer. On
architectures that does not guarantee that stores are not reordered
with older loads, the producer might put data into the ring before the
consumer had the chance to read it. As IA does guarantee this
ordering, it would only need a compiler barrier here, but there are no
primitives in barrier.h for this specific case (hinder writes to be ordered
before older reads) so I had to add a smp_mb() here which will
translate into a run-time synch operation on IA.

Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-19 09:52:37 -07:00
Andrii Nakryiko
6afcd645c8 libbpf: fix printf formatter for ptrdiff_t argument
Using %ld for printing out value of ptrdiff_t type is not portable
between 32-bit and 64-bit archs. This is causing compilation errors for
libbpf on 32-bit platform (discovered as part of an effort to integrate
libbpf into systemd ([0])). Proper formatter is %td, which is used in
this patch.

v2->v1:
  - add Reported-by
  - provide more context on how this issue was discovered

[0] https://github.com/systemd/systemd/pull/12151

Reported-by: Evgeny Vereshchagin <evvers@ya.ru>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-19 09:52:37 -07:00
Alan Maguire
7ee515fed9 bpf: fix whitespace for ENCAP_L2 defines in bpf.h
replace tab after #define with space in line with rest of definitions

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-19 09:52:37 -07:00
Andrii Nakryiko
f0e76d6e9e btf: add support for VAR and DATASEC in btf_dedup()
This patch adds support for VAR and DATASEC in btf_dedup(). VAR/DATASEC
are never deduplicated, but they need to be processed anyway as types
they refer to might need to be remapped due to deduplication and
compaction.

Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Yonghong Song <yhs@fb.com>
Cc: Alexei Starovoitov <ast@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-19 09:52:37 -07:00
Andrey Ignatov
6be72ba83c bpf: Sync bpf.h to tools/
Sync bpf_strtoX related bpf UAPI changes to tools/.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-19 09:52:37 -07:00
Andrey Ignatov
6cf6bed724 libbpf: Support sysctl hook
Support BPF_PROG_TYPE_CGROUP_SYSCTL program in libbpf: identifying
program and attach types by section name, probe.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-19 09:52:37 -07:00
Andrey Ignatov
805f4593af bpf: Sync bpf.h to tools/
Sync BPF_PROG_TYPE_CGROUP_SYSCTL related bpf UAPI changes to tools/.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-19 09:52:37 -07:00
Alan Maguire
b18e6122b6 bpf: sync bpf.h to tools/ for BPF_F_ADJ_ROOM_ENCAP_L2
Sync include/uapi/linux/bpf.h with tools/ equivalent to add
BPF_F_ADJ_ROOM_ENCAP_L2(len) macro.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-19 09:52:37 -07:00
Andrii Nakryiko
61419aae26 sync-kernel.sh: detect synced commits, interactively resolve conflicts
1. There is one commit ("libbpf: fix btf_dedup equivalence check
   handling of different kinds"), which was originally committed into bpf
   tree and manually synced into libbpf (it was a critical bug fix). Now
   that bpf and bpf-next trees were merged, this causes duplicate commits in
   libbpf. To address this, check is added to see if libbpf repo already has
   a commit with the same subject and allow to skip it.
2. Just checking by subject is not good enough, as we currently have few
   different commits with the same subject "bpf: Sync bpf.h to tools/".
   To address this, script now will ask user whether to skip or cherry-pick
   commit. This allows for human to double-check the commit and make
   informed devision.
3. There is a commit ('bfb35c27c65f bpf: fix whitespace for ENCAP_L2
   defines in bpf.h'), which modifies both include/uapi/linux/bpf.h and
   tools/include/uapi/linux/bpf.h in the same commit. This break the
   sync script, as it doesn't cherry-pick commits that hasn't modified
   libbpf files. So at the time that commit bfb35c27c65f is applied, linux
   repo is missing some preceding changes to include/uapi/linux/bpf.h. They
   would normally be ignored later during repo filtering, but to do
   cherry-pick they are necessary. It's hard to fix it automatically, so
   script will wait for user to resolve the issue and continue.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-04-18 17:00:48 -07:00
hex
2d1959ff60 Introduce Travis CI tests
In order to libbpf to be used in systemd some testing is required, see related
discussions in https://github.com/systemd/systemd/pull/12151 and
https://github.com/libbpf/libbpf/pull/29
The tests introduced here mirrors the tests of systemd:
For Debian: build with gcc, gcc + asan, clang, clang + asan
Debian tests use `docker` virtualization
Fror Ubuntu Xenial: build with gcc

The differences:
Install only libelf and it's dependencies.
Instead of Meson build system `make` is used, so `make` remains the preferred
method of building and `meson.build` doesn't get rooted in `libbpf`.

`travis_wait.bash` is kept as a workaround for
https://github.com/travis-ci/travis-ci/issues/9979

An example of testing UI: https://travis-ci.org/wat-ze-hex/libbpf
2019-04-17 14:19:40 -07:00
hex
65b4e9359a add meson.build for meson build system
Make it possible to include libbpf into projects built by Meson build system as
a subproject. This is the case e.g. for systemd.

meson.build declares dependencies which may be used in parent projects:
`libbpf_static_dep` for static library
`libbpf_shared_dep` for shared library
`bpf_includes` provides the location of bpf.h, btf.h and libbpf.h

Parent projects need to introduce git wrap to build libbpf:
https://github.com/mesonbuild/meson/blob/master/docs/markdown/Wrap-dependency-system-manual.md#wrap-git

An alternative approach is applying a patch with meson.build for libbpf while
building a parent project:
https://github.com/mesonbuild/meson/blob/master/docs/markdown/Wrap-dependency-system-manual.md#wrap-file
Imo it's less transparent and would add more headache.

meson.build is placed in the source root because that's how Meson expects it to
be placed.

The related discussion is in https://github.com/systemd/systemd/pull/12151
2019-04-15 21:39:11 -07:00
Daniel Borkmann
8bbb21c227 libbpf: fix extraversion in makefile to that of upstream
Backport in f49907472f ("sync: latest libbpf changes from kernel")
missed to bump Makefile's EXTRAVERSION to 3.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-12 08:19:53 -07:00
Daniel Borkmann
dd3fc8bde1 sync: latest tooling changes from kernel
[ upstream commit 6b7a21140fca461c6d8d5c65a3746e7da50a409e ]

Needed a custom backport. I've changed fallbacks so that we can
partially pull in barriers step by step. I've fixed x86 ones and
added corresponding arm64 barriers.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-11 15:55:59 -07:00
Yonghong Song
5844f6e4dd sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline commit:   50bd645b3a21a374dbd0fa8273a5f4e98001fb05
Checkpoint commit: 3daf8e703ec3dcf73a27a7dcabbac152793eb114

Andrey Ignatov (1):
  libbpf: Fix build with gcc-8

Stanislav Fomichev (1):
  libbpf: add support for ctx_{size, }_{in, out} in BPF_PROG_TEST_RUN

 include/uapi/linux/bpf.h | 7 +++++++
 src/bpf.c                | 5 +++++
 src/bpf.h                | 5 +++++
 src/libbpf.c             | 2 +-
 4 files changed, 18 insertions(+), 1 deletion(-)

--
2.17.1
2019-04-11 09:01:12 -07:00
Stanislav Fomichev
e2871d1c75 libbpf: add support for ctx_{size, }_{in, out} in BPF_PROG_TEST_RUN
Support recently introduced input/output context for test runs.
We extend only bpf_prog_test_run_xattr. bpf_prog_test_run is
unextendable and left as is.

Signed-off-by: Stanislav Fomichev <sdf@google.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-11 09:01:12 -07:00
Andrey Ignatov
ac2254d0ea libbpf: Fix build with gcc-8
Reported in [1].

With gcc 8.3.0 the following error is issued:

  cc -Ibpf@sta -I. -I.. -I.././include -I.././include/uapi
  -fdiagnostics-color=always -fsanitize=address,undefined -fno-omit-frame-pointer
  -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -g -fPIC -g -O2
  -Werror -Wall -Wno-pointer-arith -Wno-sign-compare  -MD -MQ
  'bpf@sta/src_libbpf.c.o' -MF 'bpf@sta/src_libbpf.c.o.d' -o
  'bpf@sta/src_libbpf.c.o' -c ../src/libbpf.c
  ../src/libbpf.c: In function 'bpf_object__elf_collect':
  ../src/libbpf.c:947:18: error: 'map_def_sz' may be used uninitialized in this
  function [-Werror=maybe-uninitialized]
     if (map_def_sz <= sizeof(struct bpf_map_def)) {
         ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ../src/libbpf.c:827:18: note: 'map_def_sz' was declared here
    int i, map_idx, map_def_sz, nr_syms, nr_maps = 0, nr_maps_glob = 0;
                    ^~~~~~~~~~

According to [2] -Wmaybe-uninitialized is enabled by -Wall.
Same error is generated by clang's -Wconditional-uninitialized.

[1] https://github.com/libbpf/libbpf/pull/29#issuecomment-481902601
[2] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html

Fixes: d859900c4c56 ("bpf, libbpf: support global data/bss/rodata sections")
Reported-by: Evgeny Vereshchagin <evvers@ya.ru>
Signed-off-by: Andrey Ignatov <rdna@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-11 09:01:12 -07:00
Yonghong Song
f49907472f sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline commit:   dd399ac9e343c7573c47d6820e4a23013c54749d
Checkpoint commit: 50bd645b3a21a374dbd0fa8273a5f4e98001fb05

Alexei Starovoitov (1):
  libbpf: teach libbpf about log_level bit 2

Andrey Ignatov (1):
  libbpf: Ignore -Wformat-nonliteral warning

Daniel Borkmann (3):
  bpf: sync {btf, bpf}.h uapi header from tools infrastructure
  bpf, libbpf: support global data/bss/rodata sections
  bpf, libbpf: add support for BTF Var and DataSec

Joe Stringer (1):
  bpf, libbpf: refactor relocation handling

Magnus Karlsson (1):
  libbpf: fix crash in XDP socket part with new larger BPF_LOG_BUF_SIZE

Yonghong Song (1):
  bpf, bpftool: fix a few ubsan warnings

 include/uapi/linux/bpf.h |  20 +-
 include/uapi/linux/btf.h |  32 ++-
 src/bpf.c                |  31 ++-
 src/bpf.h                |   3 +-
 src/btf.c                |  97 ++++++-
 src/btf.h                |   3 +
 src/libbpf.c             | 567 ++++++++++++++++++++++++++++++++-------
 src/libbpf.h             |   6 +
 src/libbpf.map           |   7 +
 src/xsk.c                |   9 +-
 10 files changed, 658 insertions(+), 117 deletions(-)

--
2.17.1
2019-04-10 14:27:33 -07:00
Magnus Karlsson
a74350fc04 libbpf: fix crash in XDP socket part with new larger BPF_LOG_BUF_SIZE
In commit da11b417583e ("libbpf: teach libbpf about log_level bit 2"),
the BPF_LOG_BUF_SIZE was increased to 16M. The XDP socket part of
libbpf allocated the log_buf on the stack, but for the new 16M buffer
size this is not going to work. Change the code so it uses a 16K buffer
instead.

Fixes: da11b417583e ("libbpf: teach libbpf about log_level bit 2")
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-10 14:27:33 -07:00
Yonghong Song
1770ac49cf bpf, bpftool: fix a few ubsan warnings
The issue is reported at https://github.com/libbpf/libbpf/issues/28.

Basically, per C standard, for
  void *memcpy(void *dest, const void *src, size_t n)
if "dest" or "src" is NULL, regardless of whether "n" is 0 or not,
the result of memcpy is undefined. clang ubsan reported three such
instances in bpf.c with the following pattern:
  memcpy(dest, 0, 0).

Although in practice, no known compiler will cause issues when
copy size is 0. Let us still fix the issue to silence ubsan
warnings.

Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-10 14:27:33 -07:00
Daniel Borkmann
0c4ac0d576 bpf, libbpf: add support for BTF Var and DataSec
This adds libbpf support for BTF Var and DataSec kinds. Main point
here is that libbpf needs to do some preparatory work before the
whole BTF object can be loaded into the kernel, that is, fixing up
of DataSec size taken from the ELF section size and non-static
variable offset which needs to be taken from the ELF's string section.

Upstream LLVM doesn't fix these up since at time of BTF emission
it is too early in the compilation process thus this information
isn't available yet, hence loader needs to take care of it.

Note, deduplication handling has not been in the scope of this work
and needs to be addressed in a future commit.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://reviews.llvm.org/D59441
Acked-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-10 14:27:33 -07:00
Daniel Borkmann
d99a28343f bpf, libbpf: support global data/bss/rodata sections
This work adds BPF loader support for global data sections
to libbpf. This allows to write BPF programs in more natural
C-like way by being able to define global variables and const
data.

Back at LPC 2018 [0] we presented a first prototype which
implemented support for global data sections by extending BPF
syscall where union bpf_attr would get additional memory/size
pair for each section passed during prog load in order to later
add this base address into the ldimm64 instruction along with
the user provided offset when accessing a variable. Consensus
from LPC was that for proper upstream support, it would be
more desirable to use maps instead of bpf_attr extension as
this would allow for introspection of these sections as well
as potential live updates of their content. This work follows
this path by taking the following steps from loader side:

 1) In bpf_object__elf_collect() step we pick up ".data",
    ".rodata", and ".bss" section information.

 2) If present, in bpf_object__init_internal_map() we add
    maps to the obj's map array that corresponds to each
    of the present sections. Given section size and access
    properties can differ, a single entry array map is
    created with value size that is corresponding to the
    ELF section size of .data, .bss or .rodata. These
    internal maps are integrated into the normal map
    handling of libbpf such that when user traverses all
    obj maps, they can be differentiated from user-created
    ones via bpf_map__is_internal(). In later steps when
    we actually create these maps in the kernel via
    bpf_object__create_maps(), then for .data and .rodata
    sections their content is copied into the map through
    bpf_map_update_elem(). For .bss this is not necessary
    since array map is already zero-initialized by default.
    Additionally, for .rodata the map is frozen as read-only
    after setup, such that neither from program nor syscall
    side writes would be possible.

 3) In bpf_program__collect_reloc() step, we record the
    corresponding map, insn index, and relocation type for
    the global data.

 4) And last but not least in the actual relocation step in
    bpf_program__relocate(), we mark the ldimm64 instruction
    with src_reg = BPF_PSEUDO_MAP_VALUE where in the first
    imm field the map's file descriptor is stored as similarly
    done as in BPF_PSEUDO_MAP_FD, and in the second imm field
    (as ldimm64 is 2-insn wide) we store the access offset
    into the section. Given these maps have only single element
    ldimm64's off remains zero in both parts.

 5) On kernel side, this special marked BPF_PSEUDO_MAP_VALUE
    load will then store the actual target address in order
    to have a 'map-lookup'-free access. That is, the actual
    map value base address + offset. The destination register
    in the verifier will then be marked as PTR_TO_MAP_VALUE,
    containing the fixed offset as reg->off and backing BPF
    map as reg->map_ptr. Meaning, it's treated as any other
    normal map value from verification side, only with
    efficient, direct value access instead of actual call to
    map lookup helper as in the typical case.

Currently, only support for static global variables has been
added, and libbpf rejects non-static global variables from
loading. This can be lifted until we have proper semantics
for how BPF will treat multi-object BPF loads. From BTF side,
libbpf will set the value type id of the types corresponding
to the ".bss", ".data" and ".rodata" names which LLVM will
emit without the object name prefix. The key type will be
left as zero, thus making use of the key-less BTF option in
array maps.

Simple example dump of program using globals vars in each
section:

  # bpftool prog
  [...]
  6784: sched_cls  name load_static_dat  tag a7e1291567277844  gpl
        loaded_at 2019-03-11T15:39:34+0000  uid 0
        xlated 1776B  jited 993B  memlock 4096B  map_ids 2238,2237,2235,2236,2239,2240

  # bpftool map show id 2237
  2237: array  name test_glo.bss  flags 0x0
        key 4B  value 64B  max_entries 1  memlock 4096B
  # bpftool map show id 2235
  2235: array  name test_glo.data  flags 0x0
        key 4B  value 64B  max_entries 1  memlock 4096B
  # bpftool map show id 2236
  2236: array  name test_glo.rodata  flags 0x80
        key 4B  value 96B  max_entries 1  memlock 4096B

  # bpftool prog dump xlated id 6784
  int load_static_data(struct __sk_buff * skb):
  ; int load_static_data(struct __sk_buff *skb)
     0: (b7) r6 = 0
  ; test_reloc(number, 0, &num0);
     1: (63) *(u32 *)(r10 -4) = r6
     2: (bf) r2 = r10
  ; int load_static_data(struct __sk_buff *skb)
     3: (07) r2 += -4
  ; test_reloc(number, 0, &num0);
     4: (18) r1 = map[id:2238]
     6: (18) r3 = map[id:2237][0]+0    <-- direct addr in .bss area
     8: (b7) r4 = 0
     9: (85) call array_map_update_elem#100464
    10: (b7) r1 = 1
  ; test_reloc(number, 1, &num1);
  [...]
  ; test_reloc(string, 2, str2);
   120: (18) r8 = map[id:2237][0]+16   <-- same here at offset +16
   122: (18) r1 = map[id:2239]
   124: (18) r3 = map[id:2237][0]+16
   126: (b7) r4 = 0
   127: (85) call array_map_update_elem#100464
   128: (b7) r1 = 120
  ; str1[5] = 'x';
   129: (73) *(u8 *)(r9 +5) = r1
  ; test_reloc(string, 3, str1);
   130: (b7) r1 = 3
   131: (63) *(u32 *)(r10 -4) = r1
   132: (b7) r9 = 3
   133: (bf) r2 = r10
  ; int load_static_data(struct __sk_buff *skb)
   134: (07) r2 += -4
  ; test_reloc(string, 3, str1);
   135: (18) r1 = map[id:2239]
   137: (18) r3 = map[id:2235][0]+16   <-- direct addr in .data area
   139: (b7) r4 = 0
   140: (85) call array_map_update_elem#100464
   141: (b7) r1 = 111
  ; __builtin_memcpy(&str2[2], "hello", sizeof("hello"));
   142: (73) *(u8 *)(r8 +6) = r1       <-- further access based on .bss data
   143: (b7) r1 = 108
   144: (73) *(u8 *)(r8 +5) = r1
  [...]

For Cilium use-case in particular, this enables migrating configuration
constants from Cilium daemon's generated header defines into global
data sections such that expensive runtime recompilations with LLVM can
be avoided altogether. Instead, the ELF file becomes effectively a
"template", meaning, it is compiled only once (!) and the Cilium daemon
will then rewrite relevant configuration data from the ELF's .data or
.rodata sections directly instead of recompiling the program. The
updated ELF is then loaded into the kernel and atomically replaces
the existing program in the networking datapath. More info in [0].

Based upon recent fix in LLVM, commit c0db6b6bd444 ("[BPF] Don't fail
for static variables").

  [0] LPC 2018, BPF track, "ELF relocation for static data in BPF",
      http://vger.kernel.org/lpc-bpf2018.html#session-3

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-10 14:27:33 -07:00
Joe Stringer
a3d236332e bpf, libbpf: refactor relocation handling
Adjust the code for relocations slightly with no functional changes,
so that upcoming patches that will introduce support for relocations
into the .data, .rodata and .bss sections can be added independent
of these changes.

Signed-off-by: Joe Stringer <joe@wand.net.nz>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-10 14:27:33 -07:00
Daniel Borkmann
bbd32bd59d bpf: sync {btf, bpf}.h uapi header from tools infrastructure
Pull in latest changes from both headers, so we can make use of
them in libbpf.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-10 14:27:33 -07:00
Andrey Ignatov
a6ce1c6423 libbpf: Ignore -Wformat-nonliteral warning
vsprintf() in __base_pr() uses nonliteral format string and it breaks
compilation for those who provide corresponding extra CFLAGS, e.g.:
https://github.com/libbpf/libbpf/issues/27

If libbpf is built with the flags from PR:

  libbpf.c:68:26: error: format string is not a string literal
  [-Werror,-Wformat-nonliteral]
          return vfprintf(stderr, format, args);
                                  ^~~~~~
  1 error generated.

Ignore this warning since the use case in libbpf.c is legit.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-04-10 14:27:33 -07:00
Alexei Starovoitov
6d7acdae6d libbpf: teach libbpf about log_level bit 2
Allow bpf_prog_load_xattr() to specify log_level for program loading.

Teach libbpf to accept log_level with bit 2 set.

Increase default BPF_LOG_BUF_SIZE from 256k to 16M.
There is no downside to increase it to a maximum allowed by old kernels.
Existing 256k limit caused ENOSPC errors and users were not able to see
verifier error which is printed at the end of the verifier log.

If ENOSPC is hit, double the verifier log and try again to capture
the verifier error.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-10 14:27:33 -07:00
4ast
7c27c63061 Merge pull request #25 from rdna/version-updates
DSO updates
2019-04-02 19:16:21 -07:00
Andrey Ignatov
23a177a0ce Add SONAME to DSO
Adopting common practice that was adopted earlier in kernel tree.

Resulting DSO:

  % readelf -d root/usr/lib64/libbpf.so | grep SONAME
   0x000000000000000e (SONAME)             Library soname: [libbpf.so.0]

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2019-04-02 18:43:14 -07:00
Andrey Ignatov
db1adc8658 Add symlinks for DSO
Add symlinks with version suffix in a way similar to how it's done in
kernel tree.

The differences with kernel tree version:

* LIBS is used to handle both files and targets w/o separation;

* Version symlink is created in corresponding target as opposed to do
  everything in libbpf.so.$(LIBBPF_VERSION).

Example:

  % OBJDIR=build DESTDIR=root make install
  ...
  % find root/ ! -type d -exec ls -l {} +
  -rw-r--r--. 1 rdna users   6046 Apr  2 18:09 root/usr/include/bpf/bpf.h
  -rw-r--r--. 1 rdna users   3485 Apr  2 18:09 root/usr/include/bpf/btf.h
  -rw-r--r--. 1 rdna users  16454 Apr  2 18:09 root/usr/include/bpf/libbpf.h
  -rw-r--r--. 1 rdna users 602030 Apr  2 18:09 root/usr/lib64/libbpf.a
  lrwxrwxrwx. 1 rdna users     11 Apr  2 18:09 root/usr/lib64/libbpf.so -> libbpf.so.0
  lrwxrwxrwx. 1 rdna users     15 Apr  2 18:09 root/usr/lib64/libbpf.so.0 -> libbpf.so.0.0.2
  -rwxr-xr-x. 1 rdna users 377640 Apr  2 18:09 root/usr/lib64/libbpf.so.0.0.2
  -rw-r--r--. 1 rdna users    241 Apr  2 18:09 root/usr/lib64/pkgconfig/libbpf.pc

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2019-04-02 18:37:19 -07:00
Andrey Ignatov
c3b329c387 Add LIBBPF_VERSION
LIBBPF_VERSION is already used to generate libbpf.pc file but was not
actuall set.

Set it same way it's done in kernel tree. Version is in sync with
version script for DSO.

Before:
  % grep Version src/libbpf.pc
  Version:

After:
  % grep Version src/libbpf.pc
  Version: 0.0.2

Fixes: 93bc1d0 ("makefile: sync generate pkg-config file for libbpf")
Signed-off-by: Andrey Ignatov <rdna@fb.com>
2019-04-02 16:41:13 -07:00
Andrey Ignatov
8c091e4ffd Enable version script for DSO building
Version script was synced from kernel tree but not enabled in Makefile.
Enable it same way it's done in kernel tree.

  % readelf -s --wide src/libbpf.so | grep -Eo '[^ ]+@LIBBPF_.*' | \
      cut -d@ -f 3 | sort | uniq -c
    121 LIBBPF_0.0.1
     29 LIBBPF_0.0.2

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2019-04-02 16:36:46 -07:00
Andrey Ignatov
2dd2a2d701 Fix libbpf.pc target
OBJDIR was not specified for libbpf.pc. It works by default since
default value of OBJDIR is current directory, but breaks as soon as
OBJDIR is set to some other directory:

  % OBJDIR=build DESTDIR=root make install
  ...
  ar rcs build/libbpf.a build/bpf.o build/btf.o build/libbpf.o
  build/libbpf_errno.o build/netlink.o build/nlattr.o build/str_error.o
  build/libbpf_probes.o build/bpf_prog_linfo.o build/xsk.o
  cc -shared  -lelf   build/bpf.o build/btf.o build/libbpf.o build/libbpf_errno.o
  build/netlink.o build/nlattr.o build/str_error.o build/libbpf_probes.o
  build/bpf_prog_linfo.o build/xsk.o -o build/libbpf.so
  make: *** No rule to make target `libbpf.pc', needed by `all'.  Stop.

Fix it.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2019-03-29 14:52:01 -07:00
4ast
46936194ed Merge pull request #23 from bluca/pkgconfig-elf
build shared library by default and use pkg-config to get libelf flags
2019-03-29 11:05:02 -07:00
Luca Boccassi
438584bfc2 Makefile: use pkg-config to get libelf flags
libbpf uses libelf symbols internally, so it depends on it and needs to
link against it.

Upstream this was fixed by linking directory to libelf:

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/commit/?id=89dedaef49d36adc2bb5e7e4c38b52fa3013c7c8

For this project, intended to be used as a general library, try first
to use pkg-config instead, and only fallback to just passing -lelf.
This will be useful for cross compiling, among other cases.

Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
2019-03-29 17:37:54 +00:00
Luca Boccassi
3e403451c7 Build shared lib by default, add options to turn it off
The vast majority of use cases want a shared library, so to be more
user and packager friendly invert the makefile logic and always build
both static and shared libraries by default.
Add BUILD_STATIC_ONLY variable for the corner cases where only a static
library is needed

Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
2019-03-29 16:44:25 +00:00
Luca Boccassi
e31e1a5a3f Makefile: pass CPPFLAGS to the compiler
Distro tools like dpkg-buildpackage use the preprocessor flags to pass
important flags, like hardening features. Pass CPPFLAGS to CC.

Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
2019-03-29 09:11:11 -07:00
Luca Boccassi
93bc1d03f4 makefile: sync generate pkg-config file for libbpf
Generate a libbpf.pc file at build time so that users can rely
on pkg-config to find the library, its CFLAGS and LDFLAGS.

This has been applied on bpf-next, so the corresponding change is
backported here:

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=dd399ac9e343c7573c47d6820e4a23013c54749d

Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
2019-03-29 09:11:11 -07:00
Andrii Nakryiko
27a5f60a8b sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline commit:   f01a7dbe98ae4265023fa5d3af0f076f0b18a647
Checkpoint commit: dd399ac9e343c7573c47d6820e4a23013c54749d

Arnaldo Carvalho de Melo (1):
  tools lib bpf: Fix the build by adding a missing stdarg.h include

Changbin Du (1):
  perf tools: Fix errors under optimization level '-Og'

Daniel Borkmann (1):
  bpf, libbpf: clarify bump in libbpf version info

Lorenz Bauer (1):
  tools: update include/uapi/linux/bpf.h

Luca Boccassi (1):
  tools/bpf: generate pkg-config file for libbpf

Song Liu (1):
  tools lib bpf: Introduce bpf_program__get_prog_info_linear()

Willem de Bruijn (1):
  bpf: Sync bpf.h to tools

 include/uapi/linux/bpf.h |  65 +++++++++-
 src/README.rst           |   1 +
 src/libbpf.c             | 253 ++++++++++++++++++++++++++++++++++++++-
 src/libbpf.h             |  64 ++++++++++
 src/libbpf.map           |   3 +
 src/libbpf.pc.template   |  12 ++
 6 files changed, 393 insertions(+), 5 deletions(-)
 create mode 100644 src/libbpf.pc.template

--
2.17.1
2019-03-28 19:51:55 -07:00
Luca Boccassi
9d60bfdfa6 tools/bpf: generate pkg-config file for libbpf
Generate a libbpf.pc file at build time so that users can rely
on pkg-config to find the library, its CFLAGS and LDFLAGS.

Signed-off-by: Luca Boccassi <bluca@debian.org>
Acked-by: Andrey Ignatov <rdna@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-03-28 19:51:55 -07:00
Song Liu
413703c8cb tools lib bpf: Introduce bpf_program__get_prog_info_linear()
Currently, bpf_prog_info includes 9 arrays. The user has the option to
fetch any combination of these arrays. However, this requires a lot of
handling.

This work becomes more tricky when we need to store bpf_prog_info to a
file, because these arrays are allocated independently.

This patch introduces 'struct bpf_prog_info_linear', which stores arrays
of bpf_prog_info in continuous memory.

Helper functions are introduced to unify the work to get different sets
of bpf_prog_info.  Specifically, bpf_program__get_prog_info_linear()
allows the user to select which arrays to fetch, and handles details for
the user.

Please see the comments right before 'enum bpf_prog_info_array' for more
details and examples.

Signed-off-by: Song Liu <songliubraving@fb.com>
Reviewed-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lkml.kernel.org/r/ce92c091-e80d-a0c1-4aa0-987706c42b20@iogearbox.net
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: kernel-team@fb.com
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stanislav Fomichev <sdf@google.com>
Link: http://lkml.kernel.org/r/20190312053051.2690567-3-songliubraving@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-28 19:51:55 -07:00
Changbin Du
556285a37a perf tools: Fix errors under optimization level '-Og'
Optimization level '-Og' offers a reasonable level of optimization while
maintaining fast compilation and a good debugging experience. This patch
tries to make it work.

  $ make DEBUG=1 EXTRA_CFLAGS='-Og'
  bench/epoll-ctl.c: In function ‘do_threads’:
  bench/epoll-ctl.c:274:9: error: ‘ret’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
    return ret;
           ^~~
  ...

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Reviewed-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lkml.kernel.org/r/20190316080556.3075-4-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-28 19:51:55 -07:00
Arnaldo Carvalho de Melo
7183d5132f tools lib bpf: Fix the build by adding a missing stdarg.h include
The libbpf_print_fn_t typedef uses va_list without including the header
where that type is defined, stdarg.h, breaking in places where we're
unlucky for that type not to be already defined by some previously
included header.

Noticed while building on fedora 24 cross building tools/perf to the ARC
architecture using the uClibc C library:

  28 fedora:24-x-ARC-uClibc   : FAIL arc-linux-gcc (ARCompact ISA Linux uClibc toolchain 2017.09-rc2) 7.1.1 20170710

    CC       /tmp/build/perf/tests/llvm.o
  In file included from tests/llvm.c:3:0:
  /git/linux/tools/lib/bpf/libbpf.h:57:20: error: unknown type name 'va_list'
        const char *, va_list ap);
                      ^~~~~~~
  /git/linux/tools/lib/bpf/libbpf.h:59:34: error: unknown type name 'libbpf_print_fn_t'
   LIBBPF_API void libbpf_set_print(libbpf_print_fn_t fn);
                                    ^~~~~~~~~~~~~~~~~
  mv: cannot stat '/tmp/build/perf/tests/.llvm.o.tmp': No such file or directory

Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jakub Kicinski <jakub.kicinski@netronome.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Quentin Monnet <quentin.monnet@netronome.com>
Cc: Stanislav Fomichev <sdf@google.com>
Cc: Yonghong Song <yhs@fb.com>
Fixes: a8a1f7d09cfc ("libbpf: fix libbpf_print")
Link: https://lkml.kernel.org/n/tip-5270n2quu2gqz22o7itfdx00@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-28 19:51:55 -07:00
Willem de Bruijn
2b9a24d7ce bpf: Sync bpf.h to tools
Sync include/uapi/linux/bpf.h with tools/

Changes
  v1->v2:
  - BPF_F_ADJ_ROOM_MASK moved, no longer in this commit
  v2->v3:
  - BPF_F_ADJ_ROOM_ENCAP_L3_MASK moved, no longer in this commit

Signed-off-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-03-28 19:51:55 -07:00
Lorenz Bauer
0c639aae80 tools: update include/uapi/linux/bpf.h
Pull definitions for bpf_skc_lookup_tcp and bpf_sk_check_syncookie.

Signed-off-by: Lorenz Bauer <lmb@cloudflare.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-03-28 19:51:55 -07:00
Andrii Nakryiko
5877849633 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline commit:   f01a7dbe98ae4265023fa5d3af0f076f0b18a647
Checkpoint commit: 8543e437807970166c2b66b79935c9f4b0e6d1f9

Andrii Nakryiko (1):
  libbpf: fix btf_dedup equivalence check handling of different kinds

Daniel Borkmann (1):
  bpf, libbpf: clarify bump in libbpf version info

 src/README.rst | 1 +
 src/btf.c      | 3 +++
 2 files changed, 4 insertions(+)

--
2.17.1
2019-03-28 19:51:55 -07:00
Andrii Nakryiko
d5b20f9668 libbpf: fix btf_dedup equivalence check handling of different kinds
btf_dedup_is_equiv() used to compare btf_type->info fields, before doing
kind-specific equivalence check. This comparsion implicitly verified
that candidate and canonical types are of the same kind. With enum fwd
resolution logic this check couldn't be done generically anymore, as for
enums info contains vlen, which differs between enum fwd and
fully-defined enum, so this check was subsumed by kind-specific
equivalence checks.

This change caused btf_dedup_is_equiv() to let through VOID vs other
types check to reach switch, which was never meant to be handing VOID
kind, as VOID kind is always pre-resolved to itself and is only
equivalent to itself, which is checked early in btf_dedup_is_equiv().

This change adds back BTF kind equality check in place of more generic
btf_type->info check, still defering further kind-specific checks to
a per-kind switch.

Fixes: 9768095ba97c ("btf: resolve enum fwds in btf_dedup")
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-03-28 19:51:55 -07:00
Daniel Borkmann
d3a6d054df bpf, libbpf: clarify bump in libbpf version info
The current documentation suggests that we would need to bump the
libbpf version on every change. Lets clarify this a bit more and
reflect what we do today in practice, that is, bumping it once per
development cycle.

Fixes: 76d1b894c515 ("libbpf: Document API and ABI conventions")
Reported-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-03-28 19:51:55 -07:00
Andrii Nakryiko
7c15babf40 scripts/sync_kernel.sh: skip empty merges, preventing cherry-picking
Merges are common. `git cherry-pick` can't cherry-pick merge commit and
requires specifying which of the parents should be cherry-picked, so
it's a pain. But merges are also commonly empty, so it's safe to skip them.

This change goes through all merges touching libbpf files, checks that
they don't have any conflict resolution diffs. If there are any, abort,
if not, proceed as usual, ignoring merges.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-03-28 19:51:11 -07:00
Andrii Nakryiko
33b0174985 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline commit:   ea5bade929bf04355aeed58a3c9f5ef2c3afe2ac
Checkpoint commit: f01a7dbe98ae4265023fa5d3af0f076f0b18a647

Andrii Nakryiko (2):
  libbpf: handle BTF parsing and loading properly
  btf: resolve enum fwds in btf_dedup

Magnus Karlsson (1):
  libbpf: fix to reject unknown flags in xsk_socket__create()

Martin KaFai Lau (1):
  bpf: Sync bpf.h to tools/

Nikita V. Shirokov (1):
  bpf, libbpf: fixing leak when kernel does not support btf

Quentin Monnet (1):
  tools: bpf: synchronise BPF UAPI header with tools

 include/uapi/linux/bpf.h | 188 ++++++++++++++++++++++++++-------------
 src/btf.c                |  51 ++++++++---
 src/libbpf.c             |  11 ++-
 src/xsk.c                |  15 +++-
 4 files changed, 186 insertions(+), 79 deletions(-)

--
2.17.1
2019-03-19 12:17:19 -07:00