diff --git a/README.md b/README.md index 9861b16..4642294 100644 --- a/README.md +++ b/README.md @@ -173,7 +173,7 @@ bpf-next to Github sync ======================= All the gory details of syncing can be found in `scripts/sync-kernel.sh` -script. +script. See [SYNC.md](SYNC.md) for instruction. Some header files in this repo (`include/linux/*.h`) are reduced versions of their counterpart files at diff --git a/SYNC.md b/SYNC.md new file mode 100644 index 0000000..3cd6151 --- /dev/null +++ b/SYNC.md @@ -0,0 +1,267 @@ + + + + + +Libbpf sync +=========== + +Libbpf *authoritative source code* is developed as part of [bpf-next Linux source +tree](https://kernel.googlesource.com/pub/scm/linux/kernel/git/bpf/bpf-next) under +`tools/lib/bpf` subdirectory and is periodically synced to Github. + +Most of the mundane mechanical things like bpf and bpf-next tree merge, Git +history transformation, cherry-picking relevant commits, re-generating +auto-generated headers, etc. are taken care by +[sync-kernel.sh script](https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh). +But occasionally human needs to do few extra things to make everything work +nicely. + +This document goes over the process of syncing libbpf sources from Linux repo +to this Github repository. Feel free to contribute fixes and additions if you +run into new problems not outlined here. + +Setup expectations +------------------ + +Sync script has particular expectation of upstream Linux repo setup. It +expects that current HEAD of that repo points to bpf-next's master branch and +that there is a separate local branch pointing to bpf tree's master branch. +This is important, as the script will automatically merge their histories for +the purpose of libbpf sync. + +Below, we assume that Linux repo is located at `~/linux`, it's current head is +at latest `bpf-next/master`, and libbpf's Github repo is located at +`~/libbpf`, checked out to latest commit on `master` branch. It doesn't matter +from where to run `sync-kernel.sh` script, but we'll be running it from inside +`~/libbpf`. + +``` +$ cd ~/linux && git remote -v | grep -E '^(bpf|bpf-next)' +bpf https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git (fetch) +bpf ssh://git@gitolite.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git +(push) +bpf-next +https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git (fetch) +bpf-next +ssh://git@gitolite.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git (push) +$ git branch -vv | grep -E '^? (master|bpf-master)' +* bpf-master 2d311f480b52 [bpf/master] riscv, bpf: Fix patch_text implicit declaration + master c8ee37bde402 [bpf-next/master] libbpf: Fix bpf_xdp_query() in old kernels +$ git checkout bpf-master && git pull && git checkout master && git pull +... +$ git log --oneline -n1 +c8ee37bde402 (HEAD -> master, bpf-next/master) libbpf: Fix bpf_xdp_query() in old kernels +$ cd ~/libbpf && git checkout master && git pull +Your branch is up to date with 'libbpf/master'. +Already up to date. +``` + +Running setup script +-------------------- + +First step is to always run `sync-kernel.sh` script. It expects three arguments: + +``` +$ scripts/sync-kernel.sh +``` + +Note, that we'll store script's entire output in `/tmp/libbpf-sync.txt` and +put it into PR summary later on. **Please store scripts output and include it +in PR summary for others to check for anything unexpected and suspicious.** + +``` +$ scripts/sync-kernel.sh ~/libbpf ~/linux bpf-master | tee /tmp/libbpf-sync.txt +Dumping existing libbpf commit signatures... +WORKDIR: /home/andriin/libbpf +LINUX REPO: /home/andriin/linux +LIBBPF REPO: /home/andriin/libbpf +... +``` + +Most of the time this will go very uneventful. One expected case when sync +script might require user intervention is if `bpf` tree has some libbpf fixes, +which is nowadays not a very frequent occurence. But if that happens, script +will show you a diff between expected state as of latest bpf-next and synced +Github repo state. And will ask if these changes look good. Please use your +best judgement to verify that differences are indeed from expected `bpf` tree +fixes. E.g., it might look like below: + +``` +Comparing list of files... +Comparing file contents... +--- /home/andriin/linux/include/uapi/linux/netdev.h 2023-02-27 16:54:42.270583372 -0800 ++++ /home/andriin/libbpf/include/uapi/linux/netdev.h 2023-02-27 16:54:34.615530796 -0800 +@@ -19,7 +19,7 @@ + * @NETDEV_XDP_ACT_XSK_ZEROCOPY: This feature informs if netdev supports AF_XDP + * in zero copy mode. + * @NETDEV_XDP_ACT_HW_OFFLOAD: This feature informs if netdev supports XDP hw +- * oflloading. ++ * offloading. + * @NETDEV_XDP_ACT_RX_SG: This feature informs if netdev implements non-linear + * XDP buffer support in the driver napi callback. + * @NETDEV_XDP_ACT_NDO_XMIT_SG: This feature informs if netdev implements +/home/andriin/linux/include/uapi/linux/netdev.h and /home/andriin/libbpf/include/uapi/linux/netdev.h are different! +Unfortunately, there are some inconsistencies, please double check. +Does everything look good? [y/N]: +``` + +If it looks sensible and expected, type `y` and script will proceed. + +If sync is successful, your `~/linux` repo will be left in original state on +the original HEAD commit. `~/libbpf` repo will now be on a new branch, named +`libbpf-sync-` (e.g., `libbpf-sync-2023-02-28T00-53-40.072Z`). + +Push this branch into your fork of `libbpf/libbpf` Github repo and create a PR: + +``` +$ git push --set-upstream origin libbpf-sync-2023-02-28T00-53-40.072Z +Enumerating objects: 130, done. +Counting objects: 100% (115/115), done. +Delta compression using up to 80 threads +Compressing objects: 100% (28/28), done. +Writing objects: 100% (32/32), 5.57 KiB | 1.86 MiB/s, done. +Total 32 (delta 21), reused 0 (delta 0), pack-reused 0 +remote: Resolving deltas: 100% (21/21), completed with 9 local objects. +remote: +remote: Create a pull request for 'libbpf-sync-2023-02-28T00-53-40.072Z' on GitHub by visiting: +remote: https://github.com/anakryiko/libbpf/pull/new/libbpf-sync-2023-02-28T00-53-40.072Z +remote: +To github.com:anakryiko/libbpf.git + * [new branch] libbpf-sync-2023-02-28T00-53-40.072Z -> libbpf-sync-2023-02-28T00-53-40.072Z +Branch 'libbpf-sync-2023-02-28T00-53-40.072Z' set up to track remote branch 'libbpf-sync-2023-02-28T00-53-40.072Z' from 'origin'. +``` + +**Please, adjust PR name to have a properly looking timestamp. Libbpf +maintainers will be very thankful for that!** + +By default Github will turn above branch name into PR with subject "Libbpf sync +2023 02 28 t00 53 40.072 z". Please fix this into a proper timestamp, e.g.: +"Libbpf sync 2023-02-28T00:53:40.072Z". Thank you! + +**Please don't forget to paste contents of /tmp/libbpf-sync.txt into PR +summary!** + +Once PR is created, libbpf CI will run a bunch of tests to check that +everything is good. In simple cases that would be all you'd need to do. In more +complicated cases some extra adjustments might be necessary. + +**Please, keep naming and style consistent.** Prefix CI-related fixes with `ci: ` +prefix. If you had to modify sync script, prefix it with `sync: `. Also make +sure that each such commit has `Signed-off-by: Your Full Name `, +just like you'd do that for Linux upstream patch. Libbpf closely follows kernel +conventions and styling, so please help maintaining that. + +Updating allow/deny lists +------------------------- + +Libbpf CI intentionally runs a subset of latest BPF selftests on old kernel +(4.9 and 5.5, currently). It happens from time to time that some tests that +previously were successfully running on old kernels now don't, typically due to +reliance on some freshly added kernel feature. It might look something like this in [CI logs](https://github.com/libbpf/libbpf/actions/runs/4206303272/jobs/7299609578#step:4:2733): + +``` + All error logs: + serial_test_xdp_info:FAIL:get_xdp_none errno=2 + #283 xdp_info:FAIL + Summary: 49/166 PASSED, 5 SKIPPED, 1 FAILED +``` + +In such case we can either work with upstream to fix test to be compatible with +old kernels, or we'll have to add a test into a denylist (or remove it from +allowlist, like was [done](https://github.com/libbpf/libbpf/commit/ea284299025bf85b85b4923191de6463cd43ccd6) +for the case above). + +``` +$ find . -name '*LIST*' +./ci/vmtest/configs/ALLOWLIST-4.9.0 +./ci/vmtest/configs/DENYLIST-5.5.0 +./ci/vmtest/configs/DENYLIST-latest.s390x +./ci/vmtest/configs/DENYLIST-latest +./ci/vmtest/configs/ALLOWLIST-5.5.0 +``` + +Please determine which tests need to be added/removed from which list. And then +add that as a separate commit. **Please keep using the same branch name, so +that the same PR can be updated.** There is no need to open new PRs for each +such fix. + +Regenerating vmlinux.h header +----------------------------- + +To compile latest BPF selftests against old kernels, we check in pre-generated +[vmlinux.h](https://github.com/libbpf/libbpf/blob/master/.github/actions/build-selftests/vmlinux.h) +header file, located at `.github/actions/build-selftests/vmlinux.h`, which +contains type definitions from latest upstream kernel. When after libbpf sync +upstream BPF selftests require new kernel types, we'd need to regenerate +`vmlinux.h` and check it in as well. + +This will looks something like this in [CI logs](https://github.com/libbpf/libbpf/actions/runs/4198939244/jobs/7283214243#step:4:1903): + +``` + In file included from progs/test_spin_lock_fail.c:5: + /home/runner/work/libbpf/libbpf/.kernel/tools/testing/selftests/bpf/bpf_experimental.h:73:53: error: declaration of 'struct bpf_rb_root' will not be visible outside of this function [-Werror,-Wvisibility] + extern struct bpf_rb_node *bpf_rbtree_remove(struct bpf_rb_root *root, + ^ + /home/runner/work/libbpf/libbpf/.kernel/tools/testing/selftests/bpf/bpf_experimental.h:81:35: error: declaration of 'struct bpf_rb_root' will not be visible outside of this function [-Werror,-Wvisibility] + extern void bpf_rbtree_add(struct bpf_rb_root *root, struct bpf_rb_node *node, + ^ + /home/runner/work/libbpf/libbpf/.kernel/tools/testing/selftests/bpf/bpf_experimental.h:90:52: error: declaration of 'struct bpf_rb_root' will not be visible outside of this function [-Werror,-Wvisibility] + extern struct bpf_rb_node *bpf_rbtree_first(struct bpf_rb_root *root) __ksym; + ^ + 3 errors generated. + make: *** [Makefile:572: /home/runner/work/libbpf/libbpf/.kernel/tools/testing/selftests/bpf/test_spin_lock_fail.bpf.o] Error 1 + make: *** Waiting for unfinished jobs.... + Error: Process completed with exit code 2. +``` + +You'll need to build latest upstream kernel from `bpf-next` tree, using BPF +selftest configs. Concat arch-agnostic and arch-specific configs, build kernel, +then use bpftool to dump `vmlinux.h`: + +``` +$ cd ~/linux +$ cat tools/testing/selftests/bpf/config \ + tools/testing/selftests/bpf/config.x86_64 > .config +$ make -j$(nproc) olddefconfig all +... +$ bpftool btf dump file ~/linux/vmlinux format c > ~/libbpf/.github/actions/build-selftests/vmlinux.h +$ cd ~/libbpf && git add . && git commit -s +``` + +Check in generated `vmlinux.h`, don't forget to use `ci: ` commit prefix, add +it on top of sync commits. Push to Github and let libbpf CI do the checking for +you. See [this commit](https://github.com/libbpf/libbpf/commit/34212c94a64df8eeb1dd5d064630a65e1dfd4c20) +for reference. + +Troubleshooting +--------------- + +If something goes wrong and sync script exits early or is terminated early by +user, you might end up with `~/linux` repo on temporary sync-related branch. +Don't worry, though, sync script never destroys repo state, it follows +"copy-on-write" philosophy and creates new branches where necessary. So it's +very easy to restore previous state. So if anything goes wrong, it's easy to +start fresh: + +``` +$ git branch | grep -E 'libbpf-.*Z' + libbpf-baseline-2023-02-28T00-43-35.146Z + libbpf-bpf-baseline-2023-02-28T00-43-35.146Z + libbpf-bpf-tip-2023-02-28T00-43-35.146Z + libbpf-squash-base-2023-02-28T00-43-35.146Z +* libbpf-squash-tip-2023-02-28T00-43-35.146Z +$ git cherry-pick --abort +$ git checkout master && git branch | grep -E 'libbpf-.*Z' | xargs git br -D +Switched to branch 'master' +Your branch is up to date with 'bpf-next/master'. +Deleted branch libbpf-baseline-2023-02-28T00-43-35.146Z (was 951bce29c898). +Deleted branch libbpf-bpf-baseline-2023-02-28T00-43-35.146Z (was 3a70e0d4c9d7). +Deleted branch libbpf-bpf-tip-2023-02-28T00-43-35.146Z (was 2d311f480b52). +Deleted branch libbpf-squash-base-2023-02-28T00-43-35.146Z (was 957f109ef883). +Deleted branch libbpf-squash-tip-2023-02-28T00-43-35.146Z (was be66130d2339). +Deleted branch libbpf-tip-2023-02-28T00-43-35.146Z (was 2d311f480b52). +``` + +You might need to do the same for your `~/libbpf` repo sometimes, depending at +which stage sync script was terminated.