diff --git a/study/kernel/00-DESCRIPTION/ARCH.md b/study/kernel/00-DESCRIPTION/ARCH.md index b6aff73..fb5ca30 100644 --- a/study/kernel/00-DESCRIPTION/ARCH.md +++ b/study/kernel/00-DESCRIPTION/ARCH.md @@ -445,6 +445,9 @@ TLB entry shootdown 常常或多或少的带来一些性能问题. | 2019/04/18 | Mark Brown | [KVM: arm64: SVE cleanups](https://patchwork.kernel.org/project/linux-arm-kernel/cover/1555603631-8107-1-git-send-email-Dave.Martin@arm.com) | KVM guest SVE 指令的支持. | v2 ☑ 5.2-rc1 | [Patchwork v2,00/14](https://patchwork.kernel.org/project/linux-arm-kernel/cover/1555603631-8107-1-git-send-email-Dave.Martin@arm.com) | | 2019/04/18 | Mark Brown | [arm64: Expose SVE2 features for userspace](https://patchwork.kernel.org/project/linux-arm-kernel/patch/1555609298-10498-1-git-send-email-Dave.Martin@arm.com) | 支持向用户空间报告 SVE2 的存在及其可选功能. 同时为 KVM 虚拟化 guest 提供了 SVE2 的可见性. | v2 ☑ 5.2-rc1 | [Patchwork v2,00/14](https://patchwork.kernel.org/project/linux-arm-kernel/patch/1555609298-10498-1-git-send-email-Dave.Martin@arm.com) | +2022 年 6 月, Arm 工程师 Wilco Dijkstra 为 Glibc 提供了 SVE 优化的 memcpy 实现, 超过 32 字节的 memcpy 使用 SVE 实现, 这显着改善了随机 memcpy 测试的性能. 参见 [glibc-commit](https://sourceware.org/git/?p=glibc.git;a=commit;h=9f298bfe1f183804bb54b54ff9071afc0494906c), 以及 phoronix 报道--[Glibc Adds Arm SVE-Optimized Memory Copy - Can "Significantly" Help Performance](https://www.phoronix.com/scan.php?page=news_item&px=Glibc-Arm-SVE-Memcpy-Optimize). + + ### 2.3.3 SME ------- @@ -705,6 +708,9 @@ Arm True Random Number Generator Firmware Interface 1.0 于去年发布, 最终 ## 6.3 总线 ------- +### 6.3.1 Compute Express Link +------- + FireBox: Warehouse-Scale Computers [FireBox: A Hardware Building Block for 2020 Warehouse-Scale Computers](https://www.usenix.org/conference/fast14/technical-sessions/presentation/keynote) @@ -727,11 +733,29 @@ https://blogs.vmware.com/vsphere/2021/10/introducing-project-capitola.html [Dbus-Broker 30 Released For High Performance Linux Message Bus](https://www.phoronix.com/scan.php?page=news_item&px=Dbus-Broker-30), [bus1/dbus-broker](https://github.com/bus1/dbus-broker) + +### 6.3.2 CXL +------- + +[phoronix 上关于 CXL(Compute Express Link) 的所有相关报道](https://www.phoronix.com/scan.php?page=search&q=Compute%20Express%20Link) + + +英特尔工程师 Ben Widawsky 已经开始发布一个关于 Linux 上 CXL 的博客文章系列, 参见 [Compute Express Link Overview](https://bwidawsk.net/blog/2022/6/compute-express-link-intro) + +[公众号-半导体行业观察-越来越热的 CXL](https://mp.weixin.qq.com/s/sB2bmFcEaYsH1Jg19E0-eg) + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:---:|:----:|:---:|:----:|:---------:|:----:| +| 2021/02/16 | Ben Widawsky | [CXL 2.0 Support](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=68a5a9a243354ed52f7b37b057bd5e98cba870c8) | TODO | v5 ☐☑✓ | [LORE v3,00/16](https://lore.kernel.org/lkml/20210111225121.820014-1-ben.widawsky@intel.com)
*-*-*-*-*-*-*-*
[LORE v5,0/9](https://lore.kernel.org/all/20210217040958.1354670-1-ben.widawsky@intel.com) | + + ## 6.4 CPU IDLE(C-state) ------- [AMD Updates Linux Patches For Lowering Idle Exit Latency](https://www.phoronix.com/scan.php?page=news_item&px=AMD-Prefer-MWAIT-v3) +[Linux 5.20 With AMD Zen Will Prefer MWAIT Over HALT As An HPC Optimization](https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.20-AMD-MWAIT) + | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| | 2022/04/05 | Wyes Karny | [x86: Prefer MWAIT over HALT on AMD processors](https://lore.kernel.org/all/20220405130021.557880-1-wyes.karny@amd.com) | Monitor Wait "MWAIT" 指令可用于电源管理目的, 以提示处理器在等待事件或 MONITOR 存储操作完成时可以进入指定的目标 C 状态. MWAIT 的使用旨在比 HALT 指令更有效. Intel Core 2 及更高版本已经将 MWAIT 置于 HALT 指令之上, 以进入 C1 状态. 但是 AMD CPU 此代码路径中依旧使用 HALT. 这个补丁使 AMD 使用 MWAIT 指令而不是 HALT, 当系统未使用 CPU 空闲驱动程序时, 无论是 BIOS 禁用 C 状态还是驱动程序不属于内核构建. 反过来, 这可以将受影响系统的退出延迟提高约 21%. 参见 [](https://www.phoronix.com/scan.php?page=news_item&px=AMD-MWAIT-Over-HALT-2022). | v1 ☐☑✓ | [LORE](https://lore.kernel.org/all/20220405130021.557880-1-wyes.karny@amd.com)
*-*-*-*-*-*-*-*
[LORE v3,0/3](https://lore.kernel.org/lkml/cover.fba143c82098dffab6bbf0a2f3c4be8bae07ccf1.1652176835.git-series.wyes.karny@amd.com/) | @@ -761,6 +785,13 @@ https://blogs.vmware.com/vsphere/2021/10/introducing-project-capitola.html | 2022/04/29 | Chen Zhongjin | [objtool: add base support for arm64](https://patchwork.kernel.org/project/linux-arm-kernel/cover/20220429094355.122389-1-chenzhongjin@huawei.com/) | 636883 | v4 ☐☑ | [LORE v4,0/37](https://lore.kernel.org/r/20220429094355.122389-1-chenzhongjin@huawei.com) | +## 6.7 指令转译 +------- + +Rosetta 是一个转译过程, 允许用户在 Apple Silicon 上运行包含 x86_64 指令的应用程序。在 macOS 中, 这允许为基于英特尔的 Mac 电脑构建的应用程序在 Apple Silicon 上无缝运行; Rosetta 可以在 ARM Linux 虚拟机中为英特尔 Linux 应用程序提供同样的功能. + +[macOS 13 Adding Ability To Use Rosetta In ARM Linux VMs For Speedy x86_64 Linux Binaries](https://www.phoronix.com/scan.php?page=news_item&px=macOS-13-Rosetta-Linux-Binaries) +
* 本作品/博文 ( [AderStep-紫夜阑珊-青伶巷草 Copyright ©2013-2017](http://blog.csdn.net/gatieme) ), 由 [成坚(gatieme)](http://blog.csdn.net/gatieme) 创作. diff --git a/study/kernel/00-DESCRIPTION/BPF.md b/study/kernel/00-DESCRIPTION/BPF.md index cf75bc1..750545c 100644 --- a/study/kernel/00-DESCRIPTION/BPF.md +++ b/study/kernel/00-DESCRIPTION/BPF.md @@ -117,10 +117,14 @@ GCC 的支持 eBPF 经过了 3 个阶段. # 3 BPF Core ------- +## 3.1 eBPF support +------- + v3.15 对 BPF 进行了升级扩展, 参见 [BPF updates](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=9a985cdc5ccb0d557720221d01bd70c19f04bb8c). -eBPF 已经是一个独立的模块了, 因此后来 v3.18 直接将 eBPF 从 NET 子系统中分离出来. 参见 [bpf: split eBPF out of NET](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f89b7755f517cdbb755d7543eef986ee9d54e654). +eBPF 已经是一个独立的模块了, 因此后来 3.18 直接将 eBPF 从 NET 子系统中分离出来. 参见 [bpf: split eBPF out of NET](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f89b7755f517cdbb755d7543eef986ee9d54e654). +[BPF 数据传递的桥梁 ——BPF Map（一）](https://blog.csdn.net/alex_yangchuansheng/article/details/108332511) | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| @@ -152,6 +156,147 @@ v3.19 支持了 MAP, [BPF maps](https://git.kernel.org/pub/scm/linux/kernel/git/ | 2022/05/18 | Benjamin Tissoires | [Introduce eBPF support for HID devices](https://lore.kernel.org/all/20220518205924.399291-1-benjamin.tissoires@redhat.com) | TODO | v5 ☐☑✓ | [LORE v5,0/17](https://lore.kernel.org/all/20220518205924.399291-1-benjamin.tissoires@redhat.com) | +## 3.2 BPF_CMD +------- + +[commit 3007098494be ("cgroup: add support for eBPF programs")](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3007098494bec614fb55dee7bc0410bb7db5ad18) + + +[commit f4324551489e ("bpf: add BPF_PROG_ATTACH and BPF_PROG_DETACH commands")](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f4324551489e8781d838f941b7aee4208e52e8bf) + +## 3.3 BPF_PROG_TYPE AND BPF_ATTACH_TYPE +------- + +```cpp +git grep -W 'bpf_prog_type {' include/uapi/linux/bpf.h +``` + +[BPF 程序类型及其原理s](https://blog.csdn.net/weixin_41036447/article/details/107817340) + +### 3.3.1 SOCKET +------- + + +| enum bpf_prog_type | VERSION | COMMIT | +|:------------------:|:-------:|:------:| +| BPF_PROG_TYPE_SOCKET_FILTER | + + +| enum bpf_prog_type | enum bpf_attach_type | VERSION | COMMIT | +|:------------------:|:--------------------:|:-------:|:------:| +| BPF_PROG_TYPE_SOCKET_FILTER | NA | v3.19 | [allow eBPF programs to be attached to sockets](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=fbe3310840c65f3cf97dd90d23e177d061c376f2) | +| BPF_PROG_TYPE_CGROUP_SKB | BPF_CGROUP_INET_INGRESS
BPF_CGROUP_INET_EGRESS | v4.10 | [bpf: add new prog type for cgroup socket filtering](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0e33661de493db325435d565a4a722120ae4cbf3) | +| BPF_PROG_TYPE_CGROUP_SOCK | BPF_CGROUP_INET_SOCK_CREATE | v4.10 | [bpf: Add new cgroup attach type to enable sock modifications](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=61023658760032e97869b07d54be9681d2529e77) | +| BPF_PROG_TYPE_CGROUP_SOCK_ADDR | BPF_CGROUP_INET4_BIND
BPF_CGROUP_INET6_BIND | v4.17 | [bpf: Hooks for sys_bind](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4fbac77d2d092b475dda9eea66da674369665427) | +| BPF_PROG_TYPE_CGROUP_SOCK_ADDR | BPF_CGROUP_INET4_CONNECT
BPF_CGROUP_INET4_CONNECT | v4.17 | [bpf: Hooks for sys_connect](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d74bad4e74ee373787a9ae24197c17b7cdc428d5) | +| BPF_PROG_TYPE_CGROUP_SOCK | BPF_CGROUP_INET4_POST_BIND
BPF_CGROUP_INET6_POST_BIND | v4.17 | [bpf: Post-hooks for sys_bind](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=aac3fc320d9404f2665a8b1249dc3170d5fa3caf) | +| BPF_PROG_TYPE_CGROUP_SOCK_ADDR | BPF_CGROUP_UDP4_SENDMSG
BPF_CGROUP_UDP6_SENDMSG | v4.18 | [bpf: Hooks for sys_sendmsg](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1cedee13d25ab118d325f95588c1a084e9317229) | +| BPF_PROG_TYPE_CGROUP_SOCK_ADDR | BPF_CGROUP_UDP4_RECVMSG
BPF_CGROUP_UDP6_RECVMSG | v5.6 | [bpf: fix unconnected udp hooks](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=983695fa676568fc0fe5ddd995c7267aabc24632) | +| BPF_PROG_TYPE_CGROUP_SOCK_ADDR | BPF_CGROUP_INET4_GETPEERNAME
BPF_CGROUP_INET6_GETPEERNAME
BPF_CGROUP_INET4_GETSOCKNAME
BPF_CGROUP_INET6_GETSOCKNAME | v5.8 | [bpf: Add get{peer, sock}name attach types for sock_addr](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1b66d253610c7f8f257103808a9460223a087469) | +| BPF_PROG_TYPE_CGROUP_SOCK | BPF_CGROUP_INET_SOCK_RELEASE | v5.9 | [bpf: Add BPF_CGROUP_INET_SOCK_RELEASE hook](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f5836749c9c04a10decd2742845ad4870965fdef) | + + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:----:|:----:|:---:|:----:|:---------:|:----:| +| 2014/11/26 | Alexei Starovoitov | [allow eBPF programs to be attached to sockets](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=fbe3310840c65f3cf97dd90d23e177d061c376f2) | 引入可以通过 setsockopt() 附加到套接字的 BPF_PROG_TYPE_SOCKET_FILTER 类型的 eBPF 程序. 允许这些程序通过查找/更新/删除助手访问 MAPs. | v1 ☑✓ 3.19-rc1 | [LORE v1,0/6](https://lore.kernel.org/all/1417066951-1999-1-git-send-email-ast@plumgrid.com)
*-*-*-*-*-*-*-*
[LORE v2,0/6](https://lore.kernel.org/lkml/1417475199-15950-1-git-send-email-ast@plumgrid.com) | +| 2016/11/23 | Daniel Mack | [Add eBPF hooks for cgroups](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=d8c5b17f2bc0de09fbbfa14d90e8168163a579e7) | TODO | v9 ☑✓ 4.10-rc1 | [LORE v9,0/6](https://lore.kernel.org/all/1479916350-28462-1-git-send-email-daniel@zonque.org) | +| 2016/12/01 | David Ahern | [net: Add bpf support for sockets](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=554ae6e792ef38020b80b4d5127c51d510c0918f) | eBPF 程序支持 attach 到 | v7 ☑✓ 4.10-rc1 | [LORE v7,0/6](https://lore.kernel.org/all/1480610888-31082-1-git-send-email-dsa@cumulusnetworks.com) | +| 2018/03/13 | Alexei Starovoitov | [bpf: introduce cgroup-bpf bind, connect, post-bind hooks](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=aac3fc320d9404f2665a8b1249dc3170d5fa3caf) | eBPF 程序支持 attach 到 syscal bind 和 conntect. | v1 ☑✓ 4.17-rc1 | [LORE v1,0/6](https://lore.kernel.org/all/20180314033934.3502167-1-ast@kernel.org)
*-*-*-*-*-*-*-*
[LORE v2,0/9](https://lore.kernel.org/netdev/20180328034140.291484-1-ast@kernel.org)
*-*-*-*-*-*-*-*
[LORE v3](https://lore.kernel.org/netdev/20180330220808.763556-1-ast@kernel.org) | +| 2018/05/25 | Andrey Ignatov | [bpf: Hooks for sys_sendmsg](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=04b6ab731209eac1e130fa00281a29278eca2f57) | eBPF 程序支持附加到 sendmsg SYSCALL. | v4 ☑✓ 4.18-rc1 | [LORE v4,0/6](https://lore.kernel.org/all/cover.1527263217.git.rdna@fb.com) | +| 2019/06/07 | Daniel Borkmann | [Fix unconnected bpf cgroup hooks](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=b714560f7b38de9f03b8670890ba130d4cc5604e) | eBPF 程序支持附加到 recvmsg SYSCALL. | v3 ☑✓ 5.2-rc6 | [LORE v3,0/6](https://lore.kernel.org/all/20190606234902.4300-1-daniel@iogearbox.net) | +| 2020/05/19 | Daniel Borkmann | [Add get{peer,sock}name cgroup attach types](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=566fc3f5d1c641b510ec487cf274a047f8a1e849) | 为 BPF sock_addr 程序添加 get {peer,sock} name cgroup 类型, 以便从两个调用以及 libbpf 和 bpftool 支持以及自检中重写 sockaddr structs. | v2 ☑✓ 5.8-rc1 | [LORE v2,0/4](https://lore.kernel.org/all/cover.1589841594.git.daniel@iogearbox.net) | +| 2020/07/06 | Stanislav Fomichev | [bpf: add BPF_CGROUP_INET_SOCK_RELEASE hook](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=65ffd797861a44ff97081de1db01e4aef716ed46) | 添加 BPF_CGROUP_INET_SOCK_RELEASE. | v4 ☑✓ 5.9-rc1 | [LORE v4,0/4](https://lore.kernel.org/all/20200706230128.4073544-1-sdf@google.com) | + +### 3.3.2 CGROUP +------- + + +[commit 3007098494be ("cgroup: add support for eBPF programs")](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3007098494bec614fb55dee7bc0410bb7db5ad18) + + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:---:|:----:|:---:|:---:|:----------:|:----:| +| 2016/11/23 | Daniel Mack | [Add eBPF hooks for cgroups](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=d8c5b17f2bc0de09fbbfa14d90e8168163a579e7) | TODO | v9 ☑✓ 4.10-rc1 | [LORE v9,0/6](https://lore.kernel.org/all/1479916350-28462-1-git-send-email-daniel@zonque.org) | + + +### 3.3.3 BPF_PROG_TYPE_KPROBE +------- + +| enum bpf_prog_type | enum bpf_attach_type | VERSION | COMMIT | +|:------------------:|:--------------------:|:-------:|:------:| +| BPF_PROG_TYPE_KPROBE | NA | v4.1 | [commit 2541517c32be ("tracing, perf: Implement BPF programs attached to kprobes")](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2541517c32be2531e0da59dfd7efc1ce844644f5) | + + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:---:|:----:|:---:|:----:|:---------:|:----:| +| 2015/03/25 | Alexei Starovoitov | [tracing: attach eBPF programs to kprobes](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=9811e35359d4b18baf5bb603b225e957255b9c46) | NA | v11 ☑ 4.1-rc1 | [PatchWork v11 0/9](https://lore.kernel.org/lkml/1427312966-8434-1-git-send-email-ast@plumgrid.com) | + + +### 3.3.4 Traffic Control Subsystem +------- + +[eBPF: Traffic Control Subsystem](https://blog.csdn.net/weixin_43705457/article/details/123388130) + +[eBPF 的发展历史和核心设计](https://blog.csdn.net/alex_yangchuansheng/article/details/114558126) + +[linux-next: manual merge of the tip tree with the net-next tree](https://lore.kernel.org/lkml/20150407171148.7a41ee90@canb.auug.org.au) + +| enum bpf_prog_type | enum bpf_attach_type | VERSION | COMMIT | +|:------------------:|:--------------------:|:-------:|:------:| +| BPF_PROG_TYPE_SCHED_CLS | NA | v4.1 | [ebpf: add sched_cls_type and map it to sk_filter's verifier ops](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=96be4325f443dbbfeb37d2a157675ac0736531a1) | +| BPF_PROG_TYPE_SCHED_ACT | NA | v4.1 | [ebpf: add sched_act_type and map it to sk_filter's verifier ops](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=96be4325f443dbbfeb37d2a157675ac0736531a1) | + + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:---:|:----:|:---:|:----:|:---------:|:----:| +| 2015/02/27 | Daniel Borkmann | [eBPF support for cls_bpf](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=e2e9b6541dd4b31848079da80fe2253daaafb549) | TODO | v1 ☑✓ 4.1-rc1 | [LORE v1,0/10](https://lore.kernel.org/all/cover.1425040939.git.daniel@iogearbox.net) | + + +### 3.3.5 tracepoint +------- + +#### 3.3.5.1 tracepoint +------- + + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:----:|:----:|:---:|:----:|:---------:|:----:| +| 2016/04/07 | Alexei Starovoitov | [allow bpf attach to tracepoints](https://lore.kernel.org/patchwork/cover/665728) | BPF 支持 tracepoint 跟踪 | v2 ☑ 4.7-rc1 | [PatchWork v2 00/10](https://lore.kernel.org/patchwork/cover/665728) | + +#### 3.3.5.2 raw tracepoint +------- + +raw_tracepoint 相比 tracepoint + +1. 效率更高, 跳过了参数的处理, 不必向 tracepoint 那样根据 format 解析了所有字段, 节约了不必要的开销. + +2. 自由度更大, 提供了对参数的原始访问, 可以直接操作传入 tracepoint 时的指针, 获取某个成员. + +3. 可以获得原始结构体指针. 用于做 key 比较方便, tracepoint 只能拿到里面的各个字段, 有些时候没法做 key. + + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:----:|:----:|:---:|:----:|:---------:|:----:| +| 2018/02/28 | Alexei Starovoitov | [bpf, tracing: introduce bpf raw tracepoints](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=4662a4e53890badf4da17e441606a2885f29d56d) | BPF 支持 raw tracepoint 跟踪, 提供了对 tracepoint 参数的访问. | v1 ☐☑✓ | [LORE v1,0/5](https://lore.kernel.org/all/20180301041957.399230-1-ast@kernel.org)
*-*-*-*-*-*-*-*
[LORE v4,00/10](https://lore.kernel.org/all/20180323003907.3113756-1-ast@fb.com) | +| 2018/12/13 | Matt Mullins | [bpf: support raw tracepoints in modules](https://lore.kernel.org/patchwork/cover/1024475) | NA | v2 ☑ 5.0-rc1 | [PatchWork v2](https://lore.kernel.org/patchwork/cover/1024475) | +| 2019/04/26 | Matt Mullins | [writable contexts for bpf raw tracepoints](https://lore.kernel.org/patchwork/cover/1067299) | 引入 BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, 允许 bpfs 程序修改 tracepoint 中的变量. | v5 ☑ 4.17-rc1 | [PatchWork v5](https://lore.kernel.org/patchwork/cover/1067299) | + + +## 3.4 Helper Function +------- + +| enum bpf_func_id | VERSION | COMMIT | +|:----------------:|:-------:|:------:| +| NA | NA | NA | + + +| 2016/10/27 | Thomas Graf | [bpf: Print function name in addition to function id](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ebb676daa1a340ccef25eb769aefc09b79c01f8a) | TODO | v1 ☐☑✓ 4.10-rc1 | [LORE](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ebb676daa1a340ccef25eb769aefc09b79c01f8a) | +| 2018/04/25 | Quentin Monnet | [bpf: add documentation for eBPF helpers](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=9cde0c8892b9bba07b27f2317d8d02707dc6ff92) | 提供 `scripts/bpf_helpers_doc.py` 脚本根据 `include/uapi/linux/bpf.h` 生成文档 `bpf-helpers.rst`. | v1 ☑✓ 4.18-rc1 | [LORE](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9cde0c8892b9) | +| 2022/05/29 | Lorenzo Bianconi | [introduce support for XDP programs in CPUMAP](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=cover.1594734381.git.lorenzo@kernel.org) | TODO | v ☐☑✓ | [LORE v7,0/9](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=cover.1594734381.git.lorenzo@kernel.org) | +| 2022/05/29 | Tycho Andersen | [c/r of seccomp filters via underlying eBPF](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1441382664-17437-1-git-send-email-tycho.andersen@canonical.com) | TODO | v ☐☑✓ | [LORE](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1441382664-17437-1-git-send-email-tycho.andersen@canonical.com) | + + + ### 3.1.1 dump kernel data ------- @@ -198,9 +343,6 @@ v3.19 支持了 MAP, [BPF maps](https://git.kernel.org/pub/scm/linux/kernel/git/ - - - ## 3.5 JIT ------- @@ -219,46 +361,6 @@ v3.19 支持了 MAP, [BPF maps](https://git.kernel.org/pub/scm/linux/kernel/git/ [Toward signed BPF programs](https://lwn.net/Articles/853489) -# 4 维测场景 -------- - -## 4.1 kprobe -------- - -| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | -|:----:|:----:|:---:|:----:|:---------:|:----:| -| 2015/03/25 | Alexei Starovoitov | [tracing: attach eBPF programs to kprobes](https://lore.kernel.org/patchwork/cover/555274) | NA | v11 ☑ 4.1-rc1 | [PatchWork v11 0/9](https://lore.kernel.org/patchwork/cover/555274) | - -## 4.2 tracepoint -------- - -### 4.2.1 tracepoint -------- - - -| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | -|:----:|:----:|:---:|:----:|:---------:|:----:| -| 2016/04/07 | Alexei Starovoitov | [allow bpf attach to tracepoints](https://lore.kernel.org/patchwork/cover/665728) | BPF 支持 tracepoint 跟踪 | v2 ☑ 4.7-rc1 | [PatchWork v2 00/10](https://lore.kernel.org/patchwork/cover/665728) | - -### 4.2.2 raw tracepoint -------- - -raw_tracepoint 相比 tracepoint - -1. 效率更高, 跳过了参数的处理, 不必向 tracepoint 那样根据 format 解析了所有字段, 节约了不必要的开销. - -2. 自由度更大, 提供了对参数的原始访问, 可以直接操作传入 tracepoint 时的指针, 获取某个成员. - -3. 可以获得原始结构体指针. 用于做 key 比较方便, tracepoint 只能拿到里面的各个字段, 有些时候没法做 key. - - -| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | -|:----:|:----:|:---:|:----:|:---------:|:----:| -| 2018/03/01 | Jan H. Schönherr | [bpf, tracing: introduce bpf raw tracepoints](https://lore.kernel.org/patchwork/cover/879278) | BPF 支持 raw tracepoint 跟踪, 提供了对 tracepoint 参数的访问. | v1 ☑ 4.17-rc1 | [PatchWork v1](https://lore.kernel.org/patchwork/cover/879278) | -| 2018/12/13 | Matt Mullins | [bpf: support raw tracepoints in modules](https://lore.kernel.org/patchwork/cover/1024475) | NA | v2 ☑ 5.0-rc1 | [PatchWork v2](https://lore.kernel.org/patchwork/cover/1024475) | -| 2019/04/26 | Matt Mullins | [writable contexts for bpf raw tracepoints](https://lore.kernel.org/patchwork/cover/1067299) | 引入 BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, 允许 bpfs 程序修改 tracepoint 中的变量. | v5 ☑ 4.17-rc1 | [PatchWork v5](https://lore.kernel.org/patchwork/cover/1067299) | - - # 5 网络场景 ------- diff --git a/study/kernel/00-DESCRIPTION/DEBUGGING.md b/study/kernel/00-DESCRIPTION/DEBUGGING.md index c46d2d4..28b5d20 100644 --- a/study/kernel/00-DESCRIPTION/DEBUGGING.md +++ b/study/kernel/00-DESCRIPTION/DEBUGGING.md @@ -300,14 +300,22 @@ Facebook 在 2018 年开源了一套解决重要计算集群管理问题的 Linu | 2017/08/31 | Andi Kleen | [perf arm64 metricgroup support](https://lore.kernel.org/all/20170831194036.30146-1-andi@firstfloor.org) | 为 perf stat 添加[对 JSON 文件中指定的独立指标](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b18f3e365019de1a5b26a851e123f0aedcce881f) 的通用支持. 这些指标是一个公式, 它使用多个事件来计算更高级别的结果(例如 IPC, TOPDOWN 分析等). 添加一个新的 `-M /--metrics` 选项来添加指定的度量或度量组. 并增加了对 Intel X86 平台的支持. 通过这些 JSON 文件定义的事件组合度量, 可以很好的支持 TOPDOWN 分析. | v3 ☑ 4.15-rc1 | [PatchWork v3,00/11](https://lore.kernel.org/all/20170831194036.30146-1-andi@firstfloor.org) | | 2020/09/11 | Kan Liang | [TopDown metrics support for Ice Lake (perf tool)](https://lore.kernel.org/lkml/20200911144808.27603-1-kan.liang@linux.intel.com) | 为 perf metrics 分析增加对 Ice Lake 的支持. 将原本 group 重命名为 topdown. | v3 ☑ 5.10-rc1 | [PatchWork v3,0/4](https://lore.kernel.org/lkml/20200911144808.27603-1-kan.liang@linux.intel.com) | | 2021/04/07 | John Garry | [perf arm64 metricgroup support](https://patchwork.kernel.org/project/linux-arm-kernel/cover/1617791570-165223-1-git-send-email-john.garry@huawei.com) | perf 支持 HiSilicon hip08 平台的 topdown metric. 支持到 Level 3. 自此鲲鹏 920 的 ARM64 服务器上, 可以使用:
`sudo perf stat -M TopDownL1 sleeep 1`
来进行 TopDown 分析了. | v1 ☑ 5.13-rc1 | [PatchWork 0/5](https://patchwork.kernel.org/project/linux-arm-kernel/cover/1614784938-27080-1-git-send-email-john.garry@huawei.com)
*-*-*-*-*-*-*-*
[PatchWork v2,0/6](https://patchwork.kernel.org/project/linux-arm-kernel/cover/1616668398-144648-1-git-send-email-john.garry@huawei.com)
*-*-*-*-*-*-*-*
[PatchWork v3,0/6](https://patchwork.kernel.org/project/linux-arm-kernel/cover/1617791570-165223-1-git-send-email-john.garry@huawei.com) | +| 2022/05/28 | zhengjun | [perf vendor events intel: Add metrics for Sapphirerapids](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=1bcca2b1bd67f3c0e5c3a88ed16c6389f01a5b31) | TODO | v2 ☑✓ 5.19-rc1 | [LORE v2,0/2](https://lore.kernel.org/all/20220528095933.1784141-1-zhengjun.xing@linux.intel.com) | ## 11.3 Userspace counter access ------- +x86 和 arm64 都支持直接访问用户空间中的事件计数器. 访问序列并不简单，目前存在于 perf 测试代码(tools/perf/arch/x86/tests/rdpmc.c)中, 在 PAPI 和 libpfm4 等项目中有类似的用例程序. + +为了支持 usersapce 访问，必须首先使用 perf_evsel__mmap() 映射事件. 然后, 对 perf_evsel__read() 对 PMU 进行读取. + + | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| +| 2011/12/21 | Peter Zijlstra | [perf, x86: Implement user-space RDPMC support, to allow fast, user-space access to self-monitoring counters](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fe4a330885aee20f233de36085fb15c38094e635) | TODO | v1 ☐☑✓ | [LORE](https://lore.kernel.org/all/tip-mwxab34dibqgzk5zywutfnha@git.kernel.org) | | 2021/12/08 | Rob Herring | [arm64 userspace counter support](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=aa1005d15d2aee10e5b93a25db076c47e05c4efa) | ARM64 的 PMU 是支持用户态直接访问的. 与 x86 上 rdpmc 机制类似, 为了防止出现安全问题, 当前只允许用户态访问 thread bound events, 在启用时, 为了防止从其他任务泄漏禁用的 PMU 数据, 还会有一些额外的开销来清除脏计数器. | v13 ☑✓ 5.17-rc1 | [LORE v13,0/5](https://lore.kernel.org/all/20211208201124.310740-1-robh@kernel.org) | - +| 2021/04/14 | Rob Herring | [libperf userspace counter access](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=47d01e7b9999b9591077a59a1ecec11c6ce570de) | libperf 支持用户态读取 PMU, 当前只支持了 X86 | v8 ☑✓ 5.13-rc1 | [LORE v8,0/4](https://lore.kernel.org/all/20210414155412.3697605-1-robh@kernel.org) | +| 2022/02/01 | Rob Herring | [libperf: Add arm64 support to perf_mmap__read_self()](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=407eb43ae87c969d98746c3274ae5d0f977b102e) | libperf 支持 ARM64 用户态读取 PMU. | v9 ☑✓ 5.17-rc3 | [LORE](https://lore.kernel.org/all/20220201214056.702854-1-robh@kernel.org) | ## 11.4 分支预测 ------- diff --git a/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md b/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md index 9b95dc7..6575327 100644 --- a/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md +++ b/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md @@ -5117,21 +5117,35 @@ khugepaged 处理流程 在 fork 进程的时候, 并不会为子进程直接分配物理页面, 而是使用 COW 的机制. 在 fork 之后, 父子进程都共享原来父进程的页面, 同时将父子进程的 COW mapping 都设置为只读. 这样当这两个进程触发写操作的时候, 触发缺页中断, 在处理缺页的过程中再为该进程分配出一个新的页面出来. ```cpp -copy_process -copy_mm --=> dup_mm - -=> dup_mmap - -=> copy_page_range - -=> copy_p4d_range - -=> copy_pud_range - -=> copy_pmd_range - -=> copy_one_pte - -=> ptep_set_wrprotect +// https://www.cnblogs.com/pengdonglin137/p/16131785.html +sys_fork + -> kernel_clone + -> copy_process + -> copy_mm + -> dup_mm + -> dup_mmap + -> copy_page_range + -> copy_p4d_range + -> 如果时PUD巨型页：copy_huge_pud: 分别将父子的PUD页表项设置为写保护 + -> pudp_set_wrprotect(src_mm, addr, src_pud); + -> set_pud_at(dst_mm, addr, dst_pud, pud_mkold(pud_wrprotect(pud))); + -> copy_pmd_range + -> 如果是PMD巨型页：copy_huge_pmd: 分别将父子的PMD页表项设置为写保护 + -> pmdp_set_wrprotect(src_mm, addr, src_pmd); + -> set_pmd_at(dst_mm, addr, dst_pmd, pmd_mkold(pmd_wrprotect(pmd))); + -> copy_pte_range + -> copy_present_pte + -> 如果：is_cow_mapping(vm_flags) && pte_write(pte) + -> ptep_set_wrprotect(src_mm, addr, src_pte); + -> set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte_wrprotect(pte)); ``` -### 8.2.2.1 匿名页的写时拷贝 +执行 fork 完毕后, 原先父进程中可写的区域在父子进程中都被设置了 CoW, 即 pte 中可写的属性被清除. 那么下次对此地址进行写操作时(不管是父进程还是子进程), 都会触发 PageFault. 然后新分配一个页面出来公其使用, 同时原来 pte 的可写属性也会被重新设置. +### 8.2.2.1 匿名页的写时拷贝 +------- + ### 8.2.2.2 文件页的写时拷贝 ------- @@ -5956,6 +5970,8 @@ FRONTSWAP 对应的另一个后端叫 [ZSWAP](https://lwn.net/Articles/537422). [LWN: LSFMM-2022/CXL 1: Management and tiering](https://lwn.net/Articles/894598) +[](https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.18-NUMA-Regression-Fix) + ### 12.1.2 多级内存(Top-tier memory management)/内存分级(memory tiering) 支持 ------- diff --git a/study/kernel/00-DESCRIPTION/PERFORMANCE.md b/study/kernel/00-DESCRIPTION/PERFORMANCE.md index f8946c5..d9f55f1 100644 --- a/study/kernel/00-DESCRIPTION/PERFORMANCE.md +++ b/study/kernel/00-DESCRIPTION/PERFORMANCE.md @@ -68,7 +68,14 @@ blogexcerpt: 虚拟化 & KVM 子系统 | 2019/10/22 | Vincent Guittot | [sched/fair: Fix rework of find_idlest_group()](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3318544b721d3072fdd1f85ee0f1f214c0b211ee) | TODO | v1 ☐☑✓ | [LORE](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3318544b721d3072fdd1f85ee0f1f214c0b211ee) | -# 2 网络 +# 2 内存 +------- + +[commit ff042f4a9b05 ("mm: lru_cache_disable: replace work queue synchronization with synchronize_rcu")](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ff042f4a9b050895a42cae893cc01fa2ca81b95c) 引起了 NUMA 系统 stress-ng 的性能回归, 被 phoronix 测试并报告, 参见 [Poking At A Big NUMA Benchmark Regression In Linux 5.18 Git](https://www.phoronix.com/scan.php?page=news_item&px=Linux-518-Stress-NUMA-Goes-Boom) 以及社区报告邮件 [Re: [patch v5] mm: lru_cache_disable: replace work queue synchronization with synchronize_rcu](https://lore.kernel.org/lkml/0ce05ce7-1a00-82df-f37a-bf7f9e216504@MichaelLarabel.com). Stress-NG 0.14 测试项 NUMA 的分数从 v5.17 的 412.88 下降到 v5.18 的 49.33. + +随后社区进行了修复, 参见 [A Fix Is On The Way For A Previously-Reported Linux 5.18 Performance Regression](https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.18-NUMA-Regression-Fix). + +# 3 网络 ------- @@ -85,7 +92,7 @@ https://www.phoronix.com/scan.php?page=news_item&px=UDP-IPV6-5P-Optimization [UDP IPv6 Optimizations Queued Up For Linux 5.18](https://www.phoronix.com/scan.php?page=news_item&px=UDP-IPv6-Opts-5.18) -# 3 IO +# 4 IO ------- [Linux 5.17 To Continue With I/O Optimizations, 5~6% Improvement Pending For NVMe](https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.17-Will-Continue-IO) @@ -97,7 +104,7 @@ https://www.phoronix.com/scan.php?page=news_item&px=UDP-IPV6-5P-Optimization | 2021/08/03 | Peter Oskolkov | [thread_info: use helpers to snapshot thread flags](https://lwn.net/Articles/722293) | 参见报道 [Linux 5.17 To Continue With I/O Optimizations, 5~6% Improvement Pending For NVMe](https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.17-Will-Continue-IO) | v1 ☐ | [PatchWork v4,00/10](https://lore.kernel.org/patchwork/cover/1471548) | -# ARCH +# 5 ARCH ------- | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | diff --git a/study/kernel/00-DESCRIPTION/SCHEDULER.md b/study/kernel/00-DESCRIPTION/SCHEDULER.md index 39f9463..1abe6d7 100644 --- a/study/kernel/00-DESCRIPTION/SCHEDULER.md +++ b/study/kernel/00-DESCRIPTION/SCHEDULER.md @@ -459,6 +459,17 @@ RT_RUNTIME_SHARE 这个机制本身是为了解决不同 CPU 上, 以及不同 | 2012/06/17 | Roman Zippel | [[ANNOUNCE/RFC] Really Fair Scheduler](https://lore.kernel.org/all/Pine.LNX.4.64.0708310139280.1817@scrub.home) | NA | v1 ☐☑✓ | [LORE](https://lore.kernel.org/all/Pine.LNX.4.64.0708310139280.1817@scrub.home) | +ARCHLINUX 的开发者 [hamadmarri](https://github.com/hamadmarri) 为 linux 开发和移植了很多调度器, 并开源在了其 github. + +| 调度器 | 描述 | +|:-----:|:----:| +| [CacULE CPU Scheduler](https://github.com/hamadmarri/cacule-cpu-scheduler) | 是基于交互性评分机制的 CFS 补丁集. 交互性分数的灵感来自FreeBSD 的 ULE 调度器, 可以增强系统响应能力/延迟. | +| [Task Type(TT) CPU Scheduler](https://github.com/hamadmarri/TT-CPU-Scheduler) | 根据任务的行为检测并识别任务类型, 并根据其类型控制调度. 基于任务类型的好处是允许调度程序进行更多控制, 并选择接下来在 CPU 中运行的最佳任务. 当前有5种类型: 实时(REALTIME), 交互(INTERACTIVE), 无类型(NO_TYPE), 计算密集型(CPU_BOUND), 批处理(BATCH). | +| [Baby-CPU-Scheduler](https://github.com/hamadmarri/Baby-CPU-Scheduler) | 一个非常基本, 轻量级但性能非常高的调度程序. 可以将其用作 Linux 上的基本调度程序进行学习 | + +ARCHLINUX 的开发者 ptr1337, 同样移植了 [CacULE Scheduler](https://github.com/ptr1337/linux-cacule), [CachyOS/linux-cachyos](https://github.com/CachyOS/linux-cachyos). + + ## 1.5 调度类的一些其他优化点 ------- @@ -4934,6 +4945,8 @@ ECRTS 2020(32nd Euromicro Conference on Real-Time Systems) 上 Daniel 等人发 ------- #### 12.4.3.1 System76 Scheduler +------- + [System76 Scheduler](https://github.com/pop-os/system76-scheduler) 是 POP-OS 上实现的优化 Linux 的 CPU 调度程序的服务, 他通过自动分配进程优先级等方式, 以提高桌面响应能力. 这些操作跟终端上的优化很像: 1. 当笔记本插上电源时, 将自动激活低延迟 CPU 调度(性能模式), 但是在使用电池时只设置默认调度延迟(能效模式). @@ -4946,6 +4959,11 @@ ECRTS 2020(32nd Euromicro Conference on Real-Time Systems) 上 Daniel 等人发 [System76 Scheduler v1.1 发布](https://www.phoronix.com/scan.php?page=news_item&px=System76-Scheduler-1.1) +#### 12.4.3.2 Ananicy +------- + +[Nefelim4ag/Ananicy](https://github.com/Nefelim4ag/Ananicy) 用于管理进程的 IO 和 CPU 优先级, 它主要用于桌面使用. 它的 github 允许开发人员贡献主流应用程序规则集. + **引用: **