diff --git a/study/kernel/00-DESCRIPTION/ARCH.md b/study/kernel/00-DESCRIPTION/ARCH.md index db0a988..890492d 100644 --- a/study/kernel/00-DESCRIPTION/ARCH.md +++ b/study/kernel/00-DESCRIPTION/ARCH.md @@ -231,6 +231,8 @@ ASYM_PACKING 用于平衡物理核心与 SMT 之间的负载均衡处理 (例如 #### 1.4.1.3 Intel Thread Director (ITD) ------- +[Intel支持混合内核架构的硬件线程调度器是怎么工作的？](https://mp.weixin.qq.com/s/9Zl-h61hm0kDhq68bsBEJQ) + [Thread Director](https://www.anandtech.com/show/16881/a-deep-dive-into-intels-alder-lake-microarchitectures/2) 其实是一个软硬协同优化的范畴. 早在 2018 年, 苹果就发布过自家 AMP 硬件上的软硬协同优化的调度器, 中文专利号 [CN108984282A 具有闭环性能控制器的 AMP 体系结构的调度器 / Scheduler for AMP architecture with closed loop performance controller](https://www.patentguru.com/cn/CN108984282A). @@ -1138,6 +1140,13 @@ openEuler 提供了 [openEuler/prefetch_tuning](https://gitee.com/openeuler/pref | 2023/07/10 | Mina Almasry | [Device Memory TCP](https://lore.kernel.org/all/20230710223304.1174642-1-almasrymina@google.com) | 通过该方案, 我们能够通过直接从设备存储器发送和接收数据, 达到约96.6%的线路速率. 参见 [Google Posts Experimental Linux Code For "Device Memory TCP" - Network To/From Accelerator RAM](https://www.phoronix.com/news/Linux-Device-Memory-TCP). | v1 ☐☑✓ | [LORE v1,0/10](https://lore.kernel.org/all/20230710223304.1174642-1-almasrymina@google.com) | +## 6.14 Cache 带宽管控 +------- + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:---:|:----:|:---:|:----:|:---------:|:----:| +| 2023/08/29 | Tony Luck | [Add support for Sub-NUMA cluster (SNC) systems](https://lore.kernel.org/all/20230829234426.64421-1-tony.luck@intel.com) | [Intel Fixing Up Sub-NUMA Clustering For Linux So That It Behaves With RDT](https://www.phoronix.com/news/Intel-SNT-RDT-Fixing-Up) | v5 ☐☑✓ | [LORE v5,0/8](https://lore.kernel.org/all/20230829234426.64421-1-tony.luck@intel.com) | +
diff --git a/study/kernel/00-DESCRIPTION/BPF.md b/study/kernel/00-DESCRIPTION/BPF.md index 72963fd..11cd9dd 100644 --- a/study/kernel/00-DESCRIPTION/BPF.md +++ b/study/kernel/00-DESCRIPTION/BPF.md @@ -617,6 +617,8 @@ Wasmtime 完全开源, 使用 Rust 编程语言, 是的, 并且符合 WASI 标 ## X.1 业界工具 ------- +[Issue #002 | Programming the Kernel with eBPF](https://www.kerno.io/blog/programming-the-kernel-with-ebpf) + | 工具 | 描述 | |:---:|:----:| | [nyrahul/ebpf-guidee](https://github.com/nyrahul/ebpf-guide) | eBPF 常见问题, 样例以及工具汇总 | diff --git a/study/kernel/00-DESCRIPTION/CGROUP.md b/study/kernel/00-DESCRIPTION/CGROUP.md index 222328f..7534f5b 100644 --- a/study/kernel/00-DESCRIPTION/CGROUP.md +++ b/study/kernel/00-DESCRIPTION/CGROUP.md @@ -68,6 +68,7 @@ blogexcerpt: FZF 是目前最快的模糊搜索工具. 使用golang编写. 结 | 2021/10/09 | "Pratik R. Sampat" | [kernel: Introduce CPU Namespace](https://www.phoronix.com/scan.php?page=news_item&px=Linux-CPU-Namespace) | CPU namespace | RFC ☐ | [LWN RFC, 0/5]](https://lwn.net/Articles/872507) | | 2022/1/7 | Josh Don | [cgroup: add cpu.stat_percpu](https://lkml.org/lkml/2022/1/7/833) | CPU namespace | RFC ☐ | [LORE 1/2]](https://lkml.org/lkml/2022/1/7/833) | + # 2 DEBUG ------- @@ -84,6 +85,13 @@ blogexcerpt: FZF 是目前最快的模糊搜索工具. 使用golang编写. 结 | 2022/11/11 | Tvrtko Ursulin | [DRM scheduling cgroup controller](https://lore.kernel.org/all/20221111104435.3152347-1-tvrtko.ursulin@linux.intel.com) | TODO | v2 ☐☑✓ | [LORE v2,0/13](https://lore.kernel.org/all/20221111104435.3152347-1-tvrtko.ursulin@linux.intel.com) | +# 4 RSTAT + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:---:|:----:|:---:|:----:|:---------:|:----:| +| 2023/07/17 | Hao Jia | [cgroup/rstat: record the cumulative per-cpu time of cgroup and its descendants](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0437719c1a97791481c5fd59642494f2108701a8) | [Linux 6.6 Enables Tracking Per-CPU Cgroup CPU Usage Stats](https://www.phoronix.com/news/Linux-6.6-cgroup) | v1 ☑✓ [6.6-rc1](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7716f383a58314378604eecdd66949ea2cd80ef3) | [LORE](https://lore.kernel.org/all/20230717093612.40846-1-jiahao.os@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v2](https://lore.kernel.org/all/20230807032930.87785-1-jiahao.os@bytedance.com) | + +
* 本作品/博文 ( [AderStep-紫夜阑珊-青伶巷草 Copyright ©2013-2017](http://blog.csdn.net/gatieme) ), 由 [成坚(gatieme)](http://blog.csdn.net/gatieme) 创作. diff --git a/study/kernel/00-DESCRIPTION/DEBUGGING.md b/study/kernel/00-DESCRIPTION/DEBUGGING.md index 92cc987..c69e245 100644 --- a/study/kernel/00-DESCRIPTION/DEBUGGING.md +++ b/study/kernel/00-DESCRIPTION/DEBUGGING.md @@ -66,6 +66,9 @@ blogexcerpt: FZF 是目前最快的模糊搜索工具. 使用golang编写. 结 [CPU Architectures » x86-specific Documentation » 9. ORC unwinderView page source](https://www.kernel.org/doc/html/latest/x86/orc-unwinder.html) +[内核栈回溯原理应用](https://www.cnblogs.com/mysky007/p/12539754. + + | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| | 2018/11/23 | Shile Zhang | [Speed booting by sorting ORC unwind tables at build time](https://lore.kernel.org/patchwork/cover/1162315) | ORC unwind有两个表, .orc_unwind_ip和.orc_unwind 二分搜索需要排序. 在构建时对其进行排序可以节省更多CPU周期有助于加快内核引导. 添加ORC表排序在一个独立的线程有助于避免更多的链接. | RFC v6 ☐ | [PatchWork v6](https://lore.kernel.org/patchwork/cover/1162315) | @@ -305,6 +308,9 @@ $reclaim = current\_mem \times reclaim\_ratio \times max(0,1 – \frac{psi_some} [A discussion on printk()](https://lwn.net/Articles/909980) +[Printk Cleanups Ready For Linux 6.6 - Stepping Towards Threaded/Atomic Console Printing](https://www.phoronix.com/news/Linux-6.6-printk) + + | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| | 2022/02/07 | John Ogness | [implement threaded console printing](https://lore.kernel.org/all/20220207194323.273637-1-john.ogness@linutronix.de) | 参见 phoronix 报道 [Linux Gets Patches For Threaded Console Printing](https://www.phoronix.com/scan.php?page=news_item&px=Linux-Threaded-Console-Print) 和 [Patches Updated For Linux To Enjoy Consoles Running At Full-Speed](https://www.phoronix.com/scan.php?page=news_item&px=Printk-v3-Consoles-Full-Speed) | v1 ☐ | [LORE v1,0/13](https://lore.kernel.org/all/20220207194323.273637-1-john.ogness@linutronix.de) | @@ -318,6 +324,7 @@ $reclaim = current\_mem \times reclaim\_ratio \times max(0,1 – \frac{psi_some} | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| | 2022/09/11 | Thomas Gleixner | [printk: A new approach - WIP](https://lore.kernel.org/all/20220910221947.171557773@linutronix.de) | | v1 ☐☑✓ | [LORE v1,0/29](https://lore.kernel.org/all/20220910221947.171557773@linutronix.de) | +| 2023/09/16 | John Ogness | [provide nbcon base](https://lore.kernel.org/all/20230916192007.608398-1-john.ogness@linutronix.de) | [NBCON Console Patches Updated For Eventually Unblocking Real-Time Linux Kernel](https://www.phoronix.com/news/Linux-NCON-Consoles-v5) | v5 ☐☑✓ | [LORE v5,0/8](https://lore.kernel.org/all/20230916192007.608398-1-john.ogness@linutronix.de) | @@ -467,10 +474,28 @@ x86 和 arm64 都支持直接访问用户空间中的事件计数器. 访问序 | 2022/05/28 | zhengjun | [perf vendor events intel: Add metrics for Sapphirerapids](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=1bcca2b1bd67f3c0e5c3a88ed16c6389f01a5b31) | TODO | v2 ☑✓ 5.19-rc1 | [LORE v2,0/2](https://lore.kernel.org/all/20220528095933.1784141-1-zhengjun.xing@linux.intel.com) | | 2022/08/25 | zhengjun.xing@linux.intel.com | [perf stat: Capitalize topdown metricsnel.org/all/20220825015458.3252239-1-zhengjun.xing@linux.intel.com) | TODO | v1 ☐☑✓ | [LORE](https://lore.kernel.org/9-1-zhengjun.xing@linux.intel.com) | | 2022/10/21 | Shang XiaoJing | [perf vendor events arm64: Fix incorrect Hisi hip08 L3 metrics](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=e9229d5b6254a75291536f582652c599957344d2) | TODO | v2 ☑✓ 6.1-rc3 | [LORE v2,0/3](https://lore.kernel.org/all/20221021105035.10000-1-shangxiaojing@huawei.com) | -| 2022/12/14 | Sandipan Das | [perf vendor events amd: Add Zen 4 events and metrics](https://lore.kernel.org/all/20221214082652.419965-1-sandipan.das@amd.com) | [Linux 6.2 Adds AMD Zen 4 Pipeline Utilization Data To Help Find Performance Bottlenecks](https://www.phoronix.com/news/LInux-6.2-AMD-Zen-4-Events) | v2 ☐☑✓ 6.2-rc1 | [LORE v2,0/4](https://lore.kernel.org/all/20221214082652.419965-1-sandipan.das@amd.com) | +| 2022/12/14 | Sandipan Das | [perf vendor events amd: Add Zen 4 events and metrics](https://lore.kernel.org/all/20221214082652.419965-1-sandipan.das@amd.com) | [Linux 6.2 Adds AMD Zen 4 Pipeline Utilization Data To Help Find Performance Bottlenecks](https://www.phoronix.com/news/LInux-6.2-AMD-Zen-4-Events) | v2 ☐☑✓ 6.2-rc1 | [LORE v2,0/4](https://lore.kernel.org/all/20221214082652.419965-1-sandipan.das@amd.com)| +| 2023/06/07 | kan.liang@linux.intel.com | [New metricgroup output in perf stat default mode](https://lore.kernel.org/all/20230607162700.3234712-1-kan.liang@linux.intel.com) | 在默认模式下, metricgroup 的当前输出包括事件和度量, 这是不必要的, 并且使输出难以读取. 此外, 由于度量中的事件不同, 不同的 ARCH(甚至不同代的 ARCH)可能具有不同的输出格式. 该补丁提出了一种新的输出格式, 只输出每个度量的值和度量组名称. 它可以在 ARCH 和各代之间带来干净一致的输出格式. | v1 ☐☑✓ | [LORE v1,0/8](https://lore.kernel.org/all/20230607162700.3234712-1-kan.liang@linux.intel.com) | -## 11.9 + +## 11.9 perf bperf +------- + + +perf 使用 PMC (performance monitoring counters)来监控系统的性能. PMC 是有限的硬件资源. 例如, Intel cpu 每个 cpu 有 3 个固定 PMC 和 4 个可编程 PMC. + +现代数据中心系统以许多不同的方式使用这些 PMC: 系统级监控、(可能是嵌套的)容器级监控、每个进程监控、分析 (在示例模式下) 等等. 在某些情况下, 活动的 perf_event 比可用的硬件 PMC 还多. 为了允许所有 perf_events 都有机会运行, 有必要对事件进行昂贵的时间复用. + +另一方面, 许多监视工具计算公共度量 (周期、指令). 让多个工具创建多个"循环" 的 perf_event 并占用多个 PMC 是一种浪费. + +bperf 试图通过允许多个 "周期" 或 "指令" 的 perf_event (在不同作用域) 共享 PMU 来减少这种浪费. bperf 没有让每个 perf-stat 会话读取自己的 perf_events, 而是使用 BPF 程序读取 perf_events 并将读取的数据聚合到 BPF 映射中. 然后, perf-stat 会话从这些 BPF 映射中读取值. + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:---:|:----:|:---:|:----:|:---------:|:----:| +| 2020/12/28 | Song Liu | [Introduce perf-stat -b for BPF programs](https://lore.kernel.org/all/20201228174054.907740-1-songliubraving@fb.com) | 引入了 perf-stat-b 选项来统计 BPF 程序的事件. 这与 bpftool 程序配置文件类似. 但是 perf-stat 使它更加灵活. | v6 ☐☑✓ | [LORE v6,0/4](https://lore.kernel.org/all/20201228174054.907740-1-songliubraving@fb.com) | +| 2021/03/16 | Song Liu | [perf-stat: share hardware PMCs with BPF](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7fac83aaf2eecc9e7e7b72da694c49bb4ce7fdfc) | TODO | v2 ☐☑✓ 5.13-rc1 | [LORE](https://lore.kernel.org/all/20210312020257.197137-1-songliubraving@fb.com)
*-*-*-*-*-*-*-*
[LORE v2,0/3](https://lore.kernel.org/all/20210316211837.910506-1-songliubraving@fb.com)| +| 2021/06/25 | Namhyung Kim | [perf stat: Enable BPF counters with --for-each-cgroup](https://lore.kernel.org/all/20210625071826.608504-1-namhyung@kernel.org) | bperf 被添加到使用 BPF 来计算各种性能事件目的, 对其扩展 cgroup 的支持. 与其他 bperf 不同, 它不与其他进程共享事件, 但它可以减少 perf 会话中每个被监视的 GROUP 组的不必要事件 (以及多路复用的开销). 当 `for-each-cgroup` 与 `BPF -counters` 一起使用时, 它将在每个 cpu 内部打开 cgroup-switches 事件, 并附加新的 BPF 程序来读取给定的 perf_events 并聚合 cgroups 的结果. 只有当 task 切换到另一个 group 组中的 task 时才会调用它. | v4 ☐☑✓ | [LORE v4,0/4](https://lore.kernel.org/all/20210625071826.608504-1-namhyung@kernel.org)
*-*-*-*-*-*-*-*
[LORE v5](https://lore.kernel.org/all/20210701211227.1403788-1-namhyung@kernel.org)| # 12 KPROBE @@ -739,7 +764,7 @@ Intel 编译器随后也切到 LLVM 框架, 参见 [Intel Fully Embracing LLVM F | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:---:|:----:|:---:|:----:|:---------:|:----:| -| 2023/06/12 | Peter Zijlstra | [Scope-based Resource Management](https://lore.kernel.org/all/20230612090713.652690195@infradead.org) | [Scope-based resource management for the kernel](https://lwn.net/Articles/934679) 以及 [Scope-Based Resource Management Infrastructure Merged For Linux 6.5](https://www.phoronix.com/news/Linux-6.5-Scope-Resource-Manage). | v3 ☐☑✓ | [LORE v3,0/57](https://lore.kernel.org/all/20230612090713.652690195@infradead.org) | +| 2023/06/12 | Peter Zijlstra | [Scope-based Resource Management](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=7170509cadbb76e5fa7d7b090d2cbdb93d56a2de) | [Scope-based resource management for the kernel](https://lwn.net/Articles/934679) 以及 [Scope-Based Resource Management Infrastructure Merged For Linux 6.5](https://www.phoronix.com/news/Linux-6.5-Scope-Resource-Manage). | v3 ☐☑✓ 6.6-rc1 | [LORE v3,0/57](https://lore.kernel.org/all/20230612090713.652690195@infradead.org) | # 14 FTRACE diff --git a/study/kernel/00-DESCRIPTION/LOCKING.md b/study/kernel/00-DESCRIPTION/LOCKING.md index 324a6bb..2727dbc 100644 --- a/study/kernel/00-DESCRIPTION/LOCKING.md +++ b/study/kernel/00-DESCRIPTION/LOCKING.md @@ -411,7 +411,7 @@ Lockdep 跟踪锁的获取顺序, 以检测死锁, 以及 IRQ 和 IRQ 启用/禁 | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| -| 2022/05/04 | Byungchul Park | [DEPT(Dependency Tracker)](https://lore.kernel.org/all/1651652269-15342-1-git-send-email-byungchul.park@lge.com) | 一种死锁检测工具, 通过跟踪等待/事件而不是锁的获取顺序来检测死锁的可能性, 试图覆盖所有锁(spinlock, mutex, rwlock, seqlock, rwsem)以及同步机制(包括 wait_for_completion, PG_locked, PG_writeback, swait/wakeup 等). | v6 ☐☑✓ | [RFC 00/14](https://lore.kernel.org/lkml/1643078204-12663-1-git-send-email-byungchul.park@lge.com)
*-*-*-*-*-*-*-*
[LORE v6,0/21](https://lore.kernel.org/all/1651652269-15342-1-git-send-email-byungchul.park@lge.com)
*-*-*-*-*-*-*-*
[LORE v7,0/23](https://lore.kernel.org/r/1673235231-30302-1-git-send-email-byungchul.park@lge.com)
*-*-*-*-*-*-*-*
[LORE v8,0/25](https://lore.kernel.org/r/1674782358-25542-1-git-send-email-max.byungchul.park@gmail.com)
*-*-*-*-*-*-*-*
[LORE v9,0/25](https://lore.kernel.org/r/1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com)
*-*-*-*-*-*-*-*
[LORE v10,0/25](https://lore.kernel.org/r/20230626115700.13873-1-byungchul@sk.com)
*-*-*-*-*-*-*-*
[LORE v10,0/25](https://lore.kernel.org/r/20230703094752.79269-1-byungchul@sk.com) | +| 2022/05/04 | Byungchul Park | [DEPT(Dependency Tracker)](https://lore.kernel.org/all/1651652269-15342-1-git-send-email-byungchul.park@lge.com) | 一种死锁检测工具, 通过跟踪等待/事件而不是锁的获取顺序来检测死锁的可能性, 试图覆盖所有锁(spinlock, mutex, rwlock, seqlock, rwsem)以及同步机制(包括 wait_for_completion, PG_locked, PG_writeback, swait/wakeup 等). | v6 ☐☑✓ | [RFC 00/14](https://lore.kernel.org/lkml/1643078204-12663-1-git-send-email-byungchul.park@lge.com)
*-*-*-*-*-*-*-*
[LORE v6,0/21](https://lore.kernel.org/all/1651652269-15342-1-git-send-email-byungchul.park@lge.com)
*-*-*-*-*-*-*-*
[LORE v7,0/23](https://lore.kernel.org/r/1673235231-30302-1-git-send-email-byungchul.park@lge.com)
*-*-*-*-*-*-*-*
[LORE v8,0/25](https://lore.kernel.org/r/1674782358-25542-1-git-send-email-max.byungchul.park@gmail.com)
*-*-*-*-*-*-*-*
[LORE v9,0/25](https://lore.kernel.org/r/1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com)
*-*-*-*-*-*-*-*
[LORE v10,0/25](https://lore.kernel.org/r/20230626115700.13873-1-byungchul@sk.com)
*-*-*-*-*-*-*-*
[LORE v10,0/25](https://lore.kernel.org/r/20230703094752.79269-1-byungchul@sk.com)
*-*-*-*-*-*-*-*
[LORE v10,0/25](https://lore.kernel.org/r/20230821034637.34630-1-byungchul@sk.com) | | 2022/09/15 | 刘顺 | [OSPP 2022: Add lite-lockdep as a lightweight lock validator](https://gitee.com/openeuler/kernel/issues/I5R8DS) | openEuler 开源之夏轻量级死锁检测特性. 参考了 [Low-overhead deadlock prediction](https://dl.acm.org/doi/10.1145/3377811.3380367), [PDF](https://web.cs.ucla.edu/~palsberg/paper/icse20.pdf) | ☐☑✓ | [gitee, PR](https://gitee.com/openeuler/kernel/pulls/112) | @@ -482,6 +482,14 @@ Paul McKenney's parallel programming book, [LWN](https://lwn.net/Articles/421425 | [ChunelFeng/CGraph](https://github.com/ChunelFeng/CGraph) | ChunelFeng 的图化调度并行框架, 轻量, 快捷, 暂不支持异构 | | [AthrunArthur/functionflow](https://github.com/AthrunArthur/functionflow) | 基于 C++11 的 FunctionFlow 并行编程库. | + +### 12.2.2 WorkQueue +------- + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:---:|:----:|:---:|:----:|:---------:|:----:| +| 2023/05/18 | Tejun Heo | [workqueue: Improve unbound workqueue execution locality](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=523a301e66afd1ea9856660bcf3cee3a7c84c6dd) | 重构 workqueue, 使其感知具有多个 L3 缓存的系统受益.
"默认行为是根据最后一级缓存边界进行软仿射. 从给定 LLC 排队的工作项由在同一 LLC 上运行的工作线程执行, 但工作线程可以跨缓存边界移动, 因为计划程序认为合适. 在具有多个 L3 缓存的机器上, 随着小芯片设计变得越来越流行, 这提高了缓存局部性, 同时不会过多地损害工作保护.
未绑定的工作队列现在在执行相关性方面也更加灵活. 支持不同级别的相关性作用域, 并且可以动态修改默认和按工作队列的相关性设置. 这应该有助于解决最近在非对称 ARM CPU 中观察到的次优行为. 参见 [Linux 6.6 WQ Change May Help Out AMD CPUs & Other Systems With Multiple L3 Caches](https://www.phoronix.com/news/Linux-6.6-Workqueue) | v1 ☐☑✓ [6.6-rc1](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bd30fe6a7d9b72e73c5ac9109cbc3066dde08034) | [LORE](https://lore.kernel.org/all/20230519001709.2563-1-tj@kernel.org) | +
* 本作品/博文 ( [AderStep-紫夜阑珊-青伶巷草 Copyright ©2013-2017](http://blog.csdn.net/gatieme) ), 由 [成坚(gatieme)](http://blog.csdn.net/gatieme) 创作. diff --git a/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md b/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md index 4898028..d8424cd 100644 --- a/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md +++ b/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md @@ -358,6 +358,7 @@ Linux 一开始是在一台 i386 上的机器开发的, i386 的硬件页表是 | 2023/07/15 | Matthew Wilcox (Oracle) | [Followup folio conversions for zswap](https://patchwork.kernel.org/project/linux-mm/cover/20230715042343.434588-1-willy@infradead.org/) | 766106 | v1 ☐☑ | [LORE v1,0/5](https://lore.kernel.org/r/20230715042343.434588-1-willy@infradead.org) | | 2023/07/17 | Ryan Roberts | [Optimize large folio interaction with deferred split](https://patchwork.kernel.org/project/linux-mm/cover/20230717143110.260162-1-ryan.roberts@arm.com/) | 766526 | v1 ☐☑ | [LORE v1,0/3](https://lore.kernel.org/r/20230717143110.260162-1-ryan.roberts@arm.com)
*-*-*-*-*-*-*-*
[LORE v2,0/3](https://lore.kernel.org/r/20230719135450.545227-1-ryan.roberts@arm.com)
*-*-*-*-*-*-*-*
[LORE v4,0/3](https://lore.kernel.org/r/20230727141837.3386072-1-ryan.roberts@arm.com) | | 2023/07/28 | Yin, Fengwei | [support large folio for mlock](https://patchwork.kernel.org/project/linux-mm/cover/20230728070929.2487065-1-fengwei.yin@intel.com/) | 770426 | v1 ☐☑ | [LORE v1,0/3](https://lore.kernel.org/r/20230728070929.2487065-1-fengwei.yin@intel.com) | +| 2023/08/21 | Matthew Wilcox | [Convert perf ringbuffer to folios](https://patchwork.kernel.org/project/linux-mm/cover/20230821202016.2910321-1-willy@infradead.org/) | 777998 | v1 ☐☑ | [LORE v1,0/4](https://lore.kernel.org/r/20230821202016.2910321-1-willy@infradead.org) | @@ -662,7 +663,7 @@ Linux 进程使用不同的虚拟地址空间. 因此, 管理该地址空间状 | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| -| 2022/11/03 | Muhammad Usama Anjum | [Implement IOCTL to get and optionally clear info about PTEs](https://patchwork.kernel.org/project/linux-mm/cover/20221103100736.2356351-1-usama.anjum@collabora.com/) | 本补丁系列在 procfs 上实现 IOCTL, 以获取关于页表项 (pte) 的信息. 该 ioctl 支持以下操作: 历史上, 软脏 PTE 位跟踪一直用于 CRIU 项目. procfs 接口足以查找软脏位状态, 并清除进程中所有页面的软脏位. 我们有这样的场景, 需要根据需要跟踪特定页面的软脏 PTE 位. 这就需要在进程运行时跟踪和清除内存区域的机制, 以模拟 Windows 的 getWriteWatch()系统调用. 这个系统调用被游戏用来跟踪脏页, 只处理脏页. CRIU 项目需要页面[相关信息, 如果页面是文件映射、呈现和交换的](https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com). 还需要添加[所需的掩码、任意掩码、排除掩码和返回掩码](https://lore.kernel.org/all/YyiDg79flhWoMDZB@gmail.com). | v4 ☐☑ | [LORE v4,0/3](https://lore.kernel.org/r/20221103100736.2356351-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v6,0/3](https://lore.kernel.org/r/20221109102303.851281-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v12,0/5](https://lore.kernel.org/r/20230406074005.1784728-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v14,0/5](https://lore.kernel.org/r/20230418062008.1434826-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v16,0/5](https://lore.kernel.org/r/20230525085517.281529-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v19,0/5](https://lore.kernel.org/r/20230615141144.665148-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v20,0/5](https://lore.kernel.org/r/20230621072404.2918101-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v21,0/5](https://lore.kernel.org/r/20230626113156.1274521-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v22,0/5](https://lore.kernel.org/r/20230628095426.1886064-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v24,0/5](https://lore.kernel.org/r/20230711125241.1587820-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v26,0/5](https://lore.kernel.org/r/20230727093637.1262110-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v27,0/6](https://lore.kernel.org/r/20230808104309.357852-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v28,0/6](https://lore.kernel.org/r/20230809061603.1969154-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v29,0/6](https://lore.kernel.org/r/20230811180842.3141781-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v32,0/6](https://lore.kernel.org/r/20230816113049.1697849-1-usama.anjum@collabora.com) | +| 2022/11/03 | Muhammad Usama Anjum | [Implement IOCTL to get and optionally clear info about PTEs](https://patchwork.kernel.org/project/linux-mm/cover/20221103100736.2356351-1-usama.anjum@collabora.com/) | 本补丁系列在 procfs 上实现 IOCTL, 以获取关于页表项 (pte) 的信息. 该 ioctl 支持以下操作: 历史上, 软脏 PTE 位跟踪一直用于 CRIU 项目. procfs 接口足以查找软脏位状态, 并清除进程中所有页面的软脏位. 我们有这样的场景, 需要根据需要跟踪特定页面的软脏 PTE 位. 这就需要在进程运行时跟踪和清除内存区域的机制, 以模拟 Windows 的 getWriteWatch()系统调用. 这个系统调用被游戏用来跟踪脏页, 只处理脏页. CRIU 项目需要页面[相关信息, 如果页面是文件映射、呈现和交换的](https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com). 还需要添加[所需的掩码、任意掩码、排除掩码和返回掩码](https://lore.kernel.org/all/YyiDg79flhWoMDZB@gmail.com). | v4 ☐☑ | [LORE v4,0/3](https://lore.kernel.org/r/20221103100736.2356351-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v6,0/3](https://lore.kernel.org/r/20221109102303.851281-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v12,0/5](https://lore.kernel.org/r/20230406074005.1784728-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v14,0/5](https://lore.kernel.org/r/20230418062008.1434826-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v16,0/5](https://lore.kernel.org/r/20230525085517.281529-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v19,0/5](https://lore.kernel.org/r/20230615141144.665148-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v20,0/5](https://lore.kernel.org/r/20230621072404.2918101-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v21,0/5](https://lore.kernel.org/r/20230626113156.1274521-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v22,0/5](https://lore.kernel.org/r/20230628095426.1886064-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v24,0/5](https://lore.kernel.org/r/20230711125241.1587820-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v26,0/5](https://lore.kernel.org/r/20230727093637.1262110-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v27,0/6](https://lore.kernel.org/r/20230808104309.357852-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v28,0/6](https://lore.kernel.org/r/20230809061603.1969154-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v29,0/6](https://lore.kernel.org/r/20230811180842.3141781-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v32,0/6](https://lore.kernel.org/r/20230816113049.1697849-1-usama.anjum@collabora.com)
*-*-*-*-*-*-*-*
[LORE v33,0/6](https://lore.kernel.org/r/20230821141518.870589-1-usama.anjum@collabora.com) | ### 1.7.8 new page table range API @@ -1517,6 +1518,7 @@ v2.6.16-rc1 时, 大家发现 pcp->low 水线貌似并没有什么太大的用 | 2017/01/25 | Mel Gorman | [Recalculate per-cpu page allocator batch and high limits after deferred meminit](https://lore.kernel.org/patchwork/patch/1141598) | 由于 PCP(Per CPU Page) Allocation 中不正确的高限制导致的高阶区域 zone->lock 的竞争, 在初始化阶段, 但是在初始化结束之前, PCP 分配器会计算分批分配 / 释放的页面数量, 以及 Per CPU 列表上允许的最大页面数量. 由于 zone->managed_pages 还不是最新的, pcp 初始化计算不适当的低批量和高值. 在某些情况下, 这会严重增加区域锁争用, 严重程度取决于共享一个本地区域的 cpu 数量和区域的大小. 这个问题导致了构建内核的时间比预期的长得多时, AMD epyc2 机器上的系统 CPU 消耗也过多. 这组补丁修复了这个问题 | v5 ☑ 4.11-rc1 | [LORE 0/3](https://lore.kernel.org/lkml/20191018105606.3249-1-mgorman@techsingularity.net)
*-*-*-*-*-*-*-*
[LORE v2,0/3](https://lore.kernel.org/all/20191021094808.28824-1-mgorman@techsingularity.net), [commit1](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3e8fc0075e24338b1117cdff6a79477427b8dbed) | | 2021/05/25 | Mel Gorman | [Calculate pcp->high based on zone sizes and active CPUs](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=74f44822097c665041010994502b5971d6cd9f04) | pcp->high 和 pcp->batch 根据 zone 内内存的大小进行调整. 移除了不适用的 vm.percpu_pagelist_fraction 参数. | v2 ☑ 5.14-rc1 | [LORE v1,0/6](https://lore.kernel.org/all/20210521102826.28552-1-mgorman@techsingularity.net)
*-*-*-*-*-*-*-*
[LORE v2,0/6](https://lore.kernel.org/lkml/20210525080119.5455-1-mgorman@techsingularity.net) | | 2023/07/10 | Huang, Ying | [mm: PCP high auto-tuning](https://patchwork.kernel.org/project/linux-mm/cover/2023071006532.290366-1-ying.huang@intel.com/) | 不同工作负载的页面分配性能要求通常是不同的. 因此, 我们需要自动将每个 CPU 上的 PCP(Per-CPU Pageset)调高, 以优化页面分配性能. 在该补丁集中, 实现了基于分配/释放深度的 PCP 高自动调优算法. 该算法的基本思想是在足够短的时间内 (大约 1 秒) 检测重复分配和释放模式. 周期需要短, 以便快速响应分配和释放模式更改, 并控制不必要的缓存所浪费的内存.
1. 为了检测重复的分配和释放模式, 在每个 CPU 上计算每个调优周期 (1 秒) 的分配 / 释放深度.
2. 为了计算 alloc/free 深度, 跟踪 alloc 计数. 在从 PCP 分配页面时增加, 在向 PCP 释放页面时减少. 深度是后大值与前小值之间的最大分配计数差. 而自由深度是前一个大值与后一个小值之间的最大分配计数差.
3. 然后, 计算多个调谐周期的平均分配 / 自由深度, 旧的分配 / 自由深度在平均值中逐渐衰减.
4. 最后, 在默认值和最大 PCP 高之间夹紧后, 将 PCP 高设置为平均分配深度和平均自由深度的较小值.
5. 这样, 纯粹的分配或释放不会增加 PCP 值, 因为 PCP 没有帮助. | v1 ☐☑ | [LORE v1,0/2](https://lore.kernel.org/r/20230710065325.290366-1-ying.huang@intel.com) | +| 2023/09/26 | Huang Ying | [mm: PCP high auto-tuning](https://lore.kernel.org/all/20230926060911.266511-1-ying.huang@intel.com) | 不同工作负载的页面分配性能要求通常不同. 因此, 需要自动调整每个 CPU 上的 PCP(每 CPU 页面集)高电平, 以优化页面分配性能. 参见 [Intel Has Another Series Optimizing Linux Performance With PCP High Auto-Tuning](https://www.phoronix.com/news/Intel-Linux-PCP-High-Auto-Tune) | v1 ☐☑✓ | [LORE RFC,0/2](https://lore.kernel.org/all/20230710065325.290366-1-ying.huang@intel.com)
*-*-*-*-*-*-*-*
[LORE v1,0/10](https://lore.kernel.org/all/20230920061856.257597-1-ying.huang@intel.com)
*-*-*-*-*-*-*-*
[LORE v2,0/10](https://lore.kernel.org/all/20230926060911.266511-1-ying.huang@intel.com) | #### 2.2.5.6 Other PCP Improving @@ -1808,7 +1810,7 @@ kmalloc 的 API 家族对 mm 非常关键, 但有一个缺点, 就是它的对 | 2022/04/14 | Hyeonggon Yoo <42.hyeyoo@gmail.com> | [common kmalloc subsystem on SLAB/SLUB](https://patchwork.kernel.org/project/linux-mm/cover/20220308114142.1744229-1-42.hyeyoo@gmail.com) | 清理 slab 公共代码. 在这组补丁之后后, kmalloc 子系统在 SLAB 和 SLUB 之间得到了完美的推广. | v1 ☐☑ | [2022/03/08 LORE v1,00/15](https://lore.kernel.org/r/20220308114142.1744229-1-42.hyeyoo@gmail.com)
*-*-*-*-*-*-*-*
[2022/04/14 LORE v2,0/23](https://lore.kernel.org/r/20220414085727.643099-1-42.hyeyoo@gmail.com)
*-*-*-*-*-*-*-*
[2022/07/12 LORE v3,0/15](https://lore.kernel.org/r/20220712133946.307181-1-42.hyeyoo@gmail.com) | | 2022/07/01 | Feng Tang | [mm/slub: enable debugging memory wasting of kmalloc](https://lore.kernel.org/all/20220701135954.45045-1-feng.tang@intel.com) | 这个补丁帮助显示了当前 kmalloc 下的浪费的空间, 信息在 `/sys/kernel/debug/slab/kmalloc-xx/alloc_traces` 中显示, 显示的格式为: waste = 总共浪费的字节数目 / 单词请求浪费的字节数目. | v1 ☐☑✓ | [LORE RFC](https://lore.kernel.org/r/20220630014715.73330-1-feng.tang@intel.com)
*-*-*-*-*-*-*-*
[LORE](https://lore.kernel.org/all/20220701135954.45045-1-feng.tang@intel.com)
*-*-*-*-*-*-*-*
[LORE v2,0/2](https://lore.kernel.org/r/20220725112025.22625-1-feng.tang@intel.com)
*-*-*-*-*-*-*-*
[LORE v4,0/17](https://lore.kernel.org/r/20220817101826.236819-1-42.hyeyoo@gmail.com) | | 2022/06/07 | Uladzislau Rezki (Sony) | [Reduce a vmalloc internal lock contention preparation work](https://lore.kernel.org/all/20220607093449.3100-1-urezki@gmail.com) | TODO | v1 ☐☑✓ | [LORE v1,0/5](https://lore.kernel.org/all/20220607093449.3100-1-urezki@gmail.com) | -| 2023/05/08 | Gong Ruiqi | [[RFC,v2] Randomized slab caches for kmalloc()](https://patchwork.kernel.org/project/linux-mm/patch/20230508075507.1720950-1-gongruiqi1@huawei.com/) | 745719 | v2 ☐☑ | [LORE v2,0/1](https://lore.kernel.org/r/20230508075507.1720950-1-gongruiqi1@huawei.com)
*-*-*-*-*-*-*-*
[LORE v4,0/1](https://lore.kernel.org/r/20230626031835.2279738-1-gongruiqi@huaweicloud.com) | +| 2023/05/08 | Gong Ruiqi | [[RFC,v2] Randomized slab caches for kmalloc()](https://patchwork.kernel.org/project/linux-mm/patch/20230508075507.1720950-1-gongruiqi1@huawei.com/) | [Linux 6.6 Adding Randomized Kmalloc Caches For Further System Hardening](https://www.phoronix.com/news/Linux-Randomize-Kmalloc-Cache) | v2 ☐☑ | [LORE v2,0/1](https://lore.kernel.org/r/20230508075507.1720950-1-gongruiqi1@huawei.com)
*-*-*-*-*-*-*-*
[LORE v4,0/1](https://lore.kernel.org/r/20230626031835.2279738-1-gongruiqi@huaweicloud.com) | ### 2.3.6 改进与优化 @@ -3979,7 +3981,7 @@ v2.5 的时候引入了 shrink 机制, 并提供了 API 统一了各个模块的 | 2018/08/07 | Kirill Tkhai | [Introduce lockless shrink_slab()](https://patchwork.kernel.org/project/linux-mm/cover/153365347929.19074.12509495712735843805.stgit@localhost.localdomain) | NA | RFC ☐ | [LORE v8,00/17](https://patchwork.kernel.org/project/linux-mm/cover/153365347929.19074.12509495712735843805.stgit@localhost.localdomain) | | 2022/04/02 | Hillf Danton | [[RFC] mm/vmscan: add periodic slab shrinker](https://patchwork.kernel.org/project/linux-mm/patch/20220402072103.5140-1-hdanton@sina.com/) | 当一个具有大量内存的系统在一个目录中有数百万个 negative dentries 时, 在添加 inotify watch 时可能[会发生 softlookup](https://lore.kernel.org/linux-fsdevel/20220209231406.187668-1-stephen.s.brennan@oracle.com). 为了解决这个问题可以添加独立于直接回收和后台回收运行的周期性(periodic) slab shrinker, 以回收已冷却 30 秒以上的 slab 对象. 向 shrink 控件添加 periodic flag, 让缓存所有者知道这是一个周期性收缩器, 它与以最低 recalim 优先级运行的常规 shrinker 相同, 并且可以在没有一次性对象的情况下随意执行任何操作. | v1 ☐☑ | [LORE v1,0/1](https://lore.kernel.org/r/20220402072103.5140-1-hdanton@sina.com) | | 2023/02/20 | Qi Zheng | [make slab shrink lockless](https://patchwork.kernel.org/project/linux-mm/cover/20230220091637.64865-1-zhengqi.arch@bytedance.com/) | 723376 | v1 ☐☑ | [LORE v1,0/5](https://lore.kernel.org/r/20230220091637.64865-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v2,0/7](https://lore.kernel.org/r/20230223132725.11685-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v3,0/8](https://lore.kernel.org/r/20230226144655.79778-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v4,0/8](https://lore.kernel.org/all/20230307065605.58209-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v5,0/8](https://lore.kernel.org/r/20230313112819.38938-1-zhengqi.arch@bytedance.com) | -| 2023/06/22 | Qi Zheng | [use refcount+RCU method to implement lockless slab shrink](https://patchwork.kernel.org/project/linux-mm/cover/20230622085335.77010-1-zhengqi.arch@bytedance.com/) | 759412 | v1 ☐☑ | [LORE v1,0/29](https://lore.kernel.org/r/20230622085335.77010-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v2,0/47](https://lore.kernel.org/r/20230724094354.90817-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v3,0/49](https://lore.kernel.org/r/20230727080502.77895-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v4,0/48](https://lore.kernel.org/r/20230807110936.21819-1-zhengqi.arch@bytedance.com) | +| 2023/06/22 | Qi Zheng | [use refcount+RCU method to implement lockless slab shrink](https://patchwork.kernel.org/project/linux-mm/cover/20230622085335.77010-1-zhengqi.arch@bytedance.com/) | [Shrinking shrinker locking overhead](https://lwn.net/Articles/944199). | v1 ☐☑ | [LORE v1,0/29](https://lore.kernel.org/r/20230622085335.77010-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v2,0/47](https://lore.kernel.org/r/20230724094354.90817-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v3,0/49](https://lore.kernel.org/r/20230727080502.77895-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v4,0/48](https://lore.kernel.org/r/20230807110936.21819-1-zhengqi.arch@bytedance.com) | | 2023/08/16 | Qi Zheng | [use refcount+RCU method to implement lockless slab shrink (part 1)](https://patchwork.kernel.org/project/linux-mm/cover/20230816083419.41088-1-zhengqi.arch@bytedance.com/) | 776546 | v1 ☐☑ | [LORE v1,0/5](https://lore.kernel.org/r/20230816083419.41088-1-zhengqi.arch@bytedance.com)
*-*-*-*-*-*-*-*
[LORE v2,0/5](https://lore.kernel.org/r/20230817112402.77010-1-zhengqi.arch@bytedance.com) | ### 4.3.3 THP Shrinker @@ -4036,7 +4038,7 @@ v2.5 的时候引入了 shrink 机制, 并提供了 API 统一了各个模块的 基于 kstaled/kreclaimd 的方案, 没有合入主线, 但是 idle memory tracking 的方案在优化后, 于 4.3 合入了主线, 命名为 [IDLE_PAGE_TRACKING](https://www.kernel.org/doc/html/latest/admin-guide/mm/idle_page_tracking.html). 作者基于这个特性进程运行所需的实际内存预测 (WSS), 并提供了一[系列工具 idle_page_tracking](https://github.com/sjp38/idle_page_tracking) 来完成这个工作, 参见 [Idle Page Tracking Tools](https://sjp38.github.io/post/idle_page_tracking). -Facebook 指出他们也面临过同样的问题, 所有的 workload 都需要放到 container 里去执行, 用户需要明确申明需要使用多少内存, 不过其实没人知道自己真的会用到多少内存, 因此用户申请的内存数量都太多了, 也就有了类似的 overcommit 和 reclaim 问题. Facebook 的方案是采用 [PSI(pressure-stall information)](https://lwn.net/Articles/759781), 根据这个来了解内存是否变得很紧张了, 相应的会把 LRU list 里最久未用的 page 砍掉. 假如这个举动导致更多的 refault 发生. 不过通过调整内存的回收就调整的激进程度可以缓和 refault. 从而达到较合理的结果, 同时占用的 CPU 时间也会小得多. 描述其设计方案的论文在 ASPLOS’22 中发表, 并成为四篇 Best Paper 之一, 参见 [TMO: Transparent Memory Offloading in Datacenters](https://dl.acm.org/doi/10.1145/3503222.3507731). 网络上的分析 [知乎, 大荒落 TMO: Transparent Memory Offloading in Datacenters](https://zhuanlan.zhihu.com/p/477786756). +Facebook 指出他们也面临过同样的问题, 所有的 workload 都需要放到 container 里去执行, 用户需要明确申明需要使用多少内存, 不过其实没人知道自己真的会用到多少内存, 因此用户申请的内存数量都太多了, 也就有了类似的 overcommit 和 reclaim 问题. Facebook 的方案是采用 [PSI(pressure-stall information)](https://lwn.net/Articles/759781), 根据这个来了解内存是否变得很紧张了, 相应的会把 LRU list 里最久未用的 page 砍掉. 假如这个举动导致更多的 refault 发生. 不过通过调整内存的回收就调整的激进程度可以缓和 refault. 从而达到较合理的结果, 同时占用的 CPU 时间也会小得多. 描述其设计方案的论文在 ASPLOS’22 中发表, 并成为四篇 Best Paper 之一, 参见 [TMO: Transparent Memory Offloading in Datacenters](https://dl.acm.org/doi/10.1145/3503222.3507731). 网络上的分析 [知乎, 大荒落 TMO: Transparent Memory Offloading in Datacenters](https://zhuanlan.zhihu.com/p/477786756), [论文浅析 - TMO Transparent Memory Offloading](https://zhuanlan.zhihu.com/p/578795964). 参见: @@ -5816,6 +5818,7 @@ VMA 的读锁定是使用两个序列号完成的: 一个在 vm_area_struct 中, | 2023/07/05 | Suren Baghdasaryan | [Avoid memory corruption caused by per-VMA locks](https://patchwork.kernel.org/project/linux-mm/cover/20230705063711.2670599-1-surenb@google.com/) | 762531 | v2 ☐☑ | [LORE v2,0/2](https://lore.kernel.org/r/20230705063711.2670599-1-surenb@google.com)
*-*-*-*-*-*-*-*
[LORE v3,0/2](https://lore.kernel.org/r/20230705171213.2843068-1-surenb@google.com)
*-*-*-*-*-*-*-*
[LORE v4,0/2](https://lore.kernel.org/r/20230706011400.2949242-1-surenb@google.com) | | 2023/07/24 | Matthew Wilcox (Oracle) | [Handle most file-backed faults under the VMA lock](https://patchwork.kernel.org/project/linux-mm/cover/20230724185410.1124082-1-willy@infradead.org/) | 769002 | v3 ☐☑ | [LORE v3,0/10](https://lore.kernel.org/r/20230724185410.1124082-1-willy@infradead.org) | | 2023/07/31 | Suren Baghdasaryan | [make vma locking more obvious](https://patchwork.kernel.org/project/linux-mm/cover/20230731171233.1098105-1-surenb@google.com/) | 771308 | v1 ☐☑ | [LORE v1,0/6](https://lore.kernel.org/r/20230731171233.1098105-1-surenb@google.com) | +| 2023/08/21 | Kefeng Wang | [mm: convert to generic VMA lock-based page fault](https://patchwork.kernel.org/project/linux-mm/cover/20230821123056.2109942-1-wangkefeng.wang@huawei.com/) | 777870 | v2 ☐☑ | [LORE v2,0/10](https://lore.kernel.org/r/20230821123056.2109942-1-wangkefeng.wang@huawei.com) | #### 8.2.5.5 Maple Tree @@ -7559,7 +7562,7 @@ OS 判断如果是在用户态触发这个硬件内存错误时, 处理方式是 | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:-----:|:----:|:----:|:----:|:------------:|:----:| -| 2022/01/30 | Edgecombe, Rick P | [Shadow stacks for userspace](https://patchwork.kernel.org/project/linux-mm/cover/20220130211838.8382-1-rick.p.edgecombe@intel.com) | [User-space shadow stacks (maybe) for 6.4](https://lwn.net/Articles/926649). | v1 ☐☑ | [PatchWork v1,0/35](https://lore.kernel.org/r/20220130211838.8382-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v2,0/39](https://lore.kernel.org/r/20220929222936.14584-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v3,0/37](https://lore.kernel.org/r/20221104223604.29615-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v4,0/39](https://lore.kernel.org/r/20221203003606.6838-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v5,0/39](https://lore.kernel.org/r/20230119212317.8324-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v6,0/41](https://lore.kernel.org/r/20230218211433.26859-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v7,0/41](https://lore.kernel.org/r/20230227222957.24501-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v8,0/40](https://lore.kernel.org/r/20230319001535.23210-1-rick.p.edgecombe@intel.com) | +| 2022/01/30 | Edgecombe, Rick P | [Shadow stacks for userspace](https://patchwork.kernel.org/project/linux-mm/cover/20220130211838.8382-1-rick.p.edgecombe@intel.com) | [User-space shadow stacks (maybe) for 6.4](https://lwn.net/Articles/926649), [Intel Shadow Stack Finally Merged For Linux 6.6](https://www.phoronix.com/news/Intel-Shadow-Stack-Linux-6.6). | v1 ☐☑ | [PatchWork v1,0/35](https://lore.kernel.org/r/20220130211838.8382-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v2,0/39](https://lore.kernel.org/r/20220929222936.14584-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v3,0/37](https://lore.kernel.org/r/20221104223604.29615-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v4,0/39](https://lore.kernel.org/r/20221203003606.6838-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v5,0/39](https://lore.kernel.org/r/20230119212317.8324-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v6,0/41](https://lore.kernel.org/r/20230218211433.26859-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v7,0/41](https://lore.kernel.org/r/20230227222957.24501-1-rick.p.edgecombe@intel.com)
*-*-*-*-*-*-*-*
[LORE v8,0/40](https://lore.kernel.org/r/20230319001535.23210-1-rick.p.edgecombe@intel.com) | | 2023/07/16 | Mark Brown | [arm64/gcs: Provide support for GCS in userspace](https://lore.kernel.org/all/20230716-arm64-gcs-v1-0-bf567f93bba6@kernel.org) | 影子堆栈的 64 位 Arm 实现称为"受保护的控制堆栈"("guarded control stack/GCS), 参见 LWN 报道 [Shadow stacks for 64-bit Arm systems](https://lwn.net/Articles/940403). | v1 ☐☑✓ | [LORE v1,0/35](https://lore.kernel.org/all/20230716-arm64-gcs-v1-0-bf567f93bba6@kernel.org)
*-*-*-*-*-*-*-*
[LORE v3,0/36](https://lore.kernel.org/all/20230731-arm64-gcs-v3-0-cddf9f980d98@kernel.org)
*-*-*-*-*-*-*-*
[LORE v4,0/36](https://lore.kernel.org/r/20230807-arm64-gcs-v4-0-68cfa37f9069@kernel.org) | diff --git a/study/kernel/00-DESCRIPTION/OPEN_SOURCE.md b/study/kernel/00-DESCRIPTION/OPEN_SOURCE.md index fddf7d0..d6600f3 100644 --- a/study/kernel/00-DESCRIPTION/OPEN_SOURCE.md +++ b/study/kernel/00-DESCRIPTION/OPEN_SOURCE.md @@ -105,7 +105,7 @@ | 6.3 | NA | NA | [Linux 6.3 Features Expected From AMD Auto IBRS To Pluton CRB TPM2 & Dropping Old Code](https://www.phoronix.com/news/Linux-6.3-Early-Features-Look) | | 6.4 | NA | NA | NA | | 6.5 | [The first half of the 6.5 merge window](https://lwn.net/Articles/936418) | NA | NA | - +| 6.6 | [The first half of the 6.6 merge window](https://lwn.net/Articles/942954), [The rest of the 6.6 merge window](https://lwn.net/Articles/943245) | NA | NA | # 6 业界会议 ------- diff --git a/study/kernel/00-DESCRIPTION/SCHEDULER.md b/study/kernel/00-DESCRIPTION/SCHEDULER.md index 552ea30..daaec68 100644 --- a/study/kernel/00-DESCRIPTION/SCHEDULER.md +++ b/study/kernel/00-DESCRIPTION/SCHEDULER.md @@ -666,6 +666,15 @@ a789992b7674 ck: sched: introduce 'idle seeker' and ID_IDLE_AVG b37e67a6c648 ck: sched: introduce per-cgroup identity ``` +#### 1.5.4.4 SMT control +------- + +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:---:|:----:|:---:|:----:|:---------:|:----:| +| 2023/07/05 | Laurent Dufour | [Introduce SMT level and add PowerPC support](https://lore.kernel.org/all/20230705145143.40545-1-ldufour@linux.ibm.com) | [Partial SMT Enablement Support Lands For Linux 6.6](https://www.phoronix.com/news/Linux-6.6-Partial-SMT-Enable) | v4 ☐☑✓ | [LORE v4,0/10](https://lore.kernel.org/all/20230705145143.40545-1-ldufour@linux.ibm.com) | +| 2023/09/19 | Yicong Yang | [arch_topology: Support SMT control on arm64](https://lore.kernel.org/all/20230919123319.23785-1-yangyicong@huawei.com) | [HiSilicon Posts SMT Run-Time Control Patches For ARM64 Linux](https://www.phoronix.com/news/Linux-ARM64-Run-Time-SMT-Switch) | v1 ☐☑✓ | [LORE](https://lore.kernel.org/all/20230919123319.23785-1-yangyicong@huawei.com) | + + # 2 组调度支持 (Group Scheduling) ------- @@ -967,6 +976,8 @@ v3.8 合入了 [LWN-2013/01/29, Per-entity load tracking](https://lwn.net/Articl ### 3.2.1 PELT 算法思想 ------- +[Rust401/OS-kernel-dev-config- PELT 专栏](https://github.com/Rust401/OS-kernel-dev-config/blob/main/notes/pelt) + 进程消耗的 CPU 时间 和 负载 load 是否有区别, 是的, 当然有区别, Paul Turner 在提交 per-entity load tracking 补丁集的时候对这个问题做了回答. @@ -3387,6 +3398,8 @@ v4.13 引入 NUMA WAKE AFFINE 的时候测试发现, CPU 的空闲造成了 NAS [深入理解 Linux 内核之进程唤醒](https://blog.csdn.net/21cnbao/article/details/119881140) +[郑琦-Linux 源代码跟读-Linux抢占调度](https://zhuanlan.zhihu.com/p/339378819) + ### 4.7.1 Optimize TTWU(try_to_wake_up) ------- @@ -3410,7 +3423,7 @@ v3.0 版本 [sched: Reduce runqueue lock contention -v6](https://git.kernel.org/ | 2022/08/24 | Peng Wang | [sched/fair: select waker's cpu for wakee on sync wakeup](https://lore.kernel.org/all/1508aa17d1a169077c8d8d8c22d2bd529101af0e.1661313074.git.rocking@linux.alibaba.com) | TODO | v1 ☐☑✓ | [LORE](https://lore.kernel.org/all/1508aa17d1a169077c8d8d8c22d2bd529101af0e.1661313074.git.rocking@linux.alibaba.com) | -### 4.7.1.1 TTWU 中的内存屏障 +#### 4.7.1.1 TTWU 上半部中的内存屏障 ------- [<奔跑吧 Linux 内核> 卷 2-- 附录 E 关于 try_to_wake_up() 里的内存屏障使用](https://blog.csdn.net/rlk8888/article/details/123352327) @@ -3751,6 +3764,7 @@ Chen Yu 新的思路是, 首先在 SMT 域中扫描一个空闲的同级节点. | 2023/02/03 | Chen Yu | [sched/fair: Introduce SIS_CURRENT to wake up short task on current CPU](https://lore.kernel.org/lkml/cover.1682661027.git.yu.c.chen@intel.com) | v7 之前 title [sched/fair: Wake short task on current CPU](https://lore.kernel.org/all/cover.1675361144.git.yu.c.chen@intel.com), 其主要目的是避免在不必要时出现过多的跨 CPU 唤醒. 频繁的跨 CPU 唤醒会对某些工作负载造成严重损害, 尤其是在高核心数系统上. 如果唤醒和唤醒都是短时任务, 则通过将唤醒置于唤醒 CPU 上来禁止跨 CPU 唤醒. 短时间的任务可能会成为高负载系统的麻烦制造者, 因为它会带来频繁的上下文切换. 因此, 此策略仅在系统繁忙时生效. 此外, 当仍有空闲 CPU 时, 禁止空闲 CPU 扫描是不合理的. 首先利用第一个补丁 [sched/fair: Record the average duration of a task](https://lore.kernel.org/all/155aa36ba14b8a1f8e6c3ccda7999125edfff990.1675361144.git.yu.c.chen@intel.com) 跟踪任务的平均运行时间 dur_avg, 从而更好地定义小任务. 然后选择本地 CPU 进行唤醒. 测试发现, Intel 2 x 56C/112T 平台的性能有了显著提高. 例如, 在某些情况下, 它将缩放 (1200+%), netperf(600+%). 商业 RDBMS 对 schbench、hacksbench、tbench 和 OLTP 工作负载没有明显影响. 参见 phoronix 报道 [Linux Performance Patches Revved To Avoid Too Many Unnecessary Cross-CPU Wake-ups](https://www.phoronix.com/news/Linux-Wake-Short-Task-CPU). | v5 ☐☑✓ | [LORE](https://lore.kernel.org/all/20220915165407.1776363-1-yu.c.chen@intel.com)
*-*-*-*-*-*-*-*
[LORE v2,0/2](https://lore.kernel.org/all/cover.1666531576.git.yu.c.chen@intel.com)
*-*-*-*-*-*-*-*
[LORE v5,0/2](https://lore.kernel.org/all/cover.1675361144.git.yu.c.chen@intel.com)
*-*-*-*-*-*-*-*
[LORE v6,0/2](https://lore.kernel.org/all/cover.1677069490.git.yu.c.chen@intel.com)
*-*-*-*-*-*-*-*
[LORE v7,0/2](https://lore.kernel.org/all/cover.1682060436.git.yu.c.chen@intel.com)
*-*-*-*-*-*-*-*
[LORE v8,0/2](https://lore.kernel.org/all/cover.1682661027.git.yu.c.chen@intel.com) | | 2023/03/27 | Aaron Lu | [sched/fair: Make tg->load_avg per node](https://lore.kernel.org/all/20230327053955.GA570404@ziqianlu-desk2) | 使用 sysbench 在一个 docker 实例中对 Postgres 进行基准测试, 并将 sysbench 的 nr_threads 设置为 nr_cpu 时, 可以观察到 update_cfs_group() 和 update_load_avg() 在一个 2sockets/112core/224cpu 的 Intel Sapphire Rapids 节点上显示了明显的 cpu 开销 (10% 和 7.8%), 而在另一个节点的 cpu 的热点通常较低 (4% 和 3%). 分析发现热点主要是访问 tg->load_avg, 其中 update_load_avg() 是写端, update_cfs_group() 是读端.
为什么只有一个节点的 CPU 有更大的开销, 原因是: task_group 是根据需要从 slab 分配的, 无论哪个 CPU 进行分配, 分配的 tg 将位于该节点上, 访问 tg->load_avg 将对同一节点上的 CPU 有更低的成本, 而对远程节点的 CPU 有更高的成本.
Tim Chen 告诉我, PeterZ 曾经提到过一种解决类似问题的方法, 即为每个节点设置一个计数器, 所以对 tg->load_avg 也做同样的事情.
优化后, 这两个节点上运行 5 分钟所看到的最坏的情况占比也才 2%.
针对这个工作负载有另外一个发现: 这个工作负载存在有很多唤醒时的任务迁移, 这就是为什么 update_load_avg() 和 update_cfs_group() 显示出明显的成本. 在 N 个实例中运行这个工作负载, 其中 N >= 2, sysbench 的 nr_threads 设置为 1/N nr_cpu, 在唤醒时间上的任务迁移大大减少, 上面提到的两个函数的开销也下降了很多. | v1 ☐☑✓ | [LORE](https://lore.kernel.org/all/20230327053955.GA570404@ziqianlu-desk2) | | 2023/05/16 | Chen Yu | [sched/fair: Introduce SIS_PAIR to wakeup task on local idle core first](https://lore.kernel.org/all/20230516011159.4552-1-yu.c.chen@intel.com) | 在 SMT 域中扫描一个空闲的同级节点. 在之前的上下文切换周期中, 如果唤醒器和唤醒器相互唤醒, 则它们可能共享资源, 并且可以将唤醒器放在唤醒器旁边的空闲兄弟节点上, 以避免 C2C 开销. | v1 ☐☑✓ | [LORE](https://lore.kernel.org/all/20230516011159.4552-1-yu.c.chen@intel.com) | +| 2023/09/12 | Aaron Lu | [Reduce cost of accessing tg->load_avg](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=1528c661c24b407e92194426b0adbb43de859ce0) | [New Scheduler Optimization Can Help Out PostgreSQL & More On Sapphire Rapids](https://www.phoronix.com/news/Linux-Sched-Postgres-SPR) | v2 ☐☑✓ | [LORE v2,0/1](https://lore.kernel.org/all/20230912065808.2530-1-aaron.lu@intel.com) | ### 4.7.5 sync wakeup @@ -5174,7 +5188,7 @@ CPUFreq 驱动是处理和平台相关的逻辑, Governor 中实现了具体的 |:----:|:----:|:---:|:----------:|:---:| | 2021/08/12 | Viresh Kumar | [Add callback to register with energy model](https://lore.kernel.org/patchwork/cover/1424708) | 当前许多 cpufreq 驱动程序向每个策略的注册了能耗模型, 并通过相同的操作 dev_pm_opp_of_register_em() 来完成. 但是随着 thermal-cooling 的完善, 可以在 cpufreq 层次通过新的回调 register_em 来完成这个工作. | v3 ☐ | [PatchWork V3,0/9](https://patchwork.kernel.org/project/linux-arm-kernel/cover/cover.1628742634.git.viresh.kumar@linaro.org) | | 2021/09/08| Viresh Kumar | [Inefficient OPPs](https://patchwork.kernel.org/project/linux-pm/cover/1631109930-290049-1-git-send-email-vincent.donnefort@arm.com) | schedutil 中增加了对低能效 (inefficient) OPP 的感知, 引入 CPUFREQ_RELATION_E 标记来使得 CPUFREQ 只使用和引用有效的频点.
Arm 的 Power 团队在为谷歌的 Pixel4 开发一个实验性内核, 以评估和改进现实生活中 Android 设备上的主线性能和能耗. 发现 SD855 SoC 有几个效率低下的 OPP. 这些 OPP 尽管频率较低, 但功耗却较高, 任务这种频率下工作, 性能不光下降了, 功耗也很高. 通过将它们从 EAS 能效模型中移除, 使得最高效的 CPU 在任务分配上更有吸引力, 有助于减少中、大型 CPU 的运行时间, 同时提高了集群的空闲时间. 由于集群之间存在巨大的能源成本差异, 因此增加空闲时间对该平台来说至关重要. | v7 ☑ 5.16-rc1 | [PatchWork v7,0/9](https://patchwork.kernel.org/project/linux-pm/cover/1631109930-290049-1-git-send-email-vincent.donnefort@arm.com) | -| 2023/07/24 | Jie Zhan | [cpufreq: Support per-policy performance boost](https://lore.kernel.org/all/20230724075827.4160512-1-zhanjie9@hisilicon.com) | 通过添加 "local_boost" sysfs 接口启用按策略提升. 与全局升压开关相同, 将 1/0 写入 "local_boost" 可分别启用 / 禁用 cpufreq 策略上的升压.
全局和本地增压控制的用户视图应为:
1. 启用全局增强最初会对所有策略启用本地增强, 然后可以对每个策略单独启用或禁用本地增强, 前提是平台确实支持.
2. 禁用全局 boost 会使启用本地 boost 成为非法, 而将 0 写入 "local_boost" 是可以的, 但不会生效. [Per-Policy CPU Performance Boosting Proposed For Linux](https://www.phoronix.com/news/Linux-Per-Policy-CPU-Perf-Boost) | v1 ☐☑✓ | [LORE](https://lore.kernel.org/all/20230724075827.4160512-1-zhanjie9@hisilicon.com) | +| 2023/07/24 | Jie Zhan | [cpufreq: Support per-policy performance boost](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=218a06a79d9a98a96ef46bb003d4d8adb0962056) | 通过添加 "local_boost" sysfs 接口启用按策略提升. 与全局升压开关相同, 将 1/0 写入 "local_boost" 可分别启用 / 禁用 cpufreq 策略上的升压.
全局和本地增压控制的用户视图应为:
1. 启用全局增强最初会对所有策略启用本地增强, 然后可以对每个策略单独启用或禁用本地增强, 前提是平台确实支持.
2. 禁用全局 boost 会使启用本地 boost 成为非法, 而将 0 写入 "local_boost" 是可以的, 但不会生效. [Per-Policy CPU Performance Boosting Proposed For Linux](https://www.phoronix.com/news/Linux-Per-Policy-CPU-Perf-Boost) | v1 ☐☑✓ 6.6-rc1 | [LORE](https://lore.kernel.org/all/20230724075827.4160512-1-zhanjie9@hisilicon.com) | ### 7.3.4 各个厂商基于 schedutil 的进一步优化和改进 @@ -5922,6 +5936,8 @@ enqueue_task_fair() [[SchedulerWakeupLatency] Per-task vruntime wakeup bonus](https://lore.kernel.org/all/87blla2pdt.derkling@matbug.net) +EEVDF 最终在 [v6.6-rc1 合入主线](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3ca9a836ff53db8eb76d559764c07fb3b015886a), EEVDF 取代了现有的 CFS 调度程序代码, 虽然最初可能会出现一些性能回归, 但开发人员将积极地解决这些问题. 参见 [EEVDF Scheduler Merged For Linux 6.6, Intel Hybrid Cluster Scheduling Re-Introduced](https://www.phoronix.com/news/Linux-6.6-EEVDF-Merged) 以及 [CGIT](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=d07f09a1f99cabbc86bc5c97d962eb8a466106b5). + EEVDF 全称 "Earliest Eligible Virtual Deadline First" 调度算法, 它并不是什么新事物, 是在 1995 年由 Ion Stoica 和 Hussein Abdel-Wahab 在 1995 年的论文 [Earliest Eligible Virtual Deadline First A Flexible](https://people.eecs.berkeley.edu/~istoica/papers/eevdf-tr-95.pdf) 中描述过. 它的名字就暗示, 它是跟内核的 deadline scheduler 所使用的 Earliest Deadline First algorithm 很类似. 但是这里的差异是, EEVDF 不是一个 realtime 时调度程序, 所以工作方式不一样. 理解 EEVDF 需要掌握几个 (相对) 简单的概念. EEVDF 跟 CFS 一样, 试图把可用的 CPU 时间公平地分配给正在争夺它的那些进程. 例如, 如果有五个进程试图在一个 CPU 上运行, 那么每个进程应该得到 20% 的可用时间. 每个进程的 nice 值可以用来调整其公平时间的计算结果, nice 值较低 (因此优先级较高) 的进程有权获得更多的 CPU 时间, 而牺牲那些具有较高 nice 值的进程. 这些内容都是以前就有的概念. @@ -6002,7 +6018,7 @@ latency_nice 影响的就是 `se->slice` | 2009/09/16 | Ingo Molnar | [sched: Implement a gentler fair-sleepers feature](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=51e0304ce6e55a6e59658558916b4f74da085ff0) | 引入 GENTLE_FAIR_SLEEPERS sched_feature 只给睡眠的线程 50% 的 vruntime 补偿优待, 这使它们能够更快地奔跑, 但不会让他们窃取过多的补偿. | v1 ☐☑✓ | [LORE](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=51e0304ce6e55a6e59658558916b4f74da085ff0) | | 2023/04/01 | Xi Wang | [Morphing CFS into FDL, The Fair Deadline Scheduling Class](https://lore.kernel.org/all/20230401230556.2781604-1-xii@google.com) | TODO | v1 ☐☑✓ | [LORE v1,0/1](https://lore.kernel.org/all/20230401230556.2781604-1-xii@google.com) | | 2023/03/28 | Peter Zijlstra | [sched: EEVDF using latency-nice](https://lore.kernel.org/all/20230328092622.062917921@infradead.org) | [EEVDF Scheduler Patches Updated For The Linux Kernel](https://www.phoronix.com/news/Linux-EEVDF-EO-March) | v1 ☐☑✓ | [LORE 00/10](https://lore.kernel.org/all/20230306132521.968182689@infradead.org)
*-*-*-*-*-*-*-*
[LORE v1,0/17](https://lore.kernel.org/all/20230328092622.062917921@infradead.org) | -| 2023/07/19 | Peter Zijlstra | [sched: EEVDF and latency-nice and/or slice-attr](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=b41bbb33cf75d251a816768580819aec17be718d) | [Updated EEVDF Linux CPU Scheduler Patches Posted That Plan To Replace CFS](https://www.phoronix.com/news/EEVDF-Scheduler-Linux-EO-May) 以及 [EEVDF Scheduler May Be Ready For Landing With Linux 6.6](https://www.phoronix.com/news/Linux-6.6-EEVDF-Likely). | v1 ☐☑✓ 6.6-rc1 | [LORE v1,0/15](https://lore.kernel.org/all/20230531115839.089944915@infradead.org), [CGIT](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=d07f09a1f99cabbc86bc5c97d962eb8a466106b5) | +| 2023/07/19 | Peter Zijlstra | [sched: EEVDF and latency-nice and/or slice-attr](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=b41bbb33cf75d251a816768580819aec17be718d) | [Updated EEVDF Linux CPU Scheduler Patches Posted That Plan To Replace CFS](https://www.phoronix.com/news/EEVDF-Scheduler-Linux-EO-May) 以及 [EEVDF Scheduler May Be Ready For Landing With Linux 6.6](https://www.phoronix.com/news/Linux-6.6-EEVDF-Likely), [EEVDF Scheduler Merged For Linux 6.6, Intel Hybrid Cluster Scheduling Re-Introduced](https://www.phoronix.com/news/Linux-6.6-EEVDF-Merged) | v1 ☐☑✓ 6.6-rc1 | [LORE v1,0/15](https://lore.kernel.org/all/20230531115839.089944915@infradead.org), [CGIT](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?id=d07f09a1f99cabbc86bc5c97d962eb8a466106b5) | ### 8.9.2 Xen CPU Scheduling @@ -6079,7 +6095,7 @@ CONFIG_HW_VIP_THREAD 被标记为 static_vip/dynamic_vip, VIP 线程提供了 mu CONFIG_HUAWEI_SCHED_VIP 被标记为 vip_prio, 为 VIP 线程提供了近似于优先级的功能. 同时提供了较为完善的 VIP Load Balance 机制. -#### 8.9.3.4 +#### 8.9.3.4 业界混部技术 ------- [星汉未来 - 一文看懂业界在离线混部技术](https://blog.51cto.com/u_15513890/5017537) @@ -6149,6 +6165,9 @@ Intel 的 [Wult/Wake Up Latency Tracer](https://github.com/intel/wult) 一个在 ### 10.1.1 进程创建 FORK ------- +[Race-free process creation in the GNU C Library](https://lwn.net/Articles/943022) + + | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| | 2022/10/31 | Zhang Qiao | [sched: sched_fork() optimizations](https://lore.kernel.org/all/20221031125113.72980-1-zhangqiao22@huawei.com) | sched_fork() 使用当前 CPU 初始化新任务的 vruntime, 但新任务可能不在这个 CPU 上运行. 所以这个补丁集将解决这个问题. | v1 ☐☑✓ | [LORE v1,0/2](https://lore.kernel.org/all/20221031125113.72980-1-zhangqiao22@huawei.com)
*-*-*-*-*-*-*-*
[LORE v2,0/2](https://lore.kernel.org/all/20221103120720.39873-1-zhangqiao22@huawei.com) | diff --git a/study/kernel/00-DESCRIPTION/TODO.md b/study/kernel/00-DESCRIPTION/TODO.md index b5c7773..8073670 100644 --- a/study/kernel/00-DESCRIPTION/TODO.md +++ b/study/kernel/00-DESCRIPTION/TODO.md @@ -414,7 +414,6 @@ https://lkml.kernel.org/r/424e2c81-987d-f10e-106d-8b4c611768bc@arm.com 网站 | 2023/04/17 | Vishal Moola | [Split ptdesc from struct page](https://patchwork.kernel.org/project/linux-mm/cover/20230417205048.15870-1-vishal.moola@gmail.com/) | 740662 | v1 ☐☑ | [LORE v1,0/33](https://lore.kernel.org/r/20230417205048.15870-1-vishal.moola@gmail.com) | | 2023/04/18 | Романов Алексей Васильевич | [Introduce objects folding mechanism](https://patchwork.kernel.org/project/linux-mm/cover/20230418062503.62121-1-avromanov@sberdevices.ru/) | 740774 | v1 ☐☑ | [LORE v1,0/5](https://lore.kernel.org/r/20230418062503.62121-1-avromanov@sberdevices.ru) | | 2023/04/18 | Luis Chamberlain | [[v4] module: add debug stats to help identify memory pressure](https://patchwork.kernel.org/project/linux-mm/patch/ZD7jaL8s+pPolwU1@bombadil.infradead.org/) | 741080 | v4 ☐☑ | [LORE v4,0/1](https://lore.kernel.org/r/ZD7jaL8s+pPolwU1@bombadil.infradead.org) | -| 2023/04/18 | Johannes Weiner | [mm: reliable huge page allocator](https://patchwork.kernel.org/project/linux-mm/cover/20230418191313.268131-1-hannes@cmpxchg.org/) | 741091 | v1 ☐☑ | [LORE v1,0/26](https://lore.kernel.org/r/20230418191313.268131-1-hannes@cmpxchg.org) | | 2023/04/18 | Waiman Long | [mm/mmap: Map MAP_STACK to VM_STACK](https://patchwork.kernel.org/project/linux-mm/patch/20230418210230.3495922-1-longman@redhat.com/) | 741110 | v1 ☐☑ | [LORE v1,0/1](https://lore.kernel.org/r/20230418210230.3495922-1-longman@redhat.com) | | 2023/04/18 | Luis Chamberlain | [module: add debugging auto-load duplicate module support](https://patchwork.kernel.org/project/linux-mm/patch/20230418204636.791699-1-mcgrof@kernel.org/) | 741107 | v1 ☐☑ | [LORE v1,0/1](https://lore.kernel.org/r/20230418204636.791699-1-mcgrof@kernel.org) | | 2023/04/18 | Waiman Long | [mm/mmap: Map MAP_STACK to VM_STACK](https://patchwork.kernel.org/project/linux-mm/patch/20230418210230.3495922-1-longman@redhat.com/) | 741110 | v1 ☐☑ | [LORE v1,0/1](https://lore.kernel.org/r/20230418210230.3495922-1-longman@redhat.com) | @@ -555,20 +554,31 @@ BPF verifiery 已经做了很多工作来尽量确保加载进 kernel 的 BPF pr | 2023/08/17 | Kasireddy, Vivek | [udmabuf: Add back support for mapping hugetlb pages (v3)](https://patchwork.kernel.org/project/linux-mm/cover/20230817064623.3424348-1-vivek.kasireddy@intel.com/) | 776879 | v3 ☐☑ | [LORE v3,0/2](https://lore.kernel.org/r/20230817064623.3424348-1-vivek.kasireddy@intel.com) | | 2023/08/17 | Kasireddy, Vivek | [udmabuf: Add support for page migration out of movable zone or CMA](https://patchwork.kernel.org/project/linux-mm/cover/20230817064934.3424431-1-vivek.kasireddy@intel.com/) | 776880 | v1 ☐☑ | [LORE v1,0/3](https://lore.kernel.org/r/20230817064934.3424431-1-vivek.kasireddy@intel.com) | +| 2023/04/18 | Johannes Weiner | [mm: reliable huge page allocator](https://patchwork.kernel.org/project/linux-mm/cover/20230418191313.268131-1-hannes@cmpxchg.org/) | 741091 | v1 ☐☑ | [LORE v1,0/26](https://lore.kernel.org/r/20230418191313.268131-1-hannes@cmpxchg.org) | +| 2023/08/21 | Johannes Weiner | [mm: page_alloc: freelist migratetype hygiene](https://patchwork.kernel.org/project/linux-mm/cover/20230821183733.106619-1-hannes@cmpxchg.org/) | 777975 | v1 ☐☑ | [LORE v1,0/8](https://lore.kernel.org/r/20230821183733.106619-1-hannes@cmpxchg.org) | +| 2023/08/21 | Mateusz Guzik | [execve scalability issues, part 1](https://patchwork.kernel.org/project/linux-mm/cover/20221202829.2163744-1-mjguzik@gmail.com/) | 778000 | v1 ☐☑ | [LORE v1,0/2](https://lore.kernel.org/r/20230821202829.2163744-1-mjguzik@gmail.com) +| 2023/08/21 | Yosry Ahmed | [memcg: non-unified flushing for userspace stats](https://patchwork.kernel.org/project/li-mm/cover/20230821205458.1764662-1-yosryahmed@google.com/) | 778004 | v1 ☐☑ | [LORE v1,0/3](https://lore.kernel.org/r/20230821205458.1764662-1sryahmed@google.com) | -$$$ -avg\_vruntime_{cfs\_rq}' = avg\_vruntime_{cfs\_rq} - avg\_load_{cfs\_rq} * delta -= \sum_{i=0}^{N} (vruntime_{se} - min\_vruntime_{cfs\_rq}) \times weight - \sum_{i=0}^{N} load\_weight * delta -= \sum_{i=0}^{N} [vruntime_{se} - (min\_vruntime_{cfs\_rq} - delta)] \times weight] -$$$ + +[Why glibc's fstat() is slow](https://lwn.net/Articles/944214) + +[Linux 6.7 To Make It Easier To Toggle Support For x86 32-bit Programs](https://www.phoronix.com/news/Linux-6.7-ia32_emulation-Boot) +[Red Hat Working On Delayed Module Signature Verification To Speed-Up Linux Boot Times](https://www.phoronix.com/news/Linux-Delay-Module-Verification) +[Linux's SLUB Allocator Preparing To Better Fend Off Cross-Cache Attacks](https://www.phoronix.com/news/Linux-SLUB-Prevent-Cross-Cache) -$`avg\_vruntime_{cfs\_rq}' = avg\_vruntime_{cfs\_rq} - avg\_load_{cfs\_rq} * delta -= \sum \limits_{i = 0}^{N} (vruntime_{se} - min\_vruntime_{cfs\_rq}) \times weight - \sum \limits_{i = 0}^{N} load\_weight * delta -= \sum \limits_{i = 0}^{N} [vruntime_{se} - (min\_vruntime_{cfs\_rq} - delta)] \times weight]`$ +[Intel oneAPI Initiative Evolves Into The Unified Acceleration "UXL" Foundation](https://www.phoronix.com/review/oneapi-uxl-foundation) +[](https://www.phoronix.com/news/Linux-6.6-x86-microcode) + +[Moving physical pages from user space](https://lwn.net/Articles/944115) +[Revisiting the kernel's preemption models (part 1)](https://lwn.net/Articles/944686) +[User-space spinlocks with help from rseq()](https://lwn.net/Articles/944895) + + +[Intel支持混合内核架构的硬件线程调度器是怎么工作的？](https://mp.weixin.qq.com/s/9Zl-h61hm0kDhq68bsBEJQ) \ No newline at end of file