From b947a39080ef9b32a64d772ce73d83911438e743 Mon Sep 17 00:00:00 2001 From: Cheng Jian Date: Wed, 16 Jun 2021 22:15:29 +0800 Subject: [PATCH] description/memory: thrash detection-based LRU, working set and refault --- study/kernel/00-DESCRIPTION/DEBUGGING.md | 1 + study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md | 21 +++++++++++++------ 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/study/kernel/00-DESCRIPTION/DEBUGGING.md b/study/kernel/00-DESCRIPTION/DEBUGGING.md index de7863e..56309fd 100644 --- a/study/kernel/00-DESCRIPTION/DEBUGGING.md +++ b/study/kernel/00-DESCRIPTION/DEBUGGING.md @@ -186,6 +186,7 @@ Facebook 在 2018 年开源了一套解决重要计算集群管理问题的 Linu | 2018/08/28 | Johannes Weiner | [psi: pressure stall information for CPU, memory, and IO v4](https://lwn.net/Articles/759781) | 引入 PSI 评估系统 CPU, MEMORY, IO 等资源的压力. | v4 ☑ [4.20-rc1](https://kernelnewbies.org/Linux_4.20#Core_.28various.29) | [Patchwork](https://lore.kernel.org/patchwork/patch/978495) | | 2019/03/19 | Suren Baghdasaryan | [psi: pressure stall monitors v6](https://lwn.net/Articles/775971/) | NA | v6 ☑ [5.2-rc1](https://kernelnewbies.org/Linux_5.2#Improved_Presure_Stall_Information_for_better_resource_monitoring) | [Patchwork](https://lore.kernel.org/patchwork/patch/1052413) | | 2020/03/03 | Suren Baghdasaryan | [psi: Add PSI_CPU_FULL state and some code optimization](ttps://lore.kernel.org/patchwork/patch/1388805) | 1. 添加 PSI_CPU_FULL 状态标记 cgroup 中的所有非空闲任务在 cgroup 之外的 CPU 资源上被延迟, 或者 cgroup 被 throttle
2. 使用 ONCPU 状态和当前的 in_memstall 标志来检测回收, 删除 timer tick 中的钩子, 使代码更简洁和可维护.
4. 通过移除两个任务的每个公共cgroup祖先的psi_group_change()调用来优化自愿睡眠开关。 | v2 ☑ 5.13-rc1 | [Patchwork](https://lore.kernel.org/patchwork/patch/1388805) | +| 2020/03/31 | Yafang Shao | [psi: enhance psi with the help of ebpf](https://lwn.net/Articles/1218304) | 引入 psi_memstall_type 标记 MEMSTALL 的类别, 并在 tracepoint 输出, 从而可以被 ebpf 使用来增强工具. | v4 ☑ [4.20-rc1](https://kernelnewbies.org/Linux_4.20#Core_.28various.29) | [Patchwork](https://lore.kernel.org/patchwork/patch/1218304) |
diff --git a/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md b/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md index 931317b..fb1b442 100644 --- a/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md +++ b/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md @@ -723,7 +723,7 @@ aaba9265318 [PATCH] make pagemap_lru_lock irq-safe | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| -| 2016/07/08 | Mel Gorman | [Move LRU page reclaim from zones to nodes v9](https://lore.kernel.org/patchwork/cover/696408) | 实现 SLOB 分配器 | v9 ☑ [4.8](https://kernelnewbies.org/Linux_4.8#Memory_management) | [PatchWork v21](https://lore.kernel.org/patchwork/cover/696408) | +| 2016/07/08 | Mel Gorman | [Move LRU page reclaim from zones to nodes v9](https://lore.kernel.org/patchwork/cover/696408) | 将 LRU 页面的回收从 ZONE 切换到 NODE. | v9 ☑ [4.8-rc1](https://kernelnewbies.org/Linux_4.8#Memory_management) | [PatchWork v21](https://lore.kernel.org/patchwork/cover/696408) | [memcg lru lock 血泪史](https://blog.csdn.net/bjchenxu/article/details/112504932) @@ -889,9 +889,10 @@ https://github.com/hakavlad/le9-patch | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | |:----:|:----:|:---:|:----:|:---------:|:----:| -| 2014/04/03 | Johannes Weiner | [mm: thrash detection-based file cache sizing v9](https://lwn.net/Articles/495543) | 引入基于 Refault Distance 算法的文件高速缓存页的工作集探测 | v9 ☑ [3.15](https://kernelnewbies.org/Linux_3.15#head-dbe2430cd9e5ed1d3f2362367758cd490aba4b9d) | [PatchWork v9](https://lore.kernel.org/patchwork/cover/437949) | -| 2018/08/28 | Johannes Weiner | [psi: pressure stall information for CPU, memory, and IO v4](https://lore.kernel.org/patchwork/cover/978495) | NA | v1 ☑ [4.20-rc1](https://kernelnewbies.org/Linux_4.20#Memory_management) | [PatchWork](https://lore.kernel.org/patchwork/cover/978495) | -| 2018/10/09 | Johannes Weiner | [mm: workingset & shrinker fixes](https://lore.kernel.org/patchwork/cover/997829) | NA | v1 ☑ [4.20-rc1](https://kernelnewbies.org/Linux_4.20#Memory_management) | [PatchWork](https://lore.kernel.org/patchwork/cover/997829) | +| 2014/04/03 | Johannes Weiner | [mm: thrash detection-based file cache sizing v9](https://lwn.net/Articles/495543) | 实现基于 Refault Distance 算法的文件高速缓存页的工作集探测, 引入 WORKINGSET_REFAULT, WORKINGSET_ACTIVATE, WORKINGSET_NODERECLAIM 三种工作集. | v9 ☑ [3.15](https://kernelnewbies.org/Linux_3.15#head-dbe2430cd9e5ed1d3f2362367758cd490aba4b9d) | [PatchWork v9](https://lore.kernel.org/patchwork/cover/437949) | +| 2016/07/08 | Mel Gorman | [Move LRU page reclaim from zones to nodes v9](https://lore.kernel.org/patchwork/cover/696408) | 将 LRU 页面的回收从 ZONE 切换到 NODE. 这里需要将 workingset 从 zone 切换到 node 上. | v9 ☑ [4.8](https://kernelnewbies.org/Linux_4.8#Memory_management) | [PatchWork v21](https://lore.kernel.org/patchwork/cover/696408), [commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1e6b10857f91685c60c341703ece4ae9bb775cf3) | +| 2018/08/28 | Johannes Weiner | [psi: pressure stall information for CPU, memory, and IO v4](https://lore.kernel.org/patchwork/cover/978495) | Refaults 发生在工作集转换和就地抖动期间. 在工作集转换期间, 非活动缓存发生 Refaults 并推出已建立的活动缓存. 但是, 如果活动缓存没有过期, 并且最终会出现 Refaults, 就会造成抖动. 引入一个新的页标志 WORKINGSET_RESTORE, 它在退出时告诉页面在其生命周期内是否处于活动状态. 然后将此位存储在影子条目中, 将故障分类为转换或抖动. | v1 ☑ [4.20-rc1](https://kernelnewbies.org/Linux_4.20#Memory_management) | [PatchWork](https://lore.kernel.org/patchwork/cover/978495), [commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1899ad18c6072d689896badafb81267b0a1092a4) | +| 2018/10/09 | Johannes Weiner | [mm: workingset & shrinker fixes](https://lore.kernel.org/patchwork/cover/997829) | 通过为循环中的影子节点添加一个计数器, 可以更容易地捕获影子节点收缩器中的 bug. | v1 ☑ [4.20-rc1](https://kernelnewbies.org/Linux_4.20#Memory_management) | [PatchWork](https://lore.kernel.org/patchwork/cover/997829), [commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=68d48e6a2df575b935edd420396c3cb8b6aa6ad3) | | 2020/04/03 | Joonsoo Kim | [workingset protection/detection on the anonymous LRU list](https://lwn.net/Articles/815342) | 实现对匿名 LRU 页面列表的工作集保护和检测. 在之前的实现中, 新创建的或交换中的匿名页, 都是默认加入到 active LRU list, 然后逐渐降级到 inactive LRU list. 这造成在某种场景下新申请的内存(即使被使用一次cold page)也会把在a ctive list 的 hot page 挤到 inactive list. 为了解决这个的问题, 这组补丁, 将新创建或交换的匿名页面放到 inactive LRU list 中, 只有当它们被足够引用时才会被提升到活动列表. 另外, 因为这些更改可能导致新创建的匿名页面或交换中的匿名页面交换不活动列表中的现有页面, 所以工作集检测被扩展到处理匿名LRU列表. 以做出更优的决策. | v5 ☑ [5.9-rc1](https://kernelnewbies.org/Linux_5.9#Memory_management) | [PatchWork v5](https://lore.kernel.org/patchwork/cover/1219942), [Patchwork v7](https://lore.kernel.org/patchwork/patch/1278082), [ZhiHu](https://zhuanlan.zhihu.com/p/113220105) | | 2020/05/20 | Johannes Weiner | [mm: balance LRU lists based on relative thrashing v2](https://lore.kernel.org/patchwork/cover/1245255) | 基于相对抖动平衡 LRU 列表(重新实现了页面缓存和匿名页面之间的 LRU 平衡, 以便更好地与快速随机 IO 交换设备一起工作). : 在交换和缓存回收之间平衡的回收代码试图仅基于内存引用模式预测可能的重用. 随着时间的推移, 平衡代码已经被调优到一个点, 即它主要用于页面缓存, 并推迟交换, 直到 VM 处于显著的内存压力之下. 因为 commit a528910e12ec Linux 有精确的故障 IO 跟踪-回收错误页面的最终代价. 这允许我们使用基于 IO 成本的平衡模型, 当缓存发生抖动时, 这种模型更积极地扫描匿名内存, 同时能够避免不必要的交换风暴. | v1 ☑ [5.8-rc1](https://kernelnewbies.org/Linux_5.8#Memory_management) | [PatchWork v1](https://lore.kernel.org/patchwork/cover/685701)
*-*-*-*-*-*-*-*
[PatchWork v2](https://lore.kernel.org/patchwork/cover/1245255) | @@ -1150,7 +1151,7 @@ https://lore.kernel.org/patchwork/cover/1118785 # 2.8 进程虚拟地址空间(VMA) ------- -### 2.8.1 进程 VMA +### 2.8.1 VMA ------- | 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | @@ -1161,7 +1162,7 @@ https://lore.kernel.org/patchwork/cover/1118785 -### 2.8.2 缺页异常 +### 2.8.2 Mapping ------- #### 2.8.2.1 COW @@ -1195,6 +1196,8 @@ copy_mm #### 2.8.2.2 MMAP locing ------- +[[LSF/MM TOPIC] mmap locking topics](https://www.spinics.net/lists/linux-mm/msg258803.html) + [The LRU lock and mmap_sem](https://lwn.net/Articles/753058) https://events.static.linuxfound.org/sites/events/files/slides/mm.pdf @@ -1206,8 +1209,14 @@ https://events.static.linuxfound.org/sites/events/files/slides/mm.pdf | 2018/05/17 | Laurent Dufour | [Speculative page faults](http://lore.kernel.org/patchwork/patch/906210) | SPF | v11 ☐ | [PatchWork](https://lore.kernel.org/patchwork/patch/906210) | | 2021/04/20 | Michel Lespinasse | [Speculative page faults (anon vmas only)](http://lore.kernel.org/patchwork/patch/1420569) | SPF | v11 ☐ | [PatchWork RFC,00/37](https://lore.kernel.org/patchwork/cover/1408784)
*-*-*-*-*-*-*-*
[PatchWork v1](https://lore.kernel.org/patchwork/patch/1420569) | +* Fine grained MM locking +| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 | +|:----:|:----:|:---:|:----:|:---------:|:----:| +| 2013/01/31 | Michel Lespinasse | [Mapping range lock](https://lore.kernel.org/patchwork/cover/356467) | 文件映射的 mapping lock | RFC ☐ | [PatchWork RFC](https://lore.kernel.org/patchwork/cover/356467) | +| 2020/02/24 | Michel Lespinasse | [Fine grained MM locking](https://patchwork.kernel.org/project/linux-mm/cover/20200224203057.162467-1-walken@google.com) | 细粒度 MM MMAP lock | RFC ☐ | [PatchWork RFC](https://patchwork.kernel.org/project/linux-mm/cover/20200224203057.162467-1-walken@google.com) | + ## 2.8.3 反向映射 RMAP(Reverse Mapping) -------