diff --git a/distro/OpenAnolis/README.md b/distro/OpenAnolis/README.md index 9463efd..ec59f07 100755 --- a/distro/OpenAnolis/README.md +++ b/distro/OpenAnolis/README.md @@ -225,8 +225,12 @@ d9ba6d0880e3 alinux: sched: Introduce primitives for CFS bandwidth burst | identity | 描述 | |:--------:|:---:| | ID_NORMAL | 普通 CFS 任务. | -| ID_HIGHCLASS | 任务抢占正常的和底层的任务在唤醒, 也考虑底层的 CPU 作为空闲. | -| ID_UNDERCLASS 任务在唤醒时会受到惩罚. | +| ID_HIGHCLASS | 在唤醒时可以抢占 ID_NORMAL 和 ID_HIGHCLASS 进程, 也会考虑运行着 ID_UNDERCLASS 的 CPU 作为 IDLE CPU. | +| ID_UNDERCLASS | 低优先级的任务, 任务在唤醒时会受到惩罚. | +| ID_SMT_EXPELLER | | +| ID_IDLE_SAVER | | +| ID_IDLE_SEEKER | 在选核时忽略 SIS_PROP 造成的限制, 并有更多机会在 select_idke_cpu() 中找到真正的空闲 CPU. | + 而遗留条目 'bvt_warp_ns' 是为了兼容(老版本)而保留的, 与身份的关系是: @@ -273,7 +277,60 @@ commit [b2818621d5e7 ("alinux: sched/isolation: dynamical CPU isolation support" |:-----------:|:----:|:------:| | dynamical CPU isolation | fix #28231823 | [b2818621d5e7 alinux: sched/isolation: dynamical CPU isolation support](https://github.com/gatieme/linux/commit/b2818621d5e7) | +## 1.9 sched_feature +------- +### 1.9.1 WA_STATIC_WEIGHT +------- + + +引入 WA_STATIC_WEIGHT, wake_affine 比较负载的时候, + +```cpp +d2440c99979d alinux: sched/fair: use static load in wake_affine_weight +``` + +### 1.9.2 ID_IDLE_AVG(to #30665478) +------- + + +SIS_PROP 限制空闲 CPU 的搜索范围, 在使用了 per-cgroup identity 之后经常导致高级任务的不稳定延迟. + +1. 在 identity 中引入了 ID_IDLE_SEEKER 标识, 默认情况下为 bvt 为 2 或 1 的组激活, 使得他们忽略 SIS_PROP 造成的限制, 并有更多机会在 select_idke_cpu() 中找到真正的空闲 CPU. 通过关闭 "cpu.identity" 中的第 4 位, 标识可能会失效. + +2. 使用 avg_idle 作为旋钮, 以确保下层阶级不会占用太多空闲的 cpu, 但是, 这不是很公平, 因为其他任务甚至可能不会运行. + +因此, 引入了新的 sched_feature: ID_IDLE_AVG, 引入新的 `rq->avg_id_idle` 表示 CPU 上空闲和 ID_UNDERCLASS 类别的进程执行的平均周期, 通过将其代替为 avg_idle, ID_UNDERCLASS 将仅在其他任务快速使用时节省空闲 CPU. + +此外, sched_debug 条目将立即打印 ID_HIGHCLASS 和 ID_UNDERCLASS 进程的执行时间总和. + +```cpp +bcc726c56a0f alinux: sched: introduce 'idle seeker' and ID_IDLE_AVG +``` + +### 1.9.3 ID_RESCUE_EXPELLEE(to #30665478) +------- + +```cpp +e6f05bd1d63a alinux: sched: fix the performence regression caused by update_rq_on_expel() +c9af3e52ed2a Revert "alinux: sched: fix the performence regression caused by update_rq_on_expel()" +c848a4fd1ecd alinux: sched: fix the performence regression caused by update_rq_on_expel() +a4f07eb17f13 alinux:sched: rescue the expellee on migration +``` + +### 1.9.3 ID_LAST_HIGHCLASS_STAY(fix #34923487) +------- + +在 nginx(highclass) + ffmpeg(underclass) 场景中, 开发者发现发现在应用修补程序修复了nr_high_running 下溢的错误(修复补丁 [commit 60aa4e7d6d9a "alinux: sched: fix the bug that nr_high_running underflow"](https://github.com/gatieme/linux/commit/60aa4e7d6d9a))后, 性能会下降, 这意味着当 nr_high_running 运行错误时, 性能会更好. + +最后, 开发者发现, 如果我们跳过第一个判断 "if (is_highclass_task(p) && src_rq->nr_high_running < 2)", nginx 的性能将提高 10%. + +但是这个修正并不适用于所有场景, 所以引入了一个 [sched_feature : ID_LAST_HIGHCLASS_STAY](https://github.com/gatieme/linux/commit/25f6e3e64a77). 当 ID_LAST_HIGHCLASS_STAY 处于启用状态时, HIGHCLASS 任务将更倾向于保留而不是迁移. ID_LAST_HIGHCLASS_STAY 的默认值为 true. + + +| DESCRIPTION | task | commit | +|:-----------:|:----:|:------:| +| ID_LAST_HIGHCLASS_STAY | to #34923487 | [25f6e3e64a77 alinux: sched: Introduce sched_feat ID_LAST_HIGHCLASS_STAY](https://github.com/gatieme/linux/commit/25f6e3e64a77) | ## 1.X 性能优化 ------- @@ -295,11 +352,6 @@ bcaf8afd6270 alinux: sched: Add switch for scheduler_tick load tracking bb48b716f496 alinux: sched: Add switch for update_blocked_averages ``` -* 引入 WA_STATIC_WEIGHT, wake_affine 比较负载的时候, - -```cpp -d2440c99979d alinux: sched/fair: use static load in wake_affine_weight -``` ### 1.X.2 other fix ------- @@ -320,19 +372,7 @@ f381d3d2c39c sched/core: Fix CPU controller for !RT_GROUP_SCHED 417cf53b4b85 sched/fair: Fix imbalance due to CPU affinity ``` -### 1.X.3 ID_LAST_HIGHCLASS_STAY(fix #34923487) -------- -在 nginx(highclass) + ffmpeg(underclass) 场景中, 开发者发现发现在应用修补程序修复了nr_high_running 下溢的错误(修复补丁 [commit 60aa4e7d6d9a "alinux: sched: fix the bug that nr_high_running underflow"](https://github.com/gatieme/linux/commit/60aa4e7d6d9a))后, 性能会下降, 这意味着当 nr_high_running 运行错误时, 性能会更好. - -最后, 开发者发现, 如果我们跳过第一个判断 "if (is_highclass_task(p) && src_rq->nr_high_running < 2)", nginx 的性能将提高 10%. - -但是这个修正并不适用于所有场景, 所以引入了一个 [sched_feature : ID_HIGHCLASS_STAY](https://github.com/gatieme/linux/commit/25f6e3e64a77). 当 ID_LAST_HIGHCLASS_STAY 处于启用状态时, HIGHCLASS 任务将更倾向于保留而不是迁移. ID_LAST_HIGHCLASS_STAY 的默认值为 true. - - -| DESCRIPTION | task | commit | -|:-----------:|:----:|:------:| -| ID_LAST_HIGHCLASS_STAY | to #34923487 | [25f6e3e64a77 alinux: sched: Introduce sched_feat ID_LAST_HIGHCLASS_STAY](https://github.com/gatieme/linux/commit/25f6e3e64a77) | diff --git a/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md b/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md index fb7a721..80bbb0c 100644 --- a/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md +++ b/study/kernel/00-DESCRIPTION/MEMORY_MANAGER.md @@ -1471,11 +1471,11 @@ Refault Distance 算法是为了解决前者, 在第二次读时, 人为地把 p | 2016/01/24 | Vladimir Davydov / | [Make workingset detection logic memcg aware](https://lwn.net/Articles/586023) | 工作集探测感知 memcg.
目前, 工作集检测逻辑不是 memcg 感知的 - inactive_age 是每个 zone 域维护的. 因此, 如果使用了内存组, 则会随机激活错误的文件页. 这个补丁集使每个 lruvec 具有 inactive_age, 以便工作集检测将正确地工作在内存 cgroup 回收. | v2 ☐ | [PatchWork 0/3](https://lore.kernel.org/patchwork/cover/586023)
*-*-*-*-*-*-*-*
[2016/01/24 PatchWork v2](https://lore.kernel.org/patchwork/patch/638421/) | | 2016/01/29 | Johannes Weiner | [mm: workingset: per-cgroup thrash detection](https://lore.kernel.org/patchwork/cover/641469) | 这组补丁使工作集检测具备 cgroup 感知的能力, 因此当使用 cgroup 内存控制器时, 页面回收可以正常工作.
缓存抖动检测目前仅在系统级别工作, 而不在 cgroup 层次工作. 更糟糕的是, 由于将 refaults 与 active LRU 的全局数量相比较, 当 cgroup 的页面比其他组的页面更热时, 它们可能会错误地激活所有 refaults 的页面.
1. 将 refault 的统计 inactive_age 从区域移动到 lruvec.
2. 然后用 memcg ID 标记逐出条目. | v2 ☑ [4.6-rc1](https://kernelnewbies.org/Linux_4.20#Memory_management) | [2016/01/26 PatchWork 0/5](https://lore.kernel.org/patchwork/cover/639567)
*-*-*-*-*-*-*-*
[2016/01/29 PatchWork v2,0/5](https://lore.kernel.org/patchwork/cover/641469) | | 2016/02/09 | Vladimir Davydov | [mm: workingset: make shadow node shrinker memcg aware](https://lwn.net/Articles/586023) | 工作集探测已经被 memcg 识别, 但阴影节点收缩器仍然是全局的. 因此, 一个小 cgroup 可能会消耗阴影节点的所有可用内存, 通过回收某个 cgroup 的阴影节点可能会损害其他 cgroup, 即使回收这些阴影节点中存储的距离没有任何效果. 为了避免这种情况， 我们需要使阴影节点收缩器也变成 memcg 感知的.
实际工作在本系列的补丁6中完成. 修补程序1和2为 memcg/shrinker 基础架构的更改做好准备. 补丁3只是一个附带的清理. 补丁4使基数树节点占位, 这是使阴影节点收缩器 memcg 感知所必需的. 补丁5在工作负载主要使用匿名页面的情况下减少了阴影节点的开销. | v2 ☑ 4.6-rc1 | [2016/02/07 PatchWork 0/5](https://lore.kernel.org/patchwork/cover/644646)
*-*-*-*-*-*-*-*
[2016/02/09 PatchWork v2,0/6](https://lore.kernel.org/patchwork/cover/645395) | -| 2016/04/04 | Johannes Weiner | [mm: support bigger cache workingsets and protect against writes](https://lore.kernel.org/patchwork/cover/664653) | NA | v1 ☑ [4.7-rc1](https://kernelnewbies.org/Linux_4.7#Memory_management) | [PatchWork v1](https://lore.kernel.org/patchwork/cover/664653) | +| 2016/04/04 | Johannes Weiner | [mm: support bigger cache workingsets and protect against writes](https://lore.kernel.org/patchwork/cover/664653) | 支持更大的缓存工作集并防止写入. 由于某些特定 PostgreSQL 工作负载存在问题, 做了两项更改.
1. 停止升级写入或重写到活动文件页列表中的页, 因为数据很有可能不会再次读取, 甚至可能在读取之前再次重写, 因此, 这些页面可能会占用活动文件列表中的空间, 这些空间可供频繁读取的页面使用.
2. 删除要求[非活动文件列表的最小文件缓存大小为 50% ](https://elixir.bootlin.com/linux/v4.6/source/mm/vmscan.c#L1929)的强制措施. 通过 workingset refault() 来进行探测, 可以使在较短时间间隔后再次访问的最近收回的文件页直接升级到活动列表. 有了这种机制, 我们可以(在更大的系统上)为活动文件列表提供更多的内存, 这样我们就可以在内存中缓存更多经常使用的文件页, 而不会让它们被流式写入、一旦使用流式文件读取等推出. 这个版本不同时允许在匿名内存使用的相同比率上增加活动列表, 这同样将有助于数据库工作负载, 其中只有一半的页面缓存可用于缓存数据库工作集. | v1 ☑ [4.7-rc1](https://kernelnewbies.org/Linux_4.7#Memory_management) | [PatchWork v1](https://lore.kernel.org/patchwork/cover/664653) | | 2016/06/24 | Johannes Weiner | [mm: fix vm-scalability regression in cgroup-aware workingset code](https://lore.kernel.org/patchwork/cover/692559) | commit 23047a96d7cf ("mm:workingset:per cgroup cache thrash detection") 在缓存逐出 workingset_evictio()、探测 workingset_refault() 和激活 workingset_activation() 路径中添加了 page->mem_cgroup 查找, 并[锁定到激活路径](https://elixir.bootlin.com/linux/v4.6/source/mm/workingset.c#L310), vm可伸缩性测试显示了-23%的回归.
虽然所讨论的测试是一种在实际工作负载中不会发生的人为最坏情况场景(并行读取两个稀疏文件, 只是为了敲碎 LRU 路径), 但仍然可以在这些路径中进行一些优化.
1. 内联查找函数以消除调用.
此外, 在计算激活次数时, 不需要持续 hold page->mem_cgroup, 我们只需要[保持 RCU 锁以防止 memcg 被释放](https://elixir.bootlin.com/linux/v4.8/source/mm/workingset.c#L316). | v1 ☑ 4.8-rc1 | [2016/06/22 PatchWork v1](https://lore.kernel.org/patchwork/cover/691734)
*-*-*-*-*-*-*-*
[2016/06/24 PatchWork rebase](https://lore.kernel.org/patchwork/patch/692559), [commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=55779ec759ccc3c12b917b3712a7716e1140c652) | | 2016/07/08 | Mel Gorman | [Move LRU page reclaim from zones to nodes v9](https://lore.kernel.org/patchwork/cover/696408) | 将 LRU 页面的回收从 ZONE 切换到 NODE. 这里需要将 workingset 从 zone 切换到 node 上. | v9 ☑ [4.8](https://kernelnewbies.org/Linux_4.8#Memory_management) | [PatchWork v9](https://lore.kernel.org/patchwork/cover/696408), [commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1e6b10857f91685c60c341703ece4ae9bb775cf3) | -| 2018/08/28 | Johannes Weiner | [psi: pressure stall information for CPU, memory, and IO v4](https://lore.kernel.org/patchwork/cover/978495) | Refaults 发生在工作集转换和就地抖动期间. 在工作集转换期间, 非活动缓存发生 Refaults 并推出已建立的活动缓存. 但是, 如果活动缓存没有过期, 并且最终会出现 Refaults, 就会造成抖动. 引入一个新的页标志 WORKINGSET_RESTORE, 它在退出时告诉页面在其生命周期内是否处于活动状态. 然后将此位存储在影子条目中, 将故障分类为转换或抖动. | v1 ☑ [4.20-rc1](https://kernelnewbies.org/Linux_4.20#Memory_management) | [PatchWork](https://lore.kernel.org/patchwork/cover/978495), [commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1899ad18c6072d689896badafb81267b0a1092a4) | +| 2018/08/28 | Johannes Weiner | [psi: pressure stall information for CPU, memory, and IO v4](https://lore.kernel.org/patchwork/cover/978495) | Refaults 发生在工作集转换和就地抖动期间. 在工作集转换期间, 非活动缓存发生 Refaults 并推出已建立的活动缓存. 但是, 如果活动缓存没有过期, 并且最终会出现 Refaults, 就会造成抖动. 引入一个新的页标志 WORKINGSET_RESTORE, 它在退出时告诉页面在其生命周期内是否处于活动状态. 然后将此位存储在影子条目中, 将故障分类为转换或抖动. | v1 ☑ [4.20-rc1](https://kernelnewbies.org/Linux_4.20#Memory_management) | [PatchWork](https://lore.kernel.org/patchwork/cover/978495), [关注 commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1899ad18c6072d689896badafb81267b0a1092a4) | | 2018/10/09 | Johannes Weiner | [mm: workingset & shrinker fixes](https://lore.kernel.org/patchwork/cover/997829) | 通过为循环中的影子节点添加一个计数器, 可以更容易地捕获影子节点收缩器中的 bug. | v1 ☑ [4.20-rc1](https://kernelnewbies.org/Linux_4.20#Memory_management) | [PatchWork](https://lore.kernel.org/patchwork/cover/997829), [commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=68d48e6a2df575b935edd420396c3cb8b6aa6ad3) | | 2019/07/01 | Johannes Weiner | [mm: vmscan: scan anonymous pages on file refaults](https://lore.kernel.org/patchwork/cover/1095936) | NA | v1 ☑ 5.3-rc1 | [PatchWork v1](https://lore.kernel.org/patchwork/cover/1090850)
*-*-*-*-*-*-*-*
[PatchWork v2](https://lore.kernel.org/patchwork/cover/1095351)
*-*-*-*-*-*-*-*
[PatchWork](https://lore.kernel.org/patchwork/cover/1095936), [commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2c012a4ad1a2cd3fb5a0f9307b9d219f84eda1fa) | | 2019/10/22 | Johannes Weiner | [mm/vmscan: cgroup-related cleanups](https://lore.kernel.org/patchwork/cover/1142997) | 这里的 8 个补丁, 清理回收代码与cgroups的交互. 它们不应该改变任何行为, 只是让实现更容易理解和使用. | v1 ☑ 5.5-rc1 | [PatchWork 0/8](https://lore.kernel.org/patchwork/cover/1142997) | diff --git a/study/kernel/00-DESCRIPTION/SCHEDULER.md b/study/kernel/00-DESCRIPTION/SCHEDULER.md index 9aff202..51eda41 100644 --- a/study/kernel/00-DESCRIPTION/SCHEDULER.md +++ b/study/kernel/00-DESCRIPTION/SCHEDULER.md @@ -78,9 +78,27 @@ Mainline Merge Window | 5.13 | 2021/06/28 | [Merge tag 'sched-core-2021-04-28', 5.13-rc1](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=16b3d0cf5bad844daaf436ad2e9061de0fe36e5c)
[Merge tag 'sched-urgent-2021-05-09', 5.13-rc1](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9819f682e48c6a7055c5d7a6746411dd3969b0e5)
[Merge tag 'sched-urgent-2021-05-15', 5.13-rc2](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c12a29ed9094b4b9cde8965c12850460b9a79d7c)
[Merge tag 'sched-urgent-2021-06-12', 5.13-rc6](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=99f925947ab0fd5c17b74460d8b32f1aa1c86e3a)
[Merge tag 'sched_urgent_for_v5.13_rc6', 5.13-rc7, 2021/06/20](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=cba5e97280f53ec7feb656fcdf0ec00a5c6dd539)
[Merge tag 'sched-urgent-2021-06-24', 5.13](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=666751701b6e4b6b6ebc82186434806fa8a09cf3)
| | 5.14 | NA | [Merge tag 'sched-core-2021-06-28'](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=54a728dc5e4feb0a9278ad62b19f34ad21ed0ee4) | +## 0.4 社区会议 +------- -## 0.4 社区几个调度的大神 +### 0.4.1 Linux Plumbers Conference +------- + +Scheduler Microconference Accepted into Linux Plumbers Conference + +`2010~2017` 年的内容, 可以在 [wiki](http://wiki.linuxplumbersconf.org/?do=search&id=scheduler) 检索. + +| 日期 | 官网 | LKML | LWN | +|:---:|:----:|:----:|:---:| +| [2021](https://linuxplumbersconf.org/event/11/page/104-accepted-microconferences) | [LPC 2021 - Overview](https://www.linuxplumbersconf.org/event/11/page/103-lpc-2021-overview) | [[ANNOUNCE] CFP: Scheduler Microconference - Linux Plumbers Conference 2021](https://lkml.org/lkml/2021/8/9/718) | [A look forward to Linux Plumbers 2021](https://lwn.net/Articles/864016) | +| [2020](https://linuxplumbersconf.org/event/7/page/44-microconferences) | [Scheduler Microconference 2020](https://www.linuxplumbersconf.org/blog/2020/scheduler-microconference-accepted-into-2020-linux-plumbers-conference) | NA | NA | +| [2019](https://linuxplumbersconf.org/event/4/page/38-microconferences) | [Scheduler Microconference 2019](https://www.linuxplumbersconf.org/blog/2019/scheduler-microconference-accepted-into-2019-linux-plumbers-conference) | NA | NA | +| [2018](https://www.linuxplumbersconf.org/2018) | [Power Management and Energy-awareness Microconference 2018](https://blog.linuxplumbersconf.org/2018/power-management-mc)
[timeline](https://www.linuxplumbersconf.org/event/2/timetable/?view=lpc) | NA | [Notes from the LPC scheduler microconference](https://lwn.net/Articles/734039) | +| [2017](https://blog.linuxplumbersconf.org/2017) | [Scheduler Workloads Microconference 2017](http://wiki.linuxplumbersconf.org/2017:scheduler_workloads) | | [Notes from the LPC scheduler microconference](https://lwn.net/Articles/734039) + + +## 0.5 社区几个调度的大神 ------- - [x] [Mel Gorman mgorman@techsingularity.net](https://lore.kernel.org/patchwork/project/lkml/list/?submitter=19167) @@ -93,7 +111,7 @@ Mainline Merge Window -## 0.5 目录 +## 0.6 目录 ------- - [x] 1. 进程调度类