mirror of
https://github.com/gatieme/LDD-LinuxDeviceDrivers.git
synced 2026-08-19 01:22:36 +08:00
description/scheduler: scheduler eBPF vs ghOSt
This commit is contained in:
Cheng Jian
@@ -1296,17 +1296,30 @@ CFS 调度器为用户和开发人员提供了非常多的调试接口和参数
|
||||
|
||||
BPF 钩子(它已经成功地用于各种内核子系统)为外部代码(安全地)更改一些内核决策提供了一种方法, BPF 工具使这变得非常容易, 部署 BPF 脚本的开发者已经非常习惯于为新的内核版本更新它们.
|
||||
|
||||
XXX 发起了 BPF 对调度器的潜在应用的讨论, 它提交的 patchset 旨在为调度器提供一些非常基本的 BPF 基础设施, 以便向调度器添加新的 BPF钩子、一组最小的有用助手以及相应的 libbpf 更改等等. 他们在 CFS 中使用 BPF 的第一次实验看起来非常有希望. 虽然还处于非常早期的阶段, 但在 Facebook 的主网页工作量已经获得了不错的延迟和约 1% 的 RPS.
|
||||
|
||||
这与谷歌的 ghOSt 非常类似, 但是 ghOSt 比 BPF 的方式要激进很多, ghOSt 的目标是将调度代码转移到用户空间. 它们的核心动机似乎有些相似:使调度器更改更容易开发、验证和部署. 尽管他们的方法不同, 他们也使用 BPF 来加速一些热点路径. 但是作者认为使用 BPF 的方式也可以达到他们的目的.
|
||||
* Facebook 的尝试
|
||||
|
||||
Roman Gushchin 发起了 BPF 对调度器的潜在应用的讨论, 它提交的 patchset 旨在为调度器提供一些非常基本的 BPF 基础设施, 以便向调度器添加新的 BPF钩子、一组最小的有用助手以及相应的 libbpf 更改等等. 他们在 CFS 中使用 BPF 的第一次实验看起来非常有希望. 虽然还处于非常早期的阶段, 但在 Facebook 的主网页工作量已经获得了不错的延迟和约 1% 的 RPS.
|
||||
|
||||
作者提供了一个用户空间部分的示例 [github/rgushchin/atc](https://github.com/rgushchin/atc), 它加载了一些简单的钩子. 它非常简单, 只是为了简化使用所提供的内核补丁.
|
||||
|
||||
* Google 的尝试
|
||||
|
||||
于此同时, google 团队的 Hao Luo 和 Barret Rhoden 等也在 eBPF 在 CPU Scheduler 领域的应用进行了探索, 并在 LPC-2021 上做了分享. 当前的工作集中在几个方向:
|
||||
|
||||
1. 调度延迟分析以及依据 profiling 来指导调度
|
||||
|
||||
2. 核心调度中的强制空闲时间计算
|
||||
|
||||
3. 使用 BPF 加速 ghOSt 内核调度器
|
||||
|
||||
这与谷歌的 ghOSt 非常类似, 但是 ghOSt 比 BPF 的方式要激进很多, ghOSt 的目标是将调度代码转移到用户空间. 它们的核心动机似乎有些相似:使调度器更改更容易开发、验证和部署. 尽管他们的方法不同, 他们也使用 BPF 来加速一些热点路径. 但是作者认为使用 BPF 的方式也可以达到他们的目的. , 参见 [eBPF in CPU Scheduler](https://linuxplumbersconf.org/event/11/contributions/954/attachments/776/1463/eBPF%20in%20CPU%20Scheduler.pdf)
|
||||
|
||||
|
||||
|
||||
| 时间 | 作者 | 特性 | 描述 | 是否合入主线 | 链接 |
|
||||
|:----:|:----:|:---:|:----:|:---------:|:----:|
|
||||
| 2021/09/15 | Roman Gushchin <guro@fb.com> | [Scheduler BPF](https://www.phoronix.com/scan.php?page=news_item&px=Orange-BPF-Memory-Cache-BMC) | NA | RFC ☐ | [PatchWork rfc,0/6](https://lore.kernel.org/bpf/CA+khW7i460ey-UFzpMSJ8AP9QeD8ufa4FzLA4PQckNP00ShQSw@mail.gmail.com)<br>*-*-*-*-*-*-*-* <br>[LPC 2021](https://linuxplumbersconf.org/event/11/contributions/954)<br>*-*-*-*-*-*-*-* <br>[LKML](https://lkml.org/lkml/2021/9/16/1049), [LWN](https://lwn.net/Articles/869433) |
|
||||
| 2021/09/15 | Roman Gushchin <guro@fb.com> | [Scheduler BPF](https://www.phoronix.com/scan.php?page=news_item&px=Linux-BPF-Scheduler) | NA | RFC ☐ | [PatchWork rfc,0/6](https://patchwork.kernel.org/project/netdevbpf/cover/20210916162451.709260-1-guro@fb.com)<br>*-*-*-*-*-*-*-* <br>[LPC 2021](https://linuxplumbersconf.org/event/11/contributions/954)<br>*-*-*-*-*-*-*-* <br>[LKML](https://lkml.org/lkml/2021/9/16/1049), [LWN](https://lwn.net/Articles/869433) |
|
||||
|
||||
|
||||
## 9.4 其他
|
||||
|
||||
@@ -71,7 +71,7 @@ blogexcerpt: 虚拟化 & KVM 子系统
|
||||
| 2019/01/23 | Kees Cook <keescook@chromium.org> | [gcc-plugins: Introduce stackinit plugin](https://patchwork.kernel.org/project/linux-kbuild/cover/20190123110349.35882-1-keescook@chromium.org) | 使用 GCC 编译内核时, 提供插件来实现早期提议的 gcc 选项 [`-finit-local-vars`](https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html)(未被 GNU/gcc 社区接纳)类似的功能, 以实现[未初始化局部变量初始化为 0 的诉求](https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html). 通过 CONFIG_GCC_PLUGIN_STACKINIT 来开启. | v1 ☐ | [PatchWork](https://patchwork.kernel.org/project/linux-kbuild/cover/20190123110349.35882-1-keescook@chromium.org) |
|
||||
| 2019/04/10 | Kees Cook <keescook@chromium.org> | [security: Implement Clang's stack initialization](https://patchwork.kernel.org/project/linux-kbuild/cover/20190423194925.32151-1-keescook@chromium.org) | clang 中提供了 `-ftrivial-auto-var-init` 来进行未初始化局部变量的默认初始化, 引入 CONFIG_INIT_STACK_ALL 来为内核开启此选项, 这比 CONFIG_GCC_PLUGINS_STRUCTLEAK_BYREF_ALL 的覆盖更广. 当启用 CONFIG_INIT_STACK_ALL 时, 当前补丁仅使用 "pattern" 模式. 开发人员可以通过使用 `__attribute__((uninitialized))` 在每个变量的基础上选择不使用该特性. | v1 ☑ 5.2-rc1 | [PatchWork v1,3/3](https://patchwork.kernel.org/project/linux-kbuild/patch/20190410161612.18545-4-keescook@chromium.org)<br>*-*-*-*-*-*-*-* <br>[PatchWork v2,3/3](https://patchwork.kernel.org/project/linux-kbuild/patch/20190411180117.27704-4-keescook@chromium.org)<br>*-*-*-*-*-*-*-* <br>[PatchWork v3,3/3](https://patchwork.kernel.org/project/linux-kbuild/patch/20190423194925.32151-4-keescook@chromium.org), [commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=709a972efb01efaeb97cad1adc87fe400119c8ab) |
|
||||
| 2020/06/14 | Alexander Potapenko <glider@google.com> | [security: allow using Clang's zero initialization for stack variables](https://lore.kernel.org/patchwork/cover/1255765) | 支持 clang 的局部变量零初始化. 通过 CONFIG_INIT_STACK_ALL_ZERO 来启用, clang 可以通过选项 `-ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang` 来保证未初始化局部变量初始化为 0. | RFC v2 ☑ 5.9-rc1 | [PatchWork v2,RFC](https://lore.kernel.org/patchwork/cover/1255765)<br>*-*-*-*-*-*-*-* <br>[PatchWork RFC](https://lore.kernel.org/patchwork/patch/1256566), [commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f0fe00d4972a8cd4b98cc2c29758615e4d51cdfe) |
|
||||
| 2021/02/18 | NA | [security: allow using Clang's zero initialization for stack variables](https://lore.kernel.org/patchwork/cover/1255765) | gcc 也引入了 [Auto Initialize Automatic Variables](https://www.phoronix.com/scan.php?page=news_item&px=GCC-12-Auto-Var-Init) 通过 `-ftrivial-auto-var-init` 选项将未初始化的[变量默认初始化为 0](https://gcc.gnu.org/pipermail/gcc-patches/2021-February/565514.html). | RFC v2 ☑ 5.9-rc1 | NA |
|
||||
| 2021/02/18 | NA | [gcc Stack variable initialization-无需合入补丁, GCC 支持即可](https://lwn.net/Articles/870045) | gcc 也引入了 [Auto Initialize Automatic Variables](https://www.phoronix.com/scan.php?page=news_item&px=GCC-12-Auto-Var-Init) 通过 `-ftrivial-auto-var-init` 选项将未初始化的[变量默认初始化为 0](https://gcc.gnu.org/pipermail/gcc-patches/2021-February/565514.html). | RFC v2 ☑ 5.9-rc1 | NA |
|
||||
|
||||
|
||||
<br>
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
ifneq ($(KERNELRELEASE),)
|
||||
|
||||
obj-m := list_vma.o
|
||||
|
||||
else
|
||||
|
||||
KERNELDIR ?= /lib/modules/$(shell uname -r)/build
|
||||
|
||||
PWD := $(shell pwd)
|
||||
|
||||
all:
|
||||
make -C $(KERNELDIR) M=$(PWD) modules
|
||||
|
||||
clean:
|
||||
make -C $(KERNELDIR) M=$(PWD) clean
|
||||
|
||||
|
||||
endif
|
||||
|
||||
|
||||
|
||||
insmod:
|
||||
sudo insmod ./kv2p.ko
|
||||
|
||||
rmmod:
|
||||
sudo rmmod kv2p
|
||||
|
||||
reinsmod:
|
||||
make rmmod
|
||||
make insmod
|
||||
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
| 日期 | 内核版本 | 架构| 作者 | GitHub| CSDN |
|
||||
| ------------- |:-------------:|:-------------:|:-------------:|:-------------:|:-------------:|
|
||||
| 2016-07-21 | [Linux-4.7](http://lxr.free-electrons.com/source/?v=4.7) | X86 & arm | [gatieme](http://blog.csdn.net/gatieme) | [LinuxDeviceDrivers](https://github.com/gatieme/LDD-LinuxDeviceDrivers) | [Linux进程管理与调度](http://blog.csdn.net/gatieme/article/category/6393814) |
|
||||
|
||||
|
||||
|
||||
#1 内核虚拟地址与物理地址的转换
|
||||
-------
|
||||
|
||||
| 函数 | 描述 | 底层函数 |
|
||||
|:---:|:---:|:--------:|
|
||||
| phys_addr_t virt_to_phys(volatile void *address) | 将内核的虚拟地址转换为物理地址 | \__pa或者 \__virt_to_phys |
|
||||
| void *phys_to_virt(phys_addr_t x) | 将物理地址转换为虚拟内核 | \__va或者\__phys_to_virt |
|
||||
|
||||
其中phy_addr_t用以表示一个内存地址, 定义在[include/linux/types.h](http://lxr.free-electrons.com/source/include/linux/types.h#L161)中
|
||||
|
||||
```c
|
||||
#ifdef CONFIG_PHYS_ADDR_T_64BIT
|
||||
typedef u64 phys_addr_t;
|
||||
#else
|
||||
typedef u32 phys_addr_t;
|
||||
#endif
|
||||
```
|
||||
|
||||
可以通过搜索我们内核的.config文件搜索CONFIG_PHYS_ADDR_T_64BIT这个配置参数
|
||||
|
||||
```c
|
||||
cat /usr/src/linux-headers-4.2.0-36-generic/.config | grep CONFIG_PHYS_ADDR_T_64BIT
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
#2 virt_to_phys/phys_to_virt实现
|
||||
-------
|
||||
|
||||
##2.1 不同架构的实现
|
||||
-------
|
||||
|
||||
|
||||
| 架构 | 定义 |
|
||||
|:-------:|:-------:|
|
||||
| 标准 | [include/asm-generic/io.h, line 724](http://lxr.free-electrons.com/source/include/asm-generic/io.h?v4.7#L724)
|
||||
| arm | [arch/arm/include/asm/memory.h, line 272](http://lxr.free-electrons.com/source/arch/arm/include/asm/memory.h?v4.7#L272) |
|
||||
| arm64 | [arch/arm64/include/asm/memory.h](http://lxr.free-electrons.com/source/arch/arm64/include/asm/memory.h?v4.7#L189) |
|
||||
| x86 |[arch/x86/include/asm/io.h, line 118](http://lxr.free-electrons.com/source/arch/x86/include/asm/io.h?v4.7#L118)
|
||||
|
||||
|
||||
|
||||
* virt_to_phys函数在内核中是通过\__pa或者\__virt_to_phys函数实现的
|
||||
|
||||
* phys_to_virt函数在内核中是通过\__va或者\__phys_to_virt函数实现的
|
||||
|
||||
|
||||
|
||||
|
||||
##2.2 arm64上的实现
|
||||
|
||||
不同架构的实现有细微差别, 但是其基本思路是一致的
|
||||
|
||||
|
||||
下面我们就以arm64上的实现为例进行讲解, 定义在[arch/arm64/include/asm/memory.h](http://lxr.free-electrons.com/source/arch/arm64/include/asm/memory.h?v4.7#L184)
|
||||
|
||||
```c
|
||||
/*
|
||||
* These are *only* valid on the kernel direct mapped RAM memory.
|
||||
* Note: Drivers should NOT use these. They are the wrong
|
||||
* translation for translating DMA addresses. Use the driver
|
||||
* DMA support - see dma-mapping.h.
|
||||
*/
|
||||
#define virt_to_phys virt_to_phys
|
||||
static inline phys_addr_t virt_to_phys(const volatile void *x)
|
||||
{
|
||||
return __virt_to_phys((unsigned long)(x));
|
||||
}
|
||||
|
||||
#define phys_to_virt phys_to_virt
|
||||
static inline void *phys_to_virt(phys_addr_t x)
|
||||
{
|
||||
return (void *)__phys_to_virt(x);
|
||||
}
|
||||
|
||||
/*
|
||||
* Drivers should NOT use these either.
|
||||
*/
|
||||
#define __pa(x) __virt_to_phys((unsigned long)(x))
|
||||
#define __va(x) ((void *)__phys_to_virt((phys_addr_t)(x)))
|
||||
#define pfn_to_kaddr(pfn) __va((phys_addr_t)(pfn) << PAGE_SHIFT)
|
||||
```
|
||||
|
||||
|
||||
可以看到真是的实现是__virt_to_phys和__phys_to_virt函数实现的
|
||||
|
||||
|
||||
|
||||
##2.3 __virt_to_phys函数
|
||||
-------
|
||||
|
||||
__virt_to_phys函数定义在[arch/arm64/include/asm/memory.h, line 109](http://lxr.free-electrons.com/source/arch/arm64/include/asm/memory.h?v=4.7#L109)
|
||||
|
||||
```c
|
||||
/*
|
||||
* Physical vs virtual RAM address space conversion. These are
|
||||
* private definitions which should NOT be used outside memory.h
|
||||
* files. Use virt_to_phys/phys_to_virt/__pa/__va instead.
|
||||
*/
|
||||
#define __virt_to_phys(x) ({ \
|
||||
phys_addr_t __x = (phys_addr_t)(x); \
|
||||
__x & BIT(VA_BITS - 1) ? (__x & ~PAGE_OFFSET) + PHYS_OFFSET : \
|
||||
(__x - kimage_voffset); })
|
||||
|
||||
```
|
||||
|
||||
其中\__x中间变量是内核的一点小技巧, 通过宏实现的函数没有检查, 比如传入一个x++
|
||||
|
||||
其主要操作就是
|
||||
|
||||
```c
|
||||
__x & BIT(VA_BITS - 1) ? (__x & ~PAGE_OFFSET) + PHYS_OFFSET : (__x - kimage_voffset); })
|
||||
```
|
||||
|
||||
##2.4 __phys_to_virt函数
|
||||
-------
|
||||
|
||||
__phys_to_virt函数定义在[arch/arm64/include/asm/memory.h, line 114](http://lxr.free-electrons.com/source/arch/arm64/include/asm/memory.h?v=4.7#L114)
|
||||
|
||||
|
||||
```c
|
||||
#define __phys_to_virt(x) ((unsigned long)((x) - PHYS_OFFSET) | PAGE_OFFSET)
|
||||
```
|
||||
|
||||
##2.5 __phys_to_kimg函数
|
||||
-------
|
||||
|
||||
```c
|
||||
#define __phys_to_kimg(x) ((unsigned long)((x) + kimage_voffset))
|
||||
```
|
||||
|
||||
##2.6 page_to_phys和phys_to_page函数
|
||||
-------
|
||||
|
||||
```c
|
||||
/*
|
||||
* Convert a page to/from a physical address
|
||||
*/
|
||||
#define page_to_phys(page) (__pfn_to_phys(page_to_pfn(page)))
|
||||
#define phys_to_page(phys) (pfn_to_page(__phys_to_pfn(phys)))
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
/*****************************************************************
|
||||
文件名：mem.c
|
||||
输入参数：
|
||||
pid 接收待查询进程的PID
|
||||
va 接收待查询的虚拟地址
|
||||
*****************************************************************/
|
||||
|
||||
|
||||
#include <linux/module.h>
|
||||
#include <linux/kernel.h>
|
||||
#include <linux/init.h>
|
||||
#include <linux/sched.h>
|
||||
#include <linux/mm.h>
|
||||
#include <asm/pgtable.h>
|
||||
#include <asm/page.h>
|
||||
|
||||
MODULE_LICENSE("GPL");
|
||||
|
||||
|
||||
static unsigned long VA;
|
||||
module_param(VA, ulong, 0644);
|
||||
|
||||
|
||||
|
||||
static int __init kv2p_init(void)
|
||||
{
|
||||
int value = 10;
|
||||
// use virt_to_phys
|
||||
// http://lxr.free-electrons.com/source/arch/arm64/include/asm/memory.h?v4.7#L189
|
||||
|
||||
phys_addr_t phyaddr = virt_to_phys((const volatile void *)&value);
|
||||
printk("0x%lx\n", (unsigned long )phyaddr);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
|
||||
static void __exit kv2p_exit(void)
|
||||
{
|
||||
printk(KERN_INFO"Goodbye!\n");
|
||||
}
|
||||
|
||||
module_init(kv2p_init);
|
||||
module_exit(kv2p_exit);
|
||||
Reference in New Issue
Block a user