m/node: build Linux with modules
This introduces modules into our Linux build. I originally didn't want
to do this, this is why this wasn't done until now. But various things
in the kernel weren't set up for this, for example the AMD and Intel KVM
modules cannot both be loaded, only the first one loaded works. Also,
the Linux kernel cannot load firmware for built-in modules reliably as
the filesystem it tries to load it from is not always mounted first,
even if the kernel itself mounts it.
The firmware issue was brought up multiple times on LKML, but Linus is
of the opinion that the firmware should be next to the kernel module,
thus either built-in (not viable for licensing and size reasons) or the
modules need to be loadable and on the same filesystem as the firmware.
Thus unless we want to carry signifcant patches against the Kernel in a
deadlock-prone area, we are forced to adopt a design with loadable
modules (or ship everything twice in an initramfs which is also not
desirable).
The kernel config currently only has the modules as non-builtin which
require firmware, everything else has been left as-is. For boot-time
performance it would eventually be a good idea to move to a setup with
more modules once we're confident in the implementation and everything
can deal with late-loaded modules/devices.
As a drive-by fix this also moves the kernel builds to out-of-tree so
that we no longer pollute the source folder. Bazel protected us from
serious issues due to this, but it's still bad practice.
Change-Id: Iced8e12234565e5b7447e732716651e05e67d55b
Reviewed-on: https://review.monogon.dev/c/monogon/+/1791
Reviewed-by: Serge Bazanski <serge@monogon.tech>
Tested-by: Jenkins CI
diff --git a/third_party/linux/def.bzl b/third_party/linux/def.bzl
index b31b34b..2090ed9 100644
--- a/third_party/linux/def.bzl
+++ b/third_party/linux/def.bzl
@@ -73,8 +73,8 @@
# (image_name) is the name of the image that will be generated by this
# rule.
(target, image_source, image_name) = {
- 'vmlinux': ('vmlinux', 'vmlinux', 'vmlinux'),
- 'bzImage': ('all', 'arch/x86/boot/bzImage', 'bzImage'),
+ 'vmlinux': ('vmlinux modules', 'vmlinux', 'vmlinux'),
+ 'bzImage': ('all modules', 'arch/x86/boot/bzImage', 'bzImage'),
}[image_format]
# Root of the given Linux sources.
@@ -82,8 +82,9 @@
image = ctx.actions.declare_file(image_name)
modinfo = ctx.actions.declare_file("modules.builtin.modinfo")
+ modules = ctx.actions.declare_directory("modules")
ctx.actions.run_shell(
- outputs = [ image, modinfo ],
+ outputs = [ image, modinfo, modules ],
inputs = [ kernel_config ] + kernel_src,
command = '''
kconfig=$1
@@ -92,12 +93,20 @@
image=$4
root=$5
modinfo=$6
+ modules=$7
+
+ builddir=$(mktemp -d)
mkdir ${root}/.bin
- cp ${kconfig} ${root}/.config
- (cd ${root} && make -j $(nproc) ${target} >/dev/null)
- cp ${root}/${image_source} ${image}
- cp ${root}/modules.builtin.modinfo ${modinfo}
+ cp ${kconfig} ${builddir}/.config
+ (cd ${root} && KBUILD_OUTPUT="${builddir}" make -j $(nproc) ${target} >/dev/null)
+ cp "${builddir}"/${image_source} ${image}
+ cp "${builddir}"/modules.builtin.modinfo ${modinfo}
+ # Not using modules_install as it tries to run depmod and friends
+ for f in $(find "${builddir}" -name '*.ko' -type f -printf "%P\n" ); do
+ install -D "${builddir}/$f" "${modules}/$f"
+ done
+ rm -Rf "$builddir"
''',
arguments = [
kernel_config.path,
@@ -105,7 +114,8 @@
image_source,
image.path,
root,
- modinfo.path
+ modinfo.path,
+ modules.path,
],
use_default_shell_env = True,
)
@@ -116,7 +126,8 @@
runfiles=ctx.runfiles(files=[image])
),
OutputGroupInfo(
- modinfo = depset([modinfo])
+ modinfo = depset([modinfo]),
+ modules = depset([modules])
)
]