This workshop should teach you how you can create your own modules.
The plan is to utilize existing guides and just link to them.
Before we start, feel free to look into Stephen Gallagher's blog posts on module packaging:
- Sausage Factory: An introduction to building modules in Fedora
- Sausage Factory: Advanced module building in Fedora
If you plan to attend the workshop, please follow the instructions bellow so you'll have all the required content ready on your laptop for the workshop.
Install the build tools:
$ sudo dnf install module-build-service mock-scm
Clone this repository...
$ git clone https://github.com/fedora-modularity/workshop
...and switch to test-module/ directory, initiate a new git repository and
build the test module so all the build dependencies are cached for the workshop
(may be up to 11G (remove bootstrap: master line from test-module.yaml if
you want to down it to just ~1G):
$ cd workshop/test-module
$ git init . && git add . && git commit -m init
$ mbs-build local
- 1-minute-intro to modularity
- modulemd
- Workflow
mbs-build- Let's create autotools module!
- Testing
- Feedback
TODO:
- everyone can see my terminal (font and colors are acceptable)
- don't use black background
Modularity cuts Linux distributions into modules, giving them independent lifecycles, and making them available in multiple streams. Giving more options of choice to users, and flexibility and control to packagers.
See the official Fedora Modularity documentation for more information
Modules are defined in a modulemd file.
As opposed to traditional distributions, where all packages are built and shipped in a single monolythic release, Modularity needs a mechanism of defining which package goes into what module. And that's modulemd.
Have a look at a real example of nodejs module.
All modules will use a platform module as a build and runtime dependency. Platform is the base system and common, shared userland.
To create a module, you need to define a top-level package set representing your module, and resolve missing dependencies between your package set and the platform module. A simple example:
Unfortunatelly, it's not easy every time. Many packages will have complex dependencies:
As you might have guessed, bundling all dependencies is not the right thing to do in this example. Instead, we need to identify other modules, and use these as dependencies:
At the beginning of development of the F27 Server, all of these modules will need to get indentified and built. However, when we are done with the initial set, packagers will be able to use what's already available.
As we saw above, there is a need for an initial coordination while idetifying and defining the initial set of modules. How to handle that?
The dependency-report repository along with the modularity-modules space is the answer.
The Server WG requires to have at least FreeIPA, PostrgreSQL, NetworkManager, Cockpit, storaged.
This is done in the modularity-modules space.
Adam is running that periodically - output is saved in the repo. Everyone can run on their own.
An example: freeipa - do you see something that could be a new module? Hint: Python 2, Python 3, Java, ...
Keep identifying new modules, defining their top-lvl components, resolving dependencies... When it feels like we have a solid base, we can start generating modulemd files and building them.
$ sudo dnf install -y modtools
$ mod-tools rpm2module nodejs npm
It will get us this modulemd:
data:
api:
rpms: [nodejs, npm]
components:
rpms:
http-parser: {rationale: Build and runtime dependency.}
nodejs: {buildorder: 10, rationale: Runtime dependency.}
dependencies:
buildrequires: {base-runtime: f26, shared-userspace: f26}
requires: {base-runtime: f26, shared-userspace: f26}
description: ''
license:
module: [MIT]
summary: ''
document: modulemd
version: 1
$ ./depchase -a x86_64 -c ./repos.cfg.sample resolve nodejs npm
http-parser-2.7.1-5.fc26.x86_64
compat-openssl10-1:1.0.2j-6.fc26.x86_64
pcre-8.41-1.fc27.x86_64
p11-kit-trust-0.23.7-1.fc27.x86_64
sed-4.4-1.fc26.x86_64
libicu-57.1-4.fc26.x86_64
libtasn1-4.12-1.fc27.x86_64
grep-3.1-1.fc27.x86_64
ca-certificates-2017.2.16-2.fc27.noarch
libuv-1:1.13.1-1.fc27.x86_64
filesystem-3.3-1.fc27.x86_64
libacl-2.2.52-16.fc27.x86_64
tzdata-2017b-1.fc27.noarch
libatomic_ops-7.4.6-1.fc27.x86_64
fedora-release-27-0.2.noarch
make-1:4.2.1-2.fc26.x86_64
libattr-2.4.47-19.fc27.x86_64
fedora-repos-27-0.2.noarch
fedora-repos-rawhide-27-0.2.noarch
npm-1:5.3.0-1.8.2.1.1.fc27.x86_64
coreutils-8.27-13.fc27.x86_64
...
Documentation how to do:
What will be the name of our module? What streams will we provide?
Good name for stream (dist-git branch) is a version of the software. Let's have a look at latest versions of packages we want to put into the module:
autoconf-2.69-25automake-1.15.1libtool-2.4.6
Versions don't seem to be in sync.
What should be in the module? automake, autoconf and libtool, right?
document: modulemd
version: 1
data:
summary: GNU toolchain (autoconf, automake, libtool)
description: >
This module is composed of well-known GNU tools, such as autoconf,
automake and libtool.
license:
module:
- MIT
content: []
references:
community: https://github.com/modularity-modules/autotools
documentation: https://github.com/modularity-modules/autotools
tracker: https://github.com/modularity-modules/autotools/issues
components:
rpms:
autoconf:
rationale: Primary component of this module.
ref: master
automake:
rationale: Primary component of this module.
ref: master
libtool:
rationale: Primary component of this module.
ref: masterWhat's missing?
Dependencies, right!
document: modulemd
version: 1
data:
summary: GNU toolchain (autoconf, automake, libtool)
description: >
This module is composed of well-known GNU tools, such as autoconf,
automake and libtool.
license:
module:
- MIT
content: []
dependencies:
buildrequires:
platform: master
host: master
requires:
platform: master
host: master
references:
community: https://github.com/modularity-modules/autotools
documentation: https://github.com/modularity-modules/autotools
tracker: https://github.com/modularity-modules/autotools/issues
components:
rpms:
autoconf:
rationale: Primary component of this module.
ref: master
automake:
rationale: Primary component of this module.
ref: master
libtool:
rationale: Primary component of this module.
ref: masterSo let's build:
$ mkdir autotools
$ cd autotools
$ vim autotools.yaml
<shift-insert>
$ git init . && git add . && git commit -m init
$ mbs-build local
Seems like the build fails with:
# /usr/bin/dnf builddep --installroot /var/lib/mock/module-autotools-master-20170824142446-Thread-25/root/ /var/lib/mock/module-autotools-master-20170824142446-Thread-25/root//builddir/build/SRPMS/libtool-2.4.6-20.module_2503b24e.src.rpm
Last metadata expiration check: 0:00:00 ago on Thu 24 Aug 2017 04:29:40 PM CEST.
No matching package to install: 'autoconf'
No matching package to install: 'automake'
No matching package to install: 'help2man'
Package libstdc++-devel-7.1.1-7.module_6faa4f4e.1.x86_64 is already installed, skipping.
Not all dependencies satisfied
Error: Some packages could not be found.
Where do we get those dependencies?
document: modulemd
version: 1
data:
summary: GNU toolchain (autoconf, automake, libtool)
description: >
This module is composed of well-known GNU tools, such as autoconf,
automake and libtool.
license:
module:
- MIT
content: []
dependencies:
buildrequires:
bootstrap: master
requires:
platform: master
host: master
references:
community: https://github.com/modularity-modules/autotools
documentation: https://github.com/modularity-modules/autotools
tracker: https://github.com/modularity-modules/autotools/issues
components:
rpms:
autoconf:
rationale: Primary component of this module.
ref: master
automake:
rationale: Primary component of this module.
ref: master
libtool:
rationale: Primary component of this module.
ref: master(running mbs-build local here for the first time downloads all 15000 packages of bootstrap module)
Does the build pass?
Finish: rpmbuild automake-1.15.1-2.module_ac4d13ab.src.rpm
Finish: build phase for automake-1.15.1-2.module_ac4d13ab.src.rpm
INFO: Done(/home/tt/modulebuild/builds/module-autotools-master-20170824143320/results/Thread-26/automake-1.15.1-2.module_ac4d13ab.src.rpm) Config(mock-Thread-26) 4 minutes 48 seconds
INFO: Results and/or logs in: /home/tt/modulebuild/builds/module-autotools-master-20170824143320/results/Thread-26
INFO: Cleaning up build root ('cleanup_on_success=True')
Start: clean chroot
DEBUG: kill orphans
DEBUG: child environment: None
DEBUG: Executing command: ['/bin/umount', '-n', '/var/lib/mock/module-autotools-master-20170824143320-Thread-26/root/var/cache/dnf/'] with env {'TERM': 'vt100', 'SHELL': '/bin/sh', 'HOME': '/builddir', 'HOSTNAME': 'mock', 'PATH': '/usr/bin:/bin:/usr/sbin:/sbin', 'LANG': 'en_US.utf8'} and shell False
DEBUG: Child return code was: 0
Indeed.
Can we install the module?
Ehm, I don't know.
So, what binary RPMs do we need?
How do we figure out in which modules these are available? And are they actually in some module?
As described in the official review process, you need to create a new bug to initiate the review process.
We suggest to go to Freenode IRC channel #fedora-modularity and find a reviewer.
Modularity testing framework Meta test family
You can find F&Q and Feedback document over here. Feel free to ask your question and someone will answer it soon or later.
You can also open an issue at any upstream repository linked in the workshop.
Alternatively, you may ask your question on webchat.freenode.net and someone should answer it right away.