Add pack option to the builder options for cloud native buildpacks #916

nickhammond · 2024-08-28T05:48:15Z

This PR introduces Cloud Native Buildpacks to the list of builder options for Kamal.

This opens up the option to utilize buildpacks instead of writing a Dockerfile from scratch for each app that you want to deploy with Kamal. The end result is still an OCI-compliant docker image that you can run the same as the currently built docker images with Kamal.

You can also use any buildpacks or builders that you'd like so if you prefer some of the Paketo buildpacks instead you can use those too. The example below is utilizing Heroku's builder with the ruby and procfile buildpack which gives you the familiar Heroku build process when you deploy your application. Auto-detection of bundler, cache management of gem and asset installation, and various other features that come along with those.

With this PR you'd need to have pack installed as well as Docker and then within your deploy.yml change your builder to:

builder:
  arch: amd64
  pack:
    builder: heroku/builder:24
    buildpacks:
    - heroku/ruby
    - heroku/procfile

The default process that the buildpack tries to boot is the web process, you can add a Procfile for this:

web: ./bin/docker-entrypoint ./bin/rails server

And lastly, buildpacks don't bind to a default port so you'll either need to set proxy.app_port(Kamal 2.0 / kamal-proxy) to your application port or set your app to use port 80 which is the default kamal-proxy port.

servers:
  web:
    hosts:
      - 123.456.78.9

Buildpacks work in a detect and then build flow. The detect step looks for common files or checks that indicate it is indeed a ruby application by looking for a Gemfile.lock for instance. If the detect step passes then it triggers the build phase which essentially triggers a bundle install in this example.

With heroku/builder:24 so far I've found that the image size is about the same, it's only 2mb off for a 235mb image. Build time is typically faster with pack but depends on how well you've optimized your Dockerfile. The win though is not having to think about how to cache your gem installs, node installs or any other package manager installs that have a buildpack. It's also following the common conventions for building containers and various stumbling blocks that Heroku and others have been blazing through over the years.

Kamal discussion: #795
Heroku discussion: heroku/buildpacks#6
Heroku official buildpacks: https://devcenter.heroku.com/articles/buildpacks
Heroku 3rd party buildpacks: https://elements.heroku.com/buildpacks
Full setup overview: https://www.fromthekeyboard.com/deploying-a-rails-app-with-kamal-heroku-style/

Todos:

Companion PR for kamal-site Started in Add a builder example section about buildpacks kamal-site#117
Excluded files mention via project.toml
Catch up with main
~~Discuss potential for a remote pack option~~ Out of scope
Test pack with the git archive context
What should kamal build create do when using buildpacks? Just point to the install docs? https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/
Update kamal build details to run pack version && pack builder inspect
Does kamal build remove need to do anything?

nickhammond · 2024-08-28T05:49:07Z

lib/kamal/commands/builder/native/pack.rb

+        buildpacks,
+        "-t", config.absolute_image,
+        "-t", config.latest_image,
+        "--env", "BP_IMAGE_LABELS=service=#{config.service}",


Kamal expects there to be a service label, this automatically adds the label via the paketo-buildpacks/image-labels buildpack.

lib/kamal/commands/builder/native/pack.rb

nickhammond · 2024-08-28T05:50:05Z

lib/kamal/commands/builder/native/pack.rb

+    end
+
+    def buildpacks
+      (pack_buildpacks << "paketo-buildpacks/image-labels").map { |buildpack| ["--buildpack", buildpack] }


Adding this buildpack automatically so that we can label the image for Kamal

nickhammond · 2024-08-28T05:50:46Z

lib/kamal/configuration/docs/builder.yml

@@ -37,6 +37,16 @@ builder:
    arch: arm64
    host: ssh://docker@docker-builder

+  # Buildpack configuration
+  #
+  # The build configuration for using pack to build a Cloud Native Buildpack image.


Add mention of project.toml to set your excluded options. https://buildpacks.io/docs/for-app-developers/how-to/build-inputs/use-project-toml/

As I was thinking about this and removing context: "." it doesn't matter as much since it's using the git clone. The exclusion list is really only relevant when you're using "." as your build context.

alexohre · 2024-09-03T23:33:21Z

hey @nickhammond and @dhh, does this change resolve the custom build issue like using the builder of choice. e.g. docker build cloud?

nickhammond · 2024-09-04T14:49:10Z

Hey @alexohre - No, that'll just be a remote builder with the engine pointing to Docker cloud(#914) which would be a different PR. The builders were just reorganized a bit as well so it might be simpler to add an additional option for a remote cloud builder in a different PR.

alexohre · 2024-09-04T15:14:48Z

Hey @alexohre - No, that'll just be a remote builder with the engine pointing to Docker cloud(#914) which would be a different PR. The builders were just reorganized a bit as well so it might be simpler to add an additional option for a remote cloud builder in a different PR.

Oh, thanks for the awareness. I would be glad if you could help me make a PR for that since I don't know how to build gems or modify it for now

lib/kamal/commands/builder/pack.rb

nickhammond · 2024-09-06T05:30:05Z

lib/kamal/commands/builder/pack.rb

+
+  private
+    def platform
+      "linux/#{local_arches.first}"


Pack only supports building for one platform, make it obvious in docs

We can add a validation for this in Kamal::Configuration::Validator::Builder.

dhh · 2024-09-22T01:09:20Z

This is fascinating work, @nickhammond. I'm surprised by how unobtrusive it is! But I'd like to understand the whole flow better. I'm not sure this is going to be all that relevant for Rails apps that now already come with well-optimized Dockerfiles out of the box, but I could see how that may well be different if you're doing a Sinatra app or some app from another framework that doesn't provide that.

Could you show how the entire flow would go with, say, a Sinatra app, using buildpacks, and deploying on something like Digital Ocean? Want to make sure that this isn't tied to any one company or platform.

nickhammond · 2024-09-23T15:23:20Z

Hey @dhh, thanks for taking a look!

I think adding support for buildpacks will be great for the adoption of Kamal but you can always still reach for the sharper tool(a full Dockerfile) when needed.

I built out a few hello world examples, the main thing is just making sure your app boots on port 80 for kamal-proxy or just ensuring that you set your app_port if it doesn't. This isn't a buildpack-specific thing but more of a change that came with kamal-proxy, packs don't export a port by default.

Here are the hello world apps that I built and tested out on Digital Ocean and wrote a more detailed overview for the whole process as well.

kamal-buildpacks-node
kamal-buildpacks-php
kamal-buildpacks-rails - Single-file Rails app
kamal-buildpacks-sinatra

hone · 2024-09-25T15:21:31Z

@nickhammond thanks for all your investigations and opening this PR. ❤️

Want to make sure that this isn't tied to any one company or platform.

@dhh 👋 It's been a while. As Cloud Native Buildpacks (CNB) maintainer I'm biased and would love to see this supported in kamal. :)

Nick touches on this in his blog, but if it's any assurance CNB as an upstream project is a CNCF Incubation project which pushes for not being a single vendor OSS project. In fact, the project was started from the get go by two companies, Heroku & Pivotal. It's really about bringing that Heroku magic to container image building, transforming your app source code into an OCI image (No Dockerfile needed). You can push image to a registry, docker run it locally, or even use it as a base image in the FROM directive of a Dockerfile. If you don't want to use the Heroku builder and buildpacks, there`s the Paketo ones, and you can also write your own.

nickhammond · 2024-10-01T21:37:21Z

Started on the docs in this kamal-site PR basecamp/kamal-site#117.

djmb

@nickhammond - I noticed in your sample apps, that you've set the context for the builder to ., which avoids using the git clone for building.

Is that just a preference or is there any reason it would be required?

djmb · 2024-10-02T07:56:17Z

lib/kamal/commands/builder/pack.rb

+
+  private
+    def platform
+      "linux/#{local_arches.first}"


We can add a validation for this in Kamal::Configuration::Validator::Builder.

djmb · 2024-10-02T08:00:05Z

lib/kamal/commands/builder/base.rb

@@ -33,7 +34,7 @@ def info
  end

  def inspect_builder
-    docker :buildx, :inspect, builder_name unless docker_driver?
+    docker :buildx, :inspect, builder_name unless docker_driver? || pack?


Maybe we could extract a buildx? method here?

def buildx? !docker_driver? && !pack? end

@djmb We could also run pack builder inspect which returns a bunch of information about the default builder. It's a lot of information but might be useful to help triage if you're not sure what builder you're using. The Pack CLI lets you set your default builder so I have mine set to heroku/builder:24 via pack config default-builder heroku/builder:24

djmb · 2024-10-02T08:14:48Z

lib/kamal/commands/builder/pack.rb

+        "-t", config.absolute_image,
+        "-t", config.latest_image,
+        "--env", "BP_IMAGE_LABELS=service=#{config.service}",
+        *argumentize("--env", secrets, sensitive: true),


Is using environment variables the standard way to get secrets into a buildpack?

@djmb Yes, they only have the --env flag.

I just tested building with a few secrets because I was concerned they'd end up in the final image but they don't.

I just found this in the docs site though. TLDR; It's just a build-time env var, they're not available at image runtime. So they're naturally "secret", neat.

https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/cli/pack_build/#options

-e, --env stringArray Build-time environment variable, in the form 'VAR=VALUE' or 'VAR'. When using latter value-less form, value will be taken from current environment at the time this command is executed. This flag may be specified multiple times and will override individual values defined by --env-file. Repeat for each env in order (comma-separated lists not accepted) NOTE: These are NOT available at image runtime.

edmorley · 2024-10-02T16:32:34Z

lib/kamal/commands/builder/pack.rb

+    end
+
+    def buildpacks
+      (pack_buildpacks << "paketo-buildpacks/image-labels").map { |buildpack| [ "--buildpack", buildpack ] }


I've filed buildpacks/pack#2268 for adding support for --label to pack build, which if/when added would mean the buildpack injection here could be dropped :-)

Love this, thank you @edmorley!

nickhammond · 2024-10-02T16:35:49Z

@nickhammond - I noticed in your sample apps, that you've set the context for the builder to ., which avoids using the git clone for building.

Is that just a preference or is there any reason it would be required?

@djmb I usually just use the old context style when initially getting a project going since it's a bit faster. I've removed the context on all of the sample apps and they're all deploying successfully.

To be honest though I'm not super familiar with how the clone process actually works. I'm passing in the same builder_context to the pack CLI that's currently being done with the Docker build and for both I'm seeing the context as "." in the output.

Full pack command that's running:

  INFO [d6b21e93] Running /usr/bin/env pack build nickhammond/hotdonuts --platform linux/amd64 --builder heroku/builder:24 --buildpack heroku/ruby --buildpack heroku/procfile --buildpack paketo-buildpacks/image-labels -t nickhammond/hotdonuts:eca189d62a8c2ba97fdfca5f85699a50a6d50ce4 -t nickhammond/hotdonuts:latest --env BP_IMAGE_LABELS=service=hotdonuts --path . && docker push nickhammond/hotdonuts:eca189d62a8c2ba97fdfca5f85699a50a6d50ce4 && docker push nickhammond/hotdonuts:latest as n@localhost

Vs. Docker command:

  INFO [e3db1e68] Running docker buildx build --push --platform linux/amd64 --builder kamal-local-docker-container -t nickhammond/hotdonuts:eca189d62a8c2ba97fdfca5f85699a50a6d50ce4 -t nickhammond/hotdonuts:latest --label service="hotdonuts" --file Dockerfile . as n@localhost

With both though I'm seeing the clone steps, does it clone into that temp directory and then drop into it?

  INFO Cloning repo into build directory `/var/folders/6q/53gfp0q92gngndncp5mmk9cr0000gn/T/kamal-clones/hotdonuts-39f3a8537243e/hotdonuts-sinatra/`...
  INFO [a351ef89] Running /usr/bin/env git -C /var/folders/6q/53gfp0q92gngndncp5mmk9cr0000gn/T/kamal-clones/hotdonuts-39f3a8537243e clone /Users/n/src/hotdonuts-sinatra --recurse-submodules as n@localhost
  INFO Resetting local clone as `/var/folders/6q/53gfp0q92gngndncp5mmk9cr0000gn/T/kamal-clones/hotdonuts-39f3a8537243e/hotdonuts-sinatra/` already exists...
  INFO [a0986137] Running /usr/bin/env git -C /var/folders/6q/53gfp0q92gngndncp5mmk9cr0000gn/T/kamal-clones/hotdonuts-39f3a8537243e/hotdonuts-sinatra/ remote set-url origin /Users/n/src/hotdonuts-sinatra as n@localhost
  INFO [a0986137] Finished in 0.005 seconds with exit status 0 (successful).
  INFO [13044d07] Running /usr/bin/env git -C /var/folders/6q/53gfp0q92gngndncp5mmk9cr0000gn/T/kamal-clones/hotdonuts-39f3a8537243e/hotdonuts-sinatra/ fetch origin as n@localhost
  INFO [13044d07] Finished in 0.019 seconds with exit status 0 (successful).
  INFO [05383764] Running /usr/bin/env git -C /var/folders/6q/53gfp0q92gngndncp5mmk9cr0000gn/T/kamal-clones/hotdonuts-39f3a8537243e/hotdonuts-sinatra/ reset --hard eca189d62a8c2ba97fdfca5f85699a50a6d50ce4 as n@localhost
  INFO [05383764] Finished in 0.007 seconds with exit status 0 (successful).
  INFO [97641014] Running /usr/bin/env git -C /var/folders/6q/53gfp0q92gngndncp5mmk9cr0000gn/T/kamal-clones/hotdonuts-39f3a8537243e/hotdonuts-sinatra/ clean -fdx as n@localhost
  INFO [97641014] Finished in 0.006 seconds with exit status 0 (successful).
  INFO [8b0ae09b] Running /usr/bin/env git -C /var/folders/6q/53gfp0q92gngndncp5mmk9cr0000gn/T/kamal-clones/hotdonuts-39f3a8537243e/hotdonuts-sinatra/ submodule update --init as n@localhost
  INFO [8b0ae09b] Finished in 0.047 seconds with exit status 0 (successful).
  INFO [753722cb] Running /usr/bin/env git -C /var/folders/6q/53gfp0q92gngndncp5mmk9cr0000gn/T/kamal-clones/hotdonuts-39f3a8537243e/hotdonuts-sinatra/ status --porcelain as n@localhost
  INFO [753722cb] Finished in 0.006 seconds with exit status 0 (successful).
  INFO [0e2a42fa] Running /usr/bin/env git -C /var/folders/6q/53gfp0q92gngndncp5mmk9cr0000gn/T/kamal-clones/hotdonuts-39f3a8537243e/hotdonuts-sinatra/ rev-parse HEAD as n@localhost
  INFO [0e2a42fa] Finished in 0.005 seconds with exit status 0 (successful).
  INFO [7ba57d96] Running /usr/bin/env  as n@localhost
  INFO [7ba57d96] Finished in 0.002 seconds with exit status 0 (successful).
  INFO [d6b21e93] Running /usr/bin/env pack build nickhammond/hotdonuts --platform linux/amd64 --builder heroku/builder:24 --buildpack heroku/ruby --buildpack heroku/procfile --buildpack paketo-buildpacks/image-labels -t nickhammond/hotdonuts:eca189d62a8c2ba97fdfca5f85699a50a6d50ce4 -t nickhammond/hotdonuts:latest --env BP_IMAGE_LABELS=service=hotdonuts --path . && docker push nickhammond/hotdonuts:eca189d62a8c2ba97fdfca5f85699a50a6d50ce4 && docker push nickhammond/hotdonuts:latest as n@localhost

nickhammond · 2024-10-02T16:58:11Z

lib/kamal/commands/builder/pack.rb

+        "-t", config.absolute_image,
+        "-t", config.latest_image,
+        "--env", "BP_IMAGE_LABELS=service=#{config.service}",
+        *argumentize("--env", args),


With docker, build args are passed as --build-arg and with Kamal you set them via:

args: ENVIRONMENT: production

You'd still set "build args" with pack via the same args section but they ultimately get passed as --env to the pack command. Trying to reduce confusion of when to use env/arg if you're testing out your builds.

Add a pack option to the builder options

897b3b4

nickhammond commented Aug 28, 2024

View reviewed changes

Clean things up via Rubocop

826308a

nickhammond changed the title ~~Add a pack option to the builder options for cloud native buildpacks~~ Add pack option to the builder options for cloud native buildpacks Aug 28, 2024

nickhammond marked this pull request as ready for review August 28, 2024 06:02

nickhammond added 2 commits September 4, 2024 09:42

Utilize repository name for pack name

d0ffb85

Catch up with main

24f4308

nickhammond commented Sep 6, 2024

View reviewed changes

lib/kamal/commands/builder/pack.rb Outdated Show resolved Hide resolved

nickhammond added 2 commits September 5, 2024 22:17

pack arch no longer needed, update builder name in tests

ae68193

Don't need to inspect the builder if pack

2c5f2a7

nickhammond commented Sep 6, 2024

View reviewed changes

Merge branch 'basecamp:main' into buildpacks

548452a

Merge branch 'basecamp:main' into buildpacks

85a5a09

Use argumentize for secrets with pack

e252004

nickhammond added 4 commits September 27, 2024 00:19

Merge branch 'basecamp:main' into buildpacks

7174174

Merge branch 'basecamp:main' into buildpacks

f7147e0

Merge branch 'basecamp:main' into buildpacks

c601241

Point to project.toml in docs

dda8efe

nickhammond mentioned this pull request Oct 1, 2024

Add a builder example section about buildpacks basecamp/kamal-site#117

Draft

djmb reviewed Oct 2, 2024

View reviewed changes

Merge branch 'basecamp:main' into buildpacks

5482052

edmorley mentioned this pull request Oct 2, 2024

Add support for pack build --label buildpacks/pack#2268

Open

edmorley reviewed Oct 2, 2024

View reviewed changes

Ensure build args and secrets are used with pack

89b4415

nickhammond commented Oct 2, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add pack option to the builder options for cloud native buildpacks #916

Add pack option to the builder options for cloud native buildpacks #916

nickhammond commented Aug 28, 2024 •

edited

Loading

nickhammond Aug 28, 2024

nickhammond Aug 28, 2024

nickhammond Aug 28, 2024

nickhammond Oct 2, 2024 •

edited

Loading

alexohre commented Sep 3, 2024

nickhammond commented Sep 4, 2024

alexohre commented Sep 4, 2024

nickhammond Sep 6, 2024 •

edited

Loading

djmb Oct 2, 2024

dhh commented Sep 22, 2024

nickhammond commented Sep 23, 2024

hone commented Sep 25, 2024 •

edited

Loading

nickhammond commented Oct 1, 2024

djmb left a comment

djmb Oct 2, 2024

djmb Oct 2, 2024

nickhammond Oct 2, 2024

djmb Oct 2, 2024

nickhammond Oct 2, 2024

edmorley Oct 2, 2024

nickhammond Oct 2, 2024

nickhammond commented Oct 2, 2024

nickhammond Oct 2, 2024

Add pack option to the builder options for cloud native buildpacks #916

Are you sure you want to change the base?

Add pack option to the builder options for cloud native buildpacks #916

Conversation

nickhammond commented Aug 28, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nickhammond Oct 2, 2024 • edited Loading

Choose a reason for hiding this comment

alexohre commented Sep 3, 2024

nickhammond commented Sep 4, 2024

alexohre commented Sep 4, 2024

nickhammond Sep 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dhh commented Sep 22, 2024

nickhammond commented Sep 23, 2024

hone commented Sep 25, 2024 • edited Loading

nickhammond commented Oct 1, 2024

djmb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nickhammond commented Oct 2, 2024

Choose a reason for hiding this comment

nickhammond commented Aug 28, 2024 •

edited

Loading

nickhammond Oct 2, 2024 •

edited

Loading

nickhammond Sep 6, 2024 •

edited

Loading

hone commented Sep 25, 2024 •

edited

Loading