Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Updated logic on when to bake Kilnfile arguments #485

Merged
merged 6 commits into from
Mar 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,7 @@ Commands:
glaze Pin versions in Kilnfile to match lock.
help prints this usage information
publish publish tile on Pivnet
re-bake re-bake constructs a tile from a bake record
release-notes generates release notes from bosh-release release notes
sync-with-local update the Kilnfile.lock based on local releases
test Test manifest for a product
Expand Down Expand Up @@ -224,6 +225,13 @@ tile. There are very few reasons a tile developer should want to do this, but if
you do, you can include these extra files here. The flag can be specified
multiple times to embed multiple files or directories.

##### `--final`

The `--final` flag is to bake a final release tile. When passing the --final flag,
Kiln creates a baked record file with metadata like source revision SHA, tile version, kiln version and
file checksums. This bake record file will be created under bake_records folder. This
bake record file can later be used to re-bake the tile.

##### `--forms-directory`

The `--forms-directory` flag takes a path to a directory that contains one
Expand Down Expand Up @@ -519,7 +527,16 @@ provides_product_versions:
```
</details>

### `re-bake`
It constructs a tile from a given bake record file.

To run the command, you simply need to be within a tile directory and execute the following command:
```
$ kiln re-bake --output-file tile.pivotal bake_records/1.0.0.json
```

Any variables that Kilnfile needs for the kiln re-bake command should be set in
~/.kiln/credentials.yml file

### `test`

Expand Down
2 changes: 1 addition & 1 deletion internal/acceptance/bake/bake_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -407,7 +407,7 @@ var _ = Describe("bake command", func() {
command := exec.Command(pathToMain, commandWithArgs...)
session, err := gexec.Start(command, GinkgoWriter, GinkgoWriter)
Expect(err).NotTo(HaveOccurred())
Eventually(session.Err).Should(gbytes.Say("non-existent-kilnfile.lock: no such file or directory"))
Eventually(session.Err).Should(gbytes.Say("no such file or directory"))
})
})
It("errors out when Kilnfile.lock cannot be unmarshalled", func() {
Expand Down
7 changes: 7 additions & 0 deletions internal/acceptance/workflows/baking_a_tile.feature
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ Feature: As a developer, I want to bake a tile
| --stub-releases |
Then a Tile is created

Scenario: it handles tiles with multiple tile names
Given I have a tile source directory "testdata/tiles/multiple-tile-names"
When I invoke kiln
| bake |
| --tile-name=goodbye |
Then a Tile is created

Scenario: it bakes a tile from a bake record
Given I have a tile source directory "testdata/tiles/bake-record"
When I invoke kiln
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
bake_configurations:
- tile_name: hello
metadata_filepath: product_template.yml
variables_filepaths:
- variables/hello.yml
icon_filepath: "gopher.png"
instance_groups_directories:
- job_types
properties_directories:
- configuration
- tile_name: goodbye
metadata_filepath: product_template.yml
variables_filepaths:
- variables/goodbye.yml
icon_filepath: "gopher.png"
instance_groups_directories:
- job_types
properties_directories:
- configuration
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
stemcell_criteria:
os: ubuntu-jammy
version: "1.329"
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
- name: port
type: port
configurable: true
default: 8080
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: hello-server
label: Server
resource_label: Server
description: HTTP Server

templates: []

static_ip: 1
dynamic_ip: 0

max_in_flight: 1
single_az_only: true

instance_definition:
name: instances
type: integer
label: Instances
configurable: true
default: 1
constraints:
min: 0
max: 1

resource_definitions:
- name: ram
type: integer
label: RAM
configurable: true
default: 1024
constraints:
min: 1024

- name: ephemeral_disk
type: integer
label: Ephemeral Disk
configurable: true
default: 4000
constraints:
min: 2000

- name: persistent_disk
type: integer
label: Persistent Disk
configurable: false
default: 4000
constraints:
min: 2000

- name: cpu
type: integer
label: CPU
configurable: true
default: 1
constraints:
min: 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
name: $( variable "tile_name" )
label: "some label"
description: "some description"
icon_image: $( icon )

metadata_version: "2.7.0"
minimum_version_for_upgrade: 0.1.0
product_version: $( version )
provides_product_versions:
- name: hello
version: $( version )

rank: 90
serial: false

releases: []

stemcell_criteria: $( stemcell )

job_types:
- $( instance_group "hello-server" )

runtime_configs: []

property_blueprints:
- $( property "port" )

form_types: []
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
label: "goodbye"
description: "some description"
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
label: "hello"
description: "some description"
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.2.3
86 changes: 52 additions & 34 deletions internal/commands/bake.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package commands

import (
"bytes"
"crypto/sha256"
"encoding/hex"
"errors"
Expand Down Expand Up @@ -125,6 +124,8 @@ func NewBake(fs billy.Filesystem, releasesService baking.ReleasesService, outLog

writeBakeRecord: writeBakeRecord,

loadKilnfile: cargo.ReadKilnfile,

metadata: metadataService,

boshVariables: builder.MetadataPartsDirectoryReader{},
Expand All @@ -141,6 +142,13 @@ func NewBake(fs billy.Filesystem, releasesService baking.ReleasesService, outLog
}
}

// WithKilnfileFunc overrides the funcion used to parse the Kilnfile.
// It is for setting up tests.
func (bake Bake) WithKilnfileFunc(fn func(string) (cargo.Kilnfile, error)) Bake {
bake.loadKilnfile = fn
return bake
}

type writeBakeRecordSignature func(string, string, string, []byte) error

type Bake struct {
Expand All @@ -153,6 +161,8 @@ type Bake struct {
stemcell stemcellService
releases fromDirectories

loadKilnfile func(string) (cargo.Kilnfile, error)

writeBakeRecord writeBakeRecordSignature

KilnVersion string
Expand Down Expand Up @@ -197,6 +207,8 @@ type BakeOptions struct {
Version string `short:"v" long:"version" description:"version of the tile"`
SkipFetchReleases bool `short:"sfr" long:"skip-fetch" description:"skips the automatic release fetch for all release directories" alias:"skip-fetch-directories"`

TileName string `short:"t" long:"tile-name" description:"select the bake_configuration matching the tile-name from the Kilnfile"`

IsFinal bool `long:"final" description:"this flag causes build metadata to be written to bake_records"`
}

Expand All @@ -213,6 +225,7 @@ func NewBakeWithInterfaces(interpolator interpolator, tileWriter tileWriter, out
icon: iconService,
metadata: metadataService,
writeBakeRecord: writeBakeRecordFn,
loadKilnfile: cargo.ReadKilnfile,

boshVariables: boshVariablesService,
forms: formsService,
Expand Down Expand Up @@ -431,11 +444,22 @@ func (b Bake) Execute(args []string) error {
b.errLogger.Println("warning: --stemcell-tarball is being deprecated in favor of --stemcells-directory")
}

if err := BakeArgumentsFromKilnfileConfiguration(&b.Options, b.loadKilnfile); err != nil {
return fmt.Errorf("failed to load bake configuration from Kilnfile: %w", err)
}

templateVariables, err := b.templateVariables.FromPathsAndPairs(b.Options.VariableFiles, b.Options.Variables)
if err != nil {
return fmt.Errorf("failed to parse template variables: %s", err)
}

if b.Options.TileName != "" {
if tileNameVariable, ok := templateVariables[builder.TileNameVariable]; ok && tileNameVariable != b.Options.TileName {
return fmt.Errorf("tile-name flag value %q does not match tile_name variable %q", b.Options.TileName, tileNameVariable)
}
templateVariables[builder.TileNameVariable] = b.Options.TileName
}

releaseManifests, err := b.releases.FromDirectories(b.Options.ReleaseDirectories)
if err != nil {
return fmt.Errorf("failed to parse releases: %s", err)
Expand All @@ -447,14 +471,6 @@ func (b Bake) Execute(args []string) error {
// TODO remove when stemcell tarball is deprecated
stemcellManifest, err = b.stemcell.FromTarball(b.Options.StemcellTarball)
} else if b.Options.Kilnfile != "" {
if err := BakeArgumentsFromKilnfileConfiguration(&b.Options, templateVariables); err != nil {
return fmt.Errorf("failed to parse releases: %s", err)
}
templateVariables, err = b.templateVariables.FromPathsAndPairs(b.Options.VariableFiles, b.Options.Variables)
if err != nil {
return fmt.Errorf("failed to parse template variables: %s", err)
}

stemcellManifests, err = b.stemcell.FromKilnfile(b.Options.Kilnfile)
} else if len(b.Options.StemcellsDirectories) > 0 {
stemcellManifests, err = b.stemcell.FromDirectories(b.Options.StemcellsDirectories)
Expand Down Expand Up @@ -596,44 +612,46 @@ func (b Bake) Usage() jhanda.Usage {
}
}

func BakeArgumentsFromKilnfileConfiguration(options *BakeOptions, variables map[string]any) error {
if options.Kilnfile == "" {
func BakeArgumentsFromKilnfileConfiguration(options *BakeOptions, loadKilnfile func(string) (cargo.Kilnfile, error)) error {
if options.Kilnfile == "" || options.StemcellTarball != "" || len(options.StemcellsDirectories) > 0 {
return nil
}
if variables == nil {
variables = make(map[string]any)
}
buf, err := os.ReadFile(options.Kilnfile)
kf, err := loadKilnfile(options.Kilnfile)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil
}
return err
}
kf, err := cargo.InterpolateAndParseKilnfile(bytes.NewReader(buf), variables)
if err != nil {
return err
}
if tileName, ok := variables[builder.TileNameVariable]; ok {
name, ok := tileName.(string)
if !ok {
return fmt.Errorf("%s value must be a string got value %#[2]v with type %[2]T", builder.TileNameVariable, tileName)
}
if index := slices.IndexFunc(kf.BakeConfigurations, func(configuration cargo.BakeConfiguration) bool {
return configuration.TileName == name
}); index >= 0 {
fromConfiguration(options, kf.BakeConfigurations[index])
}

if len(kf.BakeConfigurations) == 0 {
return nil
} else if len(kf.BakeConfigurations) == 1 {
configuration := kf.BakeConfigurations[0]
if options.TileName != "" && options.TileName != configuration.TileName {
return fmt.Errorf("the provided tile_name %q does not match the configuration %q", options.TileName, configuration.TileName)
}
fromConfiguration(options, configuration)
if configuration.TileName != "" {
variables[builder.TileNameVariable] = configuration.TileName
options.TileName = configuration.TileName
} else {
index := slices.IndexFunc(kf.BakeConfigurations, func(configuration cargo.BakeConfiguration) bool {
return configuration.TileName == options.TileName
})
if index < 0 {
return errorBakeConfigurationNotFound(options, kf)
}
fromConfiguration(options, kf.BakeConfigurations[index])
}
return nil
}

func errorBakeConfigurationNotFound(options *BakeOptions, kf cargo.Kilnfile) error {
names := make([]string, 0, len(kf.BakeConfigurations))
for _, config := range kf.BakeConfigurations {
names = append(names, config.TileName)
}
slices.Sort(names)
names = slices.Compact(names)
return fmt.Errorf("the provided tile_name %q does not match any configuration. The available names are: %v", options.TileName, names)
}

func fromConfiguration(b *BakeOptions, configuration cargo.BakeConfiguration) {
if len(configuration.Metadata) > 0 {
b.Metadata = configuration.Metadata
Expand Down
Loading
Loading