Skip to content

Commit

Permalink
Merge branch 'wayfair-incubator:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
Oded-B authored Apr 18, 2024
2 parents 4a4c226 + 05a8c31 commit 17a0543
Show file tree
Hide file tree
Showing 11 changed files with 368 additions and 10 deletions.
24 changes: 24 additions & 0 deletions .github/workflows/docker-publish-on-comment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,30 @@ jobs:
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max

- name: Extract Docker metadata - alpine
id: meta-alpine
uses: docker/metadata-action@dbef88086f6cef02e264edb7dbf63250c17cef6c
with:
context: git
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=sha
flavor: prefix=alpine-,onlatest=true
- name: Build and push Docker image - alpine
id: build-and-push-alpine
uses: docker/build-push-action@9f6f8c940b91232557f8699b21341a08624a8dce
with:
context: .
target: alpine-release
push: true
tags: ${{ steps.meta-alpine.outputs.tags }}
labels: ${{ steps.meta-alpine.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Set latest commit status as ${{ job.status }}
uses: myrotvorets/set-commit-status-action@master
if: always()
Expand Down
26 changes: 26 additions & 0 deletions .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,32 @@ jobs:
cache-from: type=gha
cache-to: type=gha,mode=max

# Extract metadata (tags, labels) for Docker
# https://github.com/docker/metadata-action
# 1st image name is for GH package repo
# 2nd image name is for DockerHub image
- name: Extract Docker metadata - alpine
id: meta-alpine
uses: docker/metadata-action@dbef88086f6cef02e264edb7dbf63250c17cef6c
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
flavor: prefix=alpine-,onlatest=true

# Build and push Docker image with Buildx (don't push on PR)
# https://github.com/docker/build-push-action
- name: Build and push Docker image - alpine
id: build-and-push-alpine
uses: docker/build-push-action@9f6f8c940b91232557f8699b21341a08624a8dce
with:
context: .
target: alpine-release
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta-alpine.outputs.tags }}
labels: ${{ steps.meta-alpine.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max


# Sign the resulting Docker image digest except on PRs.
# This will only write to the public Rekor transparency log when the Docker
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,13 @@ jobs:
steps:
- uses: actions/setup-go@v5
with:
go-version: 1.19
go-version: 1.21
- uses: actions/checkout@v4
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
uses: golangci/golangci-lint-action@v4
with:
# Optional: version of golangci-lint to use in form of v1.2 or v1.2.3 or `latest` to use the latest version
version: v1.50.1
version: v1.57.2

# Optional: working directory, useful for monorepos
# working-directory: somedir
Expand Down
11 changes: 10 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ COPY . ./
RUN make test

FROM test as build
# FROM golang:1.18.3 as build
# FROM golang:1.21.5 as build
ARG GOPROXY
ENV GOPATH=/go
ENV PATH="$PATH:$GOPATH/bin"
Expand All @@ -17,6 +17,15 @@ COPY . ./
RUN make build


FROM alpine:latest as alpine-release
WORKDIR /telefonistka
COPY --from=build /go/src/github.com/wayfair-incubator/telefonistka/telefonistka /telefonistka/bin/telefonistka
COPY templates/ /telefonistka/templates/
# This next line is hack to overcome GH actions lack of support for docker workdir override https://github.com/actions/runner/issues/878
COPY templates/ /github/workspace/templates/
USER 1001
ENTRYPOINT ["/telefonistka/bin/telefonistka"]
CMD ["server"]



Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ telefonistka bump-overwrite \
--file <(echo -e "image:\n tag: v3.4.9") \
```

It currently supports full file overwrite and regex based replacement.
It currently supports full file overwrite, regex and yaml based replacement.
See [here](docs/version_bumping.md) for more details

### GitHub Push events fanout/multiplexing
Expand Down
107 changes: 107 additions & 0 deletions cmd/telefonistka/bump-version-yaml.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
package telefonistka

import (
"context"
"fmt"
"os"
"strings"

lru "github.com/hashicorp/golang-lru/v2"
"github.com/hexops/gotextdiff"
"github.com/hexops/gotextdiff/myers"
"github.com/hexops/gotextdiff/span"
"github.com/mikefarah/yq/v4/pkg/yqlib"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/wayfair-incubator/telefonistka/internal/pkg/githubapi"
)

// This is still(https://github.com/spf13/cobra/issues/1862) the documented way to use cobra
func init() { //nolint:gochecknoinits
var targetRepo string
var targetFile string
var address string
var replacement string
var githubHost string
var triggeringRepo string
var triggeringRepoSHA string
var triggeringActor string
var autoMerge bool
eventCmd := &cobra.Command{
Use: "bump-yaml",
Short: "Bump artifact version in a file using yaml selector",
Long: `Bump artifact version in a file using yaml selector.
This will open a pull request in the target repo.
This command uses yq selector to find the yaml value to replace.
`,
Args: cobra.ExactArgs(0),
Run: func(cmd *cobra.Command, args []string) {
bumpVersionYaml(targetRepo, targetFile, address, replacement, githubHost, triggeringRepo, triggeringRepoSHA, triggeringActor, autoMerge)
},
}
eventCmd.Flags().StringVarP(&targetRepo, "target-repo", "t", getEnv("TARGET_REPO", ""), "Target Git repository slug(e.g. org-name/repo-name), defaults to TARGET_REPO env var.")
eventCmd.Flags().StringVarP(&targetFile, "target-file", "f", getEnv("TARGET_FILE", ""), "Target file path(from repo root), defaults to TARGET_FILE env var.")
eventCmd.Flags().StringVar(&address, "address", "", "Yaml value address described as a yq selector, e.g. '.db.[] | select(.name == \"postgres\").image.tag'.")
eventCmd.Flags().StringVarP(&replacement, "replacement-string", "n", "", "Replacement string that includes the version value of new artifact, e.g. 'v2.7.1'.")
eventCmd.Flags().StringVarP(&githubHost, "github-host", "g", "", "GitHub instance HOSTNAME, defaults to \"github.com\". This is used for GitHub Enterprise Server instances.")
eventCmd.Flags().StringVarP(&triggeringRepo, "triggering-repo", "p", getEnv("GITHUB_REPOSITORY", ""), "Github repo triggering the version bump(e.g. `octocat/Hello-World`) defaults to GITHUB_REPOSITORY env var.")
eventCmd.Flags().StringVarP(&triggeringRepoSHA, "triggering-repo-sha", "s", getEnv("GITHUB_SHA", ""), "Git SHA of triggering repo, defaults to GITHUB_SHA env var.")
eventCmd.Flags().StringVarP(&triggeringActor, "triggering-actor", "a", getEnv("GITHUB_ACTOR", ""), "GitHub user of the person/bot who triggered the bump, defaults to GITHUB_ACTOR env var.")
eventCmd.Flags().BoolVar(&autoMerge, "auto-merge", false, "Automatically merges the created PR, defaults to false.")
rootCmd.AddCommand(eventCmd)
}

func bumpVersionYaml(targetRepo string, targetFile string, address string, value string, githubHost string, triggeringRepo string, triggeringRepoSHA string, triggeringActor string, autoMerge bool) {
ctx := context.Background()
var githubRestAltURL string

if githubHost != "" {
githubRestAltURL = "https://" + githubHost + "/api/v3"
log.Infof("Github REST API endpoint is configured to %s", githubRestAltURL)
}
var mainGithubClientPair githubapi.GhClientPair
mainGhClientCache, _ := lru.New[string, githubapi.GhClientPair](128)

mainGithubClientPair.GetAndCache(mainGhClientCache, "GITHUB_APP_ID", "GITHUB_APP_PRIVATE_KEY_PATH", "GITHUB_OAUTH_TOKEN", strings.Split(targetRepo, "/")[0], ctx)

var ghPrClientDetails githubapi.GhPrClientDetails

ghPrClientDetails.GhClientPair = &mainGithubClientPair
ghPrClientDetails.Ctx = ctx
ghPrClientDetails.Owner = strings.Split(targetRepo, "/")[0]
ghPrClientDetails.Repo = strings.Split(targetRepo, "/")[1]
ghPrClientDetails.PrLogger = log.WithFields(log.Fields{}) // TODO what fields should be here?

defaultBranch, _ := ghPrClientDetails.GetDefaultBranch()

initialFileContent, err, _ := githubapi.GetFileContent(ghPrClientDetails, defaultBranch, targetFile)
if err != nil {
ghPrClientDetails.PrLogger.Errorf("Fail to fetch file content:%s\n", err)
os.Exit(1)
}
newFileContent, err := updateYaml(initialFileContent, address, value)
if err != nil {
ghPrClientDetails.PrLogger.Errorf("Fail to update yaml:%s\n", err)
os.Exit(1)
}

edits := myers.ComputeEdits(span.URIFromPath(""), initialFileContent, newFileContent)
ghPrClientDetails.PrLogger.Infof("Diff:\n%s", gotextdiff.ToUnified("Before", "After", initialFileContent, edits))

err = githubapi.BumpVersion(ghPrClientDetails, "main", targetFile, newFileContent, triggeringRepo, triggeringRepoSHA, triggeringActor, autoMerge)
if err != nil {
log.Errorf("Failed to bump version: %v", err)
os.Exit(1)
}
}

func updateYaml(yamlContent string, address string, value string) (string, error) {
yqExpression := fmt.Sprintf("(%s)=\"%s\"", address, value)

preferences := yqlib.NewDefaultYamlPreferences()
evaluate, err := yqlib.NewStringEvaluator().Evaluate(yqExpression, yamlContent, yqlib.NewYamlEncoder(preferences), yqlib.NewYamlDecoder(preferences))
if err != nil {
return "", err
}
return evaluate, nil
}
92 changes: 92 additions & 0 deletions cmd/telefonistka/bump-version-yaml_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
package telefonistka

import (
"testing"
)

func TestUpdateYaml(t *testing.T) {
t.Parallel()

tests := []struct {
name string
yamlContent string
address string
value string
want string
}{
{
name: "Test simple",
yamlContent: `
tag: "16.1"
`,
address: `.tag`,
value: "16.2",
want: `
tag: "16.2"
`,
},
{
name: "Test nested",
yamlContent: `
image:
repository: "postgres"
tag: "16.1"
`,
address: `.image.tag`,
value: "16.2",
want: `
image:
repository: "postgres"
tag: "16.2"
`,
},
{
name: "Test nested select",
yamlContent: `
db:
- name: "postgres"
image:
repository: "postgres"
tag: "16.1"
`,
address: `.db.[] | select(.name == "postgres").image.tag`,
value: "16.2",
want: `
db:
- name: "postgres"
image:
repository: "postgres"
tag: "16.2"
`,
},
{
name: "Test add missing",
yamlContent: `
image:
repository: "postgres"
`,
address: `.image.tag`,
value: "16.2",
want: `
image:
repository: "postgres"
tag: "16.2"
`,
},
}

for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got, err := updateYaml(tt.yamlContent, tt.address, tt.value)
if err != nil {
t.Errorf("updateYaml() error = %v", err)
return
}
if got != tt.want {
t.Errorf("updateYaml() got = %v, want %v", got, tt.want)
}
})
}
}
10 changes: 10 additions & 0 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,16 @@ The Github side of the configuration can be done via a creation of an GitHub App
* Ensure the service account has the relevant permission on the repo.
* Add `telefonistka.yaml` to repo root.

## Images

Telefonistka comes in 2 flavors:

* A normal container image, that is built from `scratch` and contains only a telefonistka binary and CA certificates.
This image is preferred and meant to be used in a production environment.
* A container image that is built on `alpine` and contains the full alpine base OS. Denoted by the `alpine-` prefix.
This image is meant for development and debugging purposes.
And in rare cases when it is used in CI and requires shell variable expansion.

## Server Configuration

Environment variables for the webhook process:
Expand Down
29 changes: 28 additions & 1 deletion docs/version_bumping.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
If your IaC repo deploys software you maintain internally you probably want to automate artifact version bumping.
Telefonistka can automate opening the IaC repo PR for the version change from the Code repo pipeline.

Currently, two modes of operation are supported:
Currently, three modes of operation are supported:

## Whole file overwrite

Expand Down Expand Up @@ -56,3 +56,30 @@ Flags:
notes:

* This assumes the target file already exist in the target repo.

## YAML based value replace

```shell
Bump artifact version in a file using yaml selector.
This will open a pull request in the target repo.
This command uses yq selector to find the yaml value to replace.

Usage:
telefonistka bump-yaml [flags]

Flags:
--address string Yaml value address described as a yq selector, e.g. '.db.[] | select(.name == "postgres").image.tag'.
--auto-merge Automatically merges the created PR, defaults to false.
-g, --github-host string GitHub instance HOSTNAME, defaults to "github.com". This is used for GitHub Enterprise Server instances.
-h, --help help for bump-yaml
-n, --replacement-string string Replacement string that includes the version value of new artifact, e.g. 'v2.7.1'.
-f, --target-file string Target file path(from repo root), defaults to TARGET_FILE env var.
-t, --target-repo string Target Git repository slug(e.g. org-name/repo-name), defaults to TARGET_REPO env var.
-a, --triggering-actor string GitHub user of the person/bot who triggered the bump, defaults to GITHUB_ACTOR env var.
-p, --triggering-repo octocat/Hello-World Github repo triggering the version bump(e.g. octocat/Hello-World) defaults to GITHUB_REPOSITORY env var.
-s, --triggering-repo-sha string Git SHA of triggering repo, defaults to GITHUB_SHA env var.
```

notes:

* This assumes the target file already exist in the target repo.
Loading

0 comments on commit 17a0543

Please sign in to comment.