This example will showcase how to use specs in Fission so that CI/CD flow can be designed better using the specs. Also we are building the function as part of this flow - so it is assumed that the Kubernetes cluster does not have internet connectivity and can not run things like pip or maven to fetch dependencies for building from source to package.
We have a simple python function which needs one library for it's working. The files are:
$ tree .
.
├── __init__.py
├── requirements.txt
└── user.py
0 directories, 4 files
The user.py is a simple function which uses yaml library and dumps a simple document:
$ cat user.py
import sys
import yaml
document = """
a: 1
b:
c: 3
d: 4
"""
def main():
return yaml.dump(yaml.load(document))
In the requirements file we declare the pyyaml library that we need
$ cat requirements.txt
pyyaml
Finally - we want to build the source code locally so that all dependencies of the function are packed with function. For this you can run pip install in a way that dependencies are placed in same directory or you can use a simple helper script below which uses a docker container to build it. The docker container is useful if the machine on which you build source code does not have pip installed.
$ cat build.sh
#/bin/bash
# A script which builds the source code using pip and a docker container so that you don't need Pip on host machine
#
docker run -it --rm -v$(pwd):/app chauffer/pip3-compile pip install -r requirements.txt -t /app
When we run above script, we can see that the yaml module is downloaded.
$ ./build.sh
Collecting pyyaml (from -r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz (270kB)
100% |████████████████████████████████| 276kB 215kB/s
Building wheels for collected packages: pyyaml
Running setup.py bdist_wheel for pyyaml ... done
Stored in directory: /root/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f
Successfully built pyyaml
Installing collected packages: pyyaml
Successfully installed pyyaml-3.13
You can also confirm that the yaml module is downloaded in same directory as source code - now the function and it's dependencies are all in same directory.
$ tree -L 1
.
├── PyYAML-3.13.dist-info
├── __init__.py
├── _yaml.cpython-37m-x86_64-linux-gnu.so
├── build.sh
├── requirements.txt
├── user.py
└── yaml
2 directories, 5 files
Now let's use spec to create environment and function and deploy to a cluster. The first step is to initialize the fission spec - so that it creates a spec directory and stores all specs in that directory.
$ fission spec init
Creating fission spec directory 'specs'
Next is to create environment spec - this does not actually create environment since we are using the --spec flag. If you look at the environment spec file, it's like a Kubernetes YAML definition.
$ fission env create --name python --image fission/python-env --spec
$ cat specs/env-python.yaml
apiVersion: fission.io/v1
kind: Environment
metadata:
creationTimestamp: null
name: python
namespace: default
spec:
TerminationGracePeriod: 360
builder: {}
keeparchive: false
poolsize: 3
resources: {}
runtime:
functionendpointport: 0
image: fission/python-env
loadendpointpath: ""
loadendpointport: 0
version: 1
Next, let's create the function specs. This will also create the package spec inside the function file.
$ fission fn create --name pyfunc --env python --executortype newdeploy --minscale 1 --deploy "*" --entrypoint user.main --spec
$ cat specs/function-pyfunc.yaml
include:
- '*'
kind: ArchiveUploadSpec
name: default-kxFr
---
apiVersion: fission.io/v1
kind: Package
metadata:
creationTimestamp: null
name: dfxr
namespace: default
spec:
deployment:
checksum: {}
type: url
url: archive://default-kxFr
environment:
name: python
namespace: default
source:
checksum: {}
status:
buildstatus: none
---
apiVersion: fission.io/v1
kind: Function
metadata:
creationTimestamp: null
name: pyfunc
namespace: default
spec:
InvokeStrategy:
ExecutionStrategy:
ExecutorType: newdeploy
MaxScale: 1
MinScale: 1
TargetCPUPercent: 80
StrategyType: execution
configmaps: null
environment:
name: python
namespace: default
package:
functionName: user.main
packageref:
name: dfxr
namespace: default
resources: {}
secrets: null
Similarly you can create a route for the function
$ fission route create --url /some/test --function pyfunc --createingress --spec
$ cat specs/route-ede2f2c7-c0fb-4801-a619-c81dbae3719e.yaml
apiVersion: fission.io/v1
kind: HTTPTrigger
metadata:
creationTimestamp: null
name: ede2f2c7-c0fb-4801-a619-c81dbae3719e
namespace: default
spec:
createingress: true
functionref:
functionweights: null
name: pyfunc
type: name
host: ""
method: GET
relativeurl: /some/test
Now next step is to validate the specs and apply them. In a typical CI/CD workflow, the developer will create specs and commit them to Git. The CI/CD system will only validate and apply specs. The apply command makes sure that the changes only are applied to the cluster.
$ fission spec validate
$ fission spec apply
uploading archive archive://default-kxFr
1 environment created: python
1 package created: dfxr
1 function created: pyfunc
Now is the time to quickly check if the function works:
$ fission fn test --name pyfunc
a: 1
b: {c: 3, d: 4}
After you are done, you can use destroy command to delete all related objects - and you don't need to individually delete one object at a time:
$ fission spec destroy
Deleted Environment default/python
Deleted Package default/dfxr
Deleted Function default/pyfunc
The specs allow you do additional things. The spec.runtime.container is basically the container spec from Kubernetes. This allows you add environment variables to Functions as shown below. In future Fission might support PodSpec - which will allow to do more things in future.
apiVersion: fission.io/v1
kind: Environment
metadata:
creationTimestamp: null
name: jvm
namespace: default
spec:
TerminationGracePeriod: 360
builder: {}
keeparchive: true
poolsize: 3
resources: {}
runtime:
functionendpointport: 0
image: fission/jvm-env
loadendpointpath: ""
loadendpointport: 0
container:
env:
- name: JVM_OPTS
value: "-Xms256M -Xmx1024M"
version: 2
If you have code in one directory - but the libraries which code uses in some other directory, then it makes sense not to replicate the library code into every function code. The spec and multiple deploy options allow you to fetch code from multiple directories and build a function.
Source code directory:
$ tree
.
├── README.md
├── __init__.py
├── build.sh
├── requirements.txt
├── specs
│ ├── README
│ ├── env-python.yaml
│ ├── fission-deployment-config.yaml
│ ├── function-pyfunc.yaml
│ └── route-ede2f2c7-c0fb-4801-a619-c81dbae3719e.yaml
└── user.py
1 directory, 10 files
And the directory which has library code:
$ tree -L 1 ../lib_pyyaml/
../lib_pyyaml/
├── PyYAML-3.13.dist-info
├── _yaml.cpython-37m-x86_64-linux-gnu.so
└── yaml
Now you can refer to both places using multiple --deploy argument and build a function:
$ fission fn create --name pyfunc --env python --executortype newdeploy --minscale 1 --deploy "*" --deploy "../lib_pyyaml/*" --entrypoint user.main --spec
You will notice that the file upload picks up all the files specified:
$ cat specs/function-pyfunc.yaml
include:
- '*'
- ../lib_pyyaml/*
kind: ArchiveUploadSpec
name: default-CRfl