KFP-Tekton v1.5.1: Tekton 0.41.0 and Kubernetes 1.25 support fixes

• Add securitycontext for Kubernetes 1.25 Tekton namespace fix
• Update SDK and cache server to address cache tasks that doesn't have command or with ambiguous commands with printf

What's Changed

• fix(backend): Add securitycontext for k8s 1.25 by @yhwang in #1132
• chore(release): add 1.5.1 release patch by @Tomcli in #1133
• fix(sdk): add command to resource op spec by @Tomcli in #1134
• fix(backend): Add format arg to printf command by @yhwang in #1135

Full Changelog: v1.5.0...v1.5.1

KFP-Tekton v1.5.0: Tekton 0.41.0 and Kubernetes 1.25 support with new image and db configs

• Support Tekton 0.41.0 and Kubernetes 1.25
• Add new configs for result step image and db tables
• Testing infra and tests are moved to GO 1.19+
• Fix SDK bugs on pipelineloop and custom tasks

What's Changed

• chore(CI): Update go version in CI to 1.19.x by @Tomcli in #1069
• update toolchain image by @yhwang in #1071
• fix(sdk): fix nested loop underscore param by @Tomcli in #1076
• fix(bug): fix underscore param by checking pipeline param instead of task param by @Tomcli in #1077
• fix(sdk): add default labels for caching by @Tomcli in #1078
• chore(PipelineLoop): upgrade to tekton v0.41.0 by @ScrapCodes in #1072
• update tekton version to 0.41 by @Tomcli in #1070
• fix(sdk): fix nested loop counter param bug by @Tomcli in #1080
• chore(v2): cleanup old v2 content from v1 branch by @Tomcli in #1083
• fix(sdk): Fix inline spec for iterateParamPassStyle and itemPassStyle by @Tomcli in #1087
• fix(sdk): fix nested param generation issue by @Tomcli in #1089
• fix(sdk): fix nested param logic for pl by @Tomcli in #1091
• chore(sdk): update sdk version to 1.4.1 (#1093) by @Tomcli in #1106
• feat(backend): specify the image for result step by @yhwang in #1104
• feat(sdk): add pipeline_conf.timeout sdk support by @Tomcli in #1108
• fix(sdk): Update custom task arg parsing for dash arguments by @Tomcli in #1114
• feat(backend): Update db table config with Primary key to improve db read performance by @Tomcli in #1116
• chore(release): Add 1.4.1 release patch by @Tomcli in #1117
• chore(release): Add 1.5.0 release patch by @Tomcli in #1118

Full Changelog: v1.4.0...v1.5.0

KFP-Tekton v1.4.1: Bug fixes for SDK on pipelineloop and custom tasks

What's Changed

• chore(CI): Update go version in CI to 1.19.x by @Tomcli in #1069
• update toolchain image by @yhwang in #1071
• fix(sdk): fix nested loop underscore param by @Tomcli in #1076
• fix(bug): fix underscore param by checking pipeline param instead of task param by @Tomcli in #1077
• fix(sdk): add default labels for caching by @Tomcli in #1078
• cheerypick(sdk): Fix counter param cherrypick (#1080) by @Tomcli in #1082
• chore(cherrypick): fix inline spec for new features by @Tomcli in #1088
• chore(cherrypick): fix nested param generation issue (#1089) by @Tomcli in #1090
• cherrypick(sdk): fix nested param logic for pl (#1091) by @Tomcli in #1092
• chore(sdk): update sdk version to 1.4.1 by @Tomcli in #1093

Full Changelog: v1.4.0...v1.4.1

KFP-Tekton v1.4.0: Tekton 0.38.4 support with new pipelineloop SDK syntax

• Tekton 0.38.4 support to handle Tekton graph validation degradation issues (#1049, #1051)
• Default timeout global values changed to 0 and generated pipelinerun yaml no longer producing default timeout values (#1042, #1043)
• New sequential loop output parameters last_idx and last_elem to retrieved last iterated value (#1044)
• Add iterateParamPassStyle and itemPassStyle params to loop (#1059)
• Add new annotation flags to enable artifact tracking at pipeline and task level (#1065)
• Optimized KFP-Tekton deployment and generated yaml (#1038, #1055, #1062)
• Improved custom task caching to also aware of input parameters (#1056)

What's Changed

• fix(sdk) include finally tasks in placeholder indir by @Udiknedormin in #1035
• Add finally context var test case for #1035 by @Tomcli in #1036
• do not mark failed until all running PRs are done by @wzhanw in #1037
• optimize(sdk): Further moving common caching annotations to pipeline level by @Tomcli in #1038
• fix(backend): add check for casting by @yhwang in #1040
• Update tekton default timeout in global config by @Tomcli in #1042
• feat(backend): handle pipelineloop status by @yhwang in #1039
• feat(sdk): Remove timeout since we moved to default unlimited timeout by @Tomcli in #1043
• fix(sdk): Fix nested numeric parameters by @Tomcli in #1047
• fix(backend): Update Tekton to 0.38.4 by @Tomcli in #1049
• feat(pipelineloop): Support last_idx and last_elem by @ScrapCodes in #1044
• test: Add a test case for the validation by @yhwang in #1051
• Update kfp-sdk to 1.8.14 by @Tomcli in #1050
• feature(backend): Add QPS and Burst settings by @yhwang in #1055
• fix(manifests): Patch cache config and break ct to default pl deployment by @Tomcli in #1057
• feat(sdk): add custom task param support in command spec by @Tomcli in #1061
• feat(sdk): add iterateParamPassStyle and itemPassStyle params to loop by @Tomcli in #1059
• chore(sdk): Remove unnecessary code by @Tomcli in #1062
• fix(pipelineloop): caching should include the params for making caching key. by @ScrapCodes in #1056
• fix(sdk): update loop dsl to handle same argument in nested loop by @Tomcli in #1052
• chore(release): add 1.4.0 release changes by @Tomcli in #1063
• chore(OWNERS): Remove Animesh as reviewer by @Tomcli in #1064
• feat(backend): add new annotation flags to enable artifact tracking at pipeline and task level by @Tomcli in #1065
• fix(sdk): update python protobuf package for security update by @Tomcli in #1066
• chore(docs): Move loop to custom feature since it's not Tekton native by @Tomcli in #1067

Full Changelog: v1.3.1...v1.4.0

KFP-Tekton v1.3.1: KFP 1.8.4 with Tekton 0.38.3 support

• KFP 1.8.4 rebase and Tekton 0.38.3 support to handle Tekton cancel regression issue (#1024, #1025, #1026)
• Optimize SDK by moving common task labels to pipeline level, reduce big data passing script size, and ability to opt-out artifact tracking annotations to save more Tekton yaml space (#1020, #1029, #1032)
• Add custom task metadata support for pipeline loop (#1028)
• Update Pipeline run timeout format to new timeouts to avoid future regression issue. (#1031)
• Update S3 CSI deployment with datashim 0.3 release to support multi-arch cpu deployment (#1019)

What's Changed

• Update s3 csi deployment by @Tomcli in #1019
• optimize(sdk): Move common task labels to pipeline labels to reduce yaml size by @Tomcli in #1020
• KFP 1.8.4 Rebase by @Tomcli in #1024
• Update tekton to 0.38.3 to resolve pipeline cancel issue with custom task by @Tomcli in #1025
• Add test cases for sequence and cache by @yhwang in #1026
• Remove google and argo specific tests to reduce repo size by @Tomcli in #1027
• sdk(feat): add metadata support for loop by @Tomcli in #1028
• optimize(sdk): Rewrite data passing script into more optimized code by @Tomcli in #1029
• feat(sdk): Add alpha env flag to opt out artifact tracking annotations by @Tomcli in #1032
• Update pipelinerun timeout format to timeouts by @Tomcli in #1031
• Update kfp-tekton release to 1.3.1 by @Tomcli in #1033

Full Changelog: v1.3.0...v1.3.1

KFP-Tekton v1.3.0: Tekton 0.36.0 support with new big data passing enhancements

• Tekton 0.36.0 support (#938, #1016, #1018), KFP SDK 1.8.13 support (#979, #999)
• Enhanced copy-artifact step to support binary and directory artifacts (#942, #985, #997)
• Big data passing supports volume passing config with user volumes (#995, #1006)
• Alpha features for artifact preview and artifact path as output parameters (#991, #993)
• Optimized Tekton script based component into command based to reduce the number of Tekton init-containers to 1 for Tekton 0.36.0 (#975, #992)
• Optimized pipelineloop status memory usage and param passing to use less etcd space. (#963, #970, #982)
• Allow task to output as big data without other task's consumption (#967)
• Generalized storage config class (#977)
• Updated cache server to support multi-container caching for Tekton 0.36.0 (#974, #1018)
• Updated output parameter name to be case sensitive during runtime. (#1013)
• Updated Tekton global config to support security context and automount_service_account_token (#955)
• Alpha feature to optimize pipeline output status to use less etcd space while caching all the pipeline information in the kfp-tekton database (#976)

What's Changed

• fix(sdk): Fix param prefix replacement by @Tomcli in #952
• feat(backend): update go version and license files by @yhwang in #949
• fix(backend): refine copy-artifacts step by @yhwang in #942
• feat(test): add backend integration test by @yhwang in #956
• fix 954, trim whitespace and newline char for range input by @wzhanw in #961
• Add pipelinerun pod template support for security context and automou… by @Tomcli in #955
• fix(pipeline-loops): PipelineLoop consumes significant amount of memory by @rafalbigaj in #963
• update install guide by @JiaxuanYang in #965
• handle step from task results by @wzhanw in #964
• upgrade tekton to 0.36.0 by @yhwang in #938
• fix(pipelineloop): Add param with empty space test by @Tomcli in #970
• fix(docs): Add explicit commands for Tekton versoned deployment to avoid doc con… by @Tomcli in #973
• make cache to support multiple containers in task pod by @wzhanw in #974
• fix(sdk): update condition template by @yhwang in #975
• feature(backend): support embedded-status: minimal by @yhwang in #976
• Update sdk dependency and release version to address private python package pull by @Tomcli in #979
• fix(backend): nil ptr exception for empty run status in pipeline-loops, fixes #981 by @Udiknedormin in #982
• feat(sdk): Add labels for artifact outputs when tasks are not referred by @Tomcli in #967
• fix(sdk): Verify binary data in step copy-results-artifacts. Fixes #984 by @maxdebayser in #985
• eliminate the use of taskSpec.script by @yhwang in #992
• chore(sdk) Bump kfp dependency to 1.8.13 by @akchinSTC in #999
• Update sdk version to 1.2.3 by @Tomcli in #1002
• sdk(feat): Add big data passing with custom volume on kfp-tekton by @Tomcli in #995
• chore(apiserver): Object Storage connection abstraction by @rimolive in #977
• Add alpha preview feature into compiler by @Tomcli in #991
• Remove old build dependencies by @Tomcli in #1005
• fix(sdk): Support passing directories in step-copy-results-artifacts and step-copy-artifacts. Fixes #996 by @FFede0 in #997
• fix(sdk): Fix big data passing with multiple type of volume templates by @Tomcli in #1006
• Add alpha feature to output big data passing file path instead of task run name by @Tomcli in #993
• Update result name to be case sensitive by @Tomcli in #1013
• fix(sdk): fix exit handler param with empty exit task by @Tomcli in #1015
• fix(sdk): Add new logic to cover new tekton runafter condition by @Tomcli in #1016
• Add kfp-tekton 1.3.0 release template by @Tomcli in #1011
• fix(sdk): fix unit test merging conflicts by @Tomcli in #1017
• fix(backend): fix cache server output by @Tomcli in #1018

New Contributors

• @JiaxuanYang made their first contribution in #965
• @maxdebayser made their first contribution in #985
• @akchinSTC made their first contribution in #999
• @rimolive made their first contribution in #977
• @FFede0 made their first contribution in #997

Full Changelog: v1.2.1...v1.3.0

KFP-Tekton v1.2.1: support negative looping values and Tekton custom task sub-dag

• Pipelineloop controller support negative loop values (#911)
• SDK: AddOnGroup feature for creating custom task sub-dag (#920)
• Add owner reference support for resourceOp component. (#922, #925)
• SDK and pipelineloop controller bug fixes. (#921, #928, #936, #941, #945)

What's Changed

• feat: Support loop ranges with -ve values. by @ScrapCodes in #911
• Support the comma as default string param separator by @wzhanw in #921
• fix(SDK): Update sanitization logic to sync with kfp dsl by @Tomcli in #919
• Add resource op owner reference mapping by @Tomcli in #922
• Add necessary env variables for resource op owner reference by @Tomcli in #925
• fix(sdk) add test for any-seq inside of loop by @Udiknedormin in #927
• Fix multiple nested condition in loops issue by @Tomcli in #928
• Update sdk release to 1.2.1 by @Tomcli in #929
• feat(sdk): Add helper class AddOnGroup by @yhwang in #920
• wait for pipelineloop-break-operation task final status by @wzhanw in #936
• [build] update toolchain image by @yhwang in #937
• [build] update github workflow to use go v1.17 by @yhwang in #939
• Loop ranges should handle from = to, and generate one iteration. by @ScrapCodes in #941
• iteration limit support, default limit is 10000 by @wzhanw in #944
• fix: typo in frontend message by @davidxia in #946
• Store the uploaded pipeline as original yaml format by @Tomcli in #945
• Update manifests with 1.2.1 release by @Tomcli in #948

New Contributors

• @davidxia made their first contribution in #946

Full Changelog: v1.2.0...v1.2.1

KFP-Tekton v1.2.0 with KFP v1.8.1 and Tekton v0.31.4

• Tekton v0.31.4 support (#917)
• Rebase KFP core to 1.8.1, KFP SDK to 1.8.11 (#862, #868, #872)
• Pipeline loop with new loop class that can support numeric python range, python enumerate, and python string separator syntax (#838, #842, #898, #901, #904)
• Pipeline loop now can pass down Tekton podTemplate and service account to nested loops (#891, #909)
• Performance enhancement: Parallel loop break task is now running with Tekton custom task, caching also works with nested loops (#854, #870, #871)
• Added new golang packages for KFP metadata tracking, log archival, and caching for Tekton Custom task controllers (#829, #852, #861)
• S3-CSI storage driver integration now can configure to run on Minikube and Kind deployment (#899, #900)
• Bug fixes on Pipelineloop, Loop SDK, and variable sanitization.

What's Changed

• feat: caching for custom task. by @ScrapCodes in #852
• [test] fix typo in pipeline-loop example by @yhwang in #860
• fix(sdk) add test for empty loops by @Udiknedormin in #864
• feat(pipelineloop): Add caching configuration support through config-map. by @ScrapCodes in #861
• Fix Tekton conditional dependency on sub-dag by @Tomcli in #866
• Kfp 1.8.0 api and frontend rebase by @Tomcli in #862
• Feat(pipelineloop): Break task as custom task. by @ScrapCodes in #854
• Update kfp sdk dependency to 1.8.10+ by @Tomcli in #868
• Add caching labels for inline loops and custom tasks by @Tomcli in #870
• Break task sdk update by @ScrapCodes in #871
• Patch kfp 1.8.1 new config by @Tomcli in #872
• Add 1.2 release changes by @Tomcli in #875
• feat(objectstore) Metadata object store by @ScrapCodes in #876
• fix(sdk) add test for custom-tasks with literals. Fixes #880 by @Udiknedormin in #881
• fix(PipelineLoop): integer divide by zero by @rafalbigaj in #883
• chore: Objectstore package refactoring. by @ScrapCodes in #884
• [test] Update mnist-e2e sample by @yhwang in #888
• fix: objectstore refactoring by @ScrapCodes in #887
• Add current iteration item to the PipelineLoopPipelineRunStatus by @wzhanw in #894
• Add yhwang as approver by @Tomcli in #893
• fix(sdk) add tests for non-literal separator by @Udiknedormin in #896
• fix(controller): Added fields serviceAccountName and PodTemplate to PipelineLoop. by @ScrapCodes in #891
• Add missing openshift standalone permission by @Tomcli in #897
• Add missing csi plugin for multi-user manifests by @Tomcli in #899
• feat: New variable iterationNumberParam to track current item. by @ScrapCodes in #898
• Parameterize kubelet path for s3-csi manifests by @Tomcli in #900
• feat(sdk): extension function Loop.enumerate() by @yhwang in #901
• IterateNumeric behavior changed to contain the value of current interation item. by @ScrapCodes in #904
• Update Tekton client and manifests to v0.31.4 by @Tomcli in #905
• pass down the podTemplate and serviceAccount from run to loopSpec by @wzhanw in #909
• fix(sdk): handle the node selectors by @yhwang in #916
• fix(sdk): fix nested loop global param bug by @Tomcli in #917

Full Changelog: v1.1.1...v1.2.0

KFP-Tekton v1.1.1: Kubernetes v1.22 Fixes for Kubeflow Multi-user mode

KFP-Tekton v1.1.1 Release Note:

• Sync upstream KFP fixes on k8s 1.22 multi-user support for cache deployer and new OpenShift rbac (#846, #847)
• Python SDK bug fixes on Argo to Tekton parameter conversions and performance optimizations (#825, #837, #841)

What's Changed

• fix(pipelineloop): break loop should cancel all pipeline runs. by @ScrapCodes in #824
• Enable pipelineparam for container env by @Tomcli in #825
• Clean up owners file by @Tomcli in #827
• Additional param to enable running copies of same pipeline concurrently by @tedhtchang in #828
• feat(objectstorelogger): objectstorelogger as a separate module. by @ScrapCodes in #829
• Update performance script to include more metrics by @Tomcli in #832
• feat(sdk): implement loop DSL ext: from_string. Part of #791 by @Udiknedormin in #837
• Update KFP-Tekton installation to give guidance on GPU worker nodes. by @Tomcli in #836
• Add support to parse pipeline param in json string by @Tomcli in #841
• Remove unsynced components by @Tomcli in #840
• fix(sdk) separator param. Part of #791 by @Udiknedormin in #842
• Add loop numeric sdk support by @Tomcli in #838
• Update manifests for multi-users k8s 1.22 support by @Tomcli in #846
• [test] replace the cache-server image by @yhwang in #848
• Add KFP-Tekton 1.1.1 release by @Tomcli in #847

New Contributors

• @tedhtchang made their first contribution in #828

Full Changelog: v1.1.0...v1.1.1

KFP-Tekton v1.1.0 with Tekton v0.30.0 and Kubernetes v1.22+ Support

KFP-Tekton 1.1.0 Release Note:

• Tekton v0.30.0 Support (#798)
• Kubernetes v1.22+ Support (#813)
• Auto strip End of files and log archives are enabled by default (#747)
• Big Data Passing now supports S3 and CSI integrations (originally only supported PVC) (#753)
• Big Data Passing supports unique and customizable artifact path between loops. (#751, #771)
• PipelineLoop now supports loop retries and multi-pipeline workspaces (Need Tekton v0.30.0+) (#781, #787)
• Recursion with PipelineLoop now can set maximum recursion stack. (#808)
• Generated Tekton CR Optimization: component.yaml annotations reduced with sha256 encoding, tasks can be generated as namespace scope templates with sha1 encoding, and shorten Tekton yaml compile time by 50%. (#736, #805, #810)
• Alpha pipelineLoop Break with Python SDK extensions (#749, #799, #807, #809)
• Alpha pipelineLoop log archives (#806)
• Bug fixes and updates on Big Data passing with parameters, Katib and Trusted-AI samples, and UI library dependencies. (#730, #740, #750, #760, #768, #815)

What's Changed

• Update reference to 1.0 release by @animeshsingh in #722
• Update SDK versions by @ckadner in #724
• Remove Argo template UI reference to avoid UX conflicts with v2 compatible mode by @Tomcli in #723
• [test] update sample for katib by @yhwang in #730
• Fix(sdk): chain nested conditions to work on non-dependable condition tasks. by @Tomcli in #732
• Optimize kfp-tekon SDK compile time by @Tomcli in #736
• fix(sdk) sanitize paths in artifact_items. Fixes #738 by @Udiknedormin in #739
• Fix #733 the pipeline parameter consumed as file by @huixa in #740
• fix(sdk) orig_params being overwritten in custom-tasks by @Udiknedormin in #742
• [test] fix variable overriding by @yhwang in #743
• Relax kfp sdk dependencies by @Tomcli in #746
• Enable strip eof and archive logs as default by @Tomcli in #747
• Install links update by @shrivs3 in #741
• Add doc for break feature for loop by @pugangxa in #749
• Support data passing pipeline. by @ScrapCodes in #750
• Update install readme with KF 1.4.0 deployment by @Tomcli in #752
• Add artifact mid path to big data passing by @Tomcli in #751
• Remove noninlined files for those tests, it is not required. Added a CI check. by @ScrapCodes in #755
• [test] update build-image-dind.sh by @yhwang in #758
• Proposal: Adding ScrapCodes as approver. by @ScrapCodes in #754
• [test] use emptyDir for mysql and minio by @yhwang in #759
• Update README.md typos. by @jritten in #756
• feat(sdk): Add an API to return prepared workflow as python dicts. by @ScrapCodes in #757
• Update trusted AI pipeline to have unique job names by @Tomcli in #760
• fix(sdk): "when" in some ParallelFor loops by @Udiknedormin in #762
• Add tag to frontend code with changes for Tekton by @drewbutlerbb4 in #764
• Fix a typo for the openshift docs by @Tomcli in #763
• Add csi-s3 storage class as big data passing storage by @Tomcli in #753
• Add UI backward compatibility for dsl.container syntax by @Tomcli in #768
• Add performance test script by @ckadner in #773
• Fix archived logs for custom tasks by @Tomcli in #777
• expose current iteration item to pipelinerun's annotation by @wzhanw in #776
• feat(sdk): factor out the images to use in copying steps by @Udiknedormin in #782
• feat(pipelineloop): Implement retry for pipelineloop controller. by @ScrapCodes in #781
• feat(pipelineloop): Support workspaces for pipelineLoop. by @ScrapCodes in #787
• [test] Add taskRun and run into the report by @yhwang in #785
• Move loop retry test as a different example by @Tomcli in #789
• support separator string for iteration string param by @wzhanw in #790
• Update workflow.go by @xawangyd in #794
• Load performance test pipelines from files by @ckadner in #796
• Fix performance test script and update testing examples by @Tomcli in #795
• Upgrade tekton api client and manifest to 0.30.0 & update default timeout by @Tomcli in #798
• Add taskrun name and uid into big data passing midpath by @Tomcli in #771
• Add tekton loop dsl extension skeleton by @Tomcli in #799
• feat(pipelineloop): Break operation for pipelineloop. by @ScrapCodes in #807
• update pipelineloop operator to check break tasks with taskrun by @Tomcli in #809
• Reduce component spec annotations by @Tomcli in #805
• fix(pipelineloop): Nested call stack depth limit by @ScrapCodes in #808
• Chore: regenerate python tests by @Tomcli in #812
• Update pipelineloop and manifests with k8s 1.22 API by @Tomcli in #813
• Move postcss to version 8 and fix Tekton UI dependency chain by @Tomcli in #815
• Add option to produce pipelinerun taskspec as taskref by @Tomcli in #810
• feat(pipelineloop): Support logging to Object store. Built as an extension for zap. by @ScrapCodes in #806
• kfp-tekton 1.1 release by @Tomcli in #814

New Contributors

• @huixa made their first contribution in #740
• @shrivs3 made their first contribution in #741
• @jritten made their first contribution in #756
• @xawangyd made their first contribution in #794

Full Changelog: v1.0.0...v1.1.0