diff --git a/docs/design/custom-resource-metrics-crd.md b/docs/design/custom-resource-metrics-crd.md new file mode 100644 index 0000000000..03be742b03 --- /dev/null +++ b/docs/design/custom-resource-metrics-crd.md @@ -0,0 +1,211 @@ +# Kube-State-Metrics - CustomResourceMonitor CRD Proposal + + +--- + +Authors: Catherine Fang (CatherineF-dev@), Christian Schlotter (chrischdi@) + +Date: 26. Jun 2023 + +Target release: v + +--- + +## Table of Contents +- [Glossary](#glossary) +- [Problem Statement](#problem-statement) +- [Goal](#goal) +- [Status Quo](#status-quo) +- [Proposal](#proposal) + - [New flags](#new-flags) + - [CustomResourceMonitor Definition](#customresourcemonitor-definition) + - [Watch and Reconcile on CustomResourceMonitor CRs](#watch-and-reconcile-on-customresourcemonitor-crs) +- [CUJ](#cuj) + + + +## Glossary + +- kube-state-metrics: “Simple service that listens to the Kubernetes API server + and generates metrics about the state of the objects” +- Custom Resource: https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources +- [CustomResourceState](https://github.com/kubernetes/kube-state-metrics/blob/main/docs/customresourcestate-metrics.md) monitoring feature: existing feature which collects custom resource metrics + + +## Problem Statement + +1. Using CustomResourceState monitoring feature is not user-friendly. Current ways on configuring CustomResourceState monitoring are: +* `--custom-resource-state-config "inline yaml (see example)"` or +* `--custom-resource-state-config-file /path/to/config.yaml`. Either mounted or configmap. + +2. Current CustomResourceState monitoring feature doesn't support multiple configuration files. + +For example, for a company with 10 teams, each team wants to collect Custom Resource metrics for their owned Custom Resources. + + +## Goal + +A better UX to collect custom resource metrics + +## Proposal + +Add a custom resource definition (CustomResourceMonitor) which contains customresourcestate.Metrics. + +kube-state-metrics watched on CustomResourceMonitor CRs and concatenate these CRs into one config `customresourcestate.Metrics` which has the same content using `--custom-resource-state-config`. + +### New flags +Apart from existing two flags (`--custom-resource-state-config ` and `--custom-resource-state-config-file`), these three flags will be added: +* `--custom_resource_monitor`: whether watch CustomResourceMonitor CRs or not. +* `--custom_resource_monitor_labels`: only watch CustomResourceMonitor with [labelSelectors](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#list-and-watch-filtering). For example, `environment=production,tier=frontend` means selecting CustomResourceMonitor CRs which have these two labels. It's used to avoid double custom metrics collection when multiple kube-state-metrics are installed. +* `--custom_resource_monitor_namespaces`: only watch CustomResourceMonitor under namespaces=x. + +If `--custom_resources_monitor_enabled` is set, `--custom-resource-state-config` and `--custom-resource-state-config-file` will be ignored. + +### CustomResourceMonitor Definition + +* GroupName: kubestatemetrics.io + * Alternative kubestatemetrics.k8s.io: 1. *.k8s.io needs approval 2. ksm isn't inside k/k repo (need to double confirm) + * Alternative ksm.io: has been used by a company +* Version: v1alpha1 +* Kind: CustomResourceMonitor + +```go +package v1alpha1 + +import ( + "k8s.io/kube-state-metrics/v2/pkg/customresourcestate" +) + +type CustomResourceMonitor struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + customresourcestate.Metrics `json:",inline"` +} +``` + +```yaml +# Example CR +apiVersion: kubestatemetrics.io/v1alpha1 +kind: CustomResourceMonitor +metadata: + name: test-cr2 + namespace: kube-system + generation: 1 +spec: + resources: + - groupVersionKind: + group: myteam.io + kind: "Foo" + version: "v1" + metrics: + - name: "uptime" + help: "Foo uptime" + each: + type: Gauge + gauge: + path: [status, uptime] +``` + +### Watch and Reconcile on CustomResourceMonitor CRs + +Kube-state-metrics listens on add, update and delete events of CustomResourceMonitor via Kubernetes +client-go reflectors. On these events kube-state-metrics lists all CustomResourceMonitor CRs and concatenate CRs into one config `customresourcestate.Metrics` which has the almost same content with `--custom-resource-state-config` config. + +This generated custom resource config updates CustomResourceStore by adding monitored custom resource stores and deleting unmonitored custom resource stores. + + +```yaml +# example cr +apiVersion: kubestatemetrics.io/v1alpha1 +kind: CustomResourceMonitor +metadata: + name: nodepool +spec: + resources: + - groupVersionKind: + group: addons.k8s.io + kind: "FakedNodePool" + version: "v1alpha1" + metrics: + - name: "nodepool_generation" + help: "Nodepool generation" + each: + type: Gauge + gauge: + path: [metadata, generation] +``` + + +``` + +---------------+ +---------------------+ +-----------------------+ + | CRM_informer | | nodepool_reflector | | custom_resource_store | + +---------------+ +---------------------+ +-----------------------+ +---------------------\ | | | +| add/update/delete |-| | | +| CustomResource | | | | +| Monitor CR | | | | +| (monitor-nodepool) | | | | +|--------------------| | | | + | | | + | ListAndAddCustomResourceMonitors() | + |-------------------------------------------------->| + | | | + | DeleteOldCustomResourceMonitors() | + |-------------------------------------------------->| + | | | + | | Update(nodepool) | + | |----------------------------->| + | | | ----------\ + | | |-| Build() | + | | | |---------| + | | | ----------------------------\ + | | |-| generateMetrics(nodepool) | + | | | |---------------------------| + | | | +``` + +
+ Code to reproduce diagram + +Build via [text-diagram](http://weidagang.github.io/text-diagram/) + +``` +object CRM_informer nodepool_reflector custom_resource_store + +note left of CRM_informer: add/update/delete \n CustomResource \n Monitor CR \n(monitor-nodepool) +CRM_informer -> custom_resource_store: ListAndAddCustomResourceMonitors() +CRM_informer -> custom_resource_store: DeleteOldCustomResourceMonitors() + + +nodepool_reflector -> custom_resource_store: Update(nodepool) + +note right of custom_resource_store: Build() + +note right of custom_resource_store: generateMetrics(nodepool) +``` + + +
+ +Custom Resource store always [adds](https://github.com/kubernetes/kube-state-metrics/blob/main/internal/store/builder.go#L186) new custom resource metrics. Deletion of custom resource metrics needs to be implemented. + +### Alternatives +- Generate metrics configuration based on field annotations: https://github.com/kubernetes/kube-state-metrics/issues/1899 + - Limitation: need to have source code permission + +## Migrate from CustomResourceState +``` ++ apiVersion: kubestatemetrics.io/v1alpha1 +- kind: CustomResourceStateMetrics ++ kind: CustomResourceMonitor ++ metadata: ++ name: crm_nodepool ++ labels: ++ monitoring.backend.io: true +spec: # copy content from --custom-resource-state-config-file +``` + +## Critical User Journey (CUJ) +* cloud-provider: watch CustomResourceMonitor CRs with label `monitoring.(gke|aks|eks).io=true` under system namespaces +* application platform: watch CustomResourceMonitor CRs with label `monitoring.frontend.io=true` under non-system namespaces +* monitoring platform team: watch CustomResourceMonitor CRs with label `monitoring.platform.io=true` under non-system namespaces diff --git a/examples/cr.yaml b/examples/cr.yaml new file mode 100644 index 0000000000..2f82c4a09e --- /dev/null +++ b/examples/cr.yaml @@ -0,0 +1,20 @@ +# TODO: use another CR +apiVersion: customresource.ksm.io/v1alpha1 +kind: CustomResourceMonitor +metadata: + name: test-cr + namespace: kube-system + generation: 2 +spec: + resources: + - groupVersionKind: + group: addons.gke.io + kind: "Stackdriver" + version: "v1alpha1" + metrics: + - name: "uptime" + help: "Foo uptime" + each: + type: Gauge + gauge: + path: [metadata, generation] diff --git a/examples/cr2.yaml b/examples/cr2.yaml new file mode 100644 index 0000000000..a46c6bb60f --- /dev/null +++ b/examples/cr2.yaml @@ -0,0 +1,19 @@ +apiVersion: customresource.ksm.io/v1alpha1 +kind: CustomResourceMonitor +metadata: + name: test-cr2 + namespace: kube-system + generation: 1 +spec: + resources: + - groupVersionKind: + group: customresource.ksm.io + kind: "CustomResourceMonitor" + version: "v1alpha1" + metrics: + - name: "uptime2" + help: "Bar uptime" + each: + type: Gauge + gauge: + path: [metadata, generation] diff --git a/go.sum b/go.sum index 91ead89693..38542011c2 100644 --- a/go.sum +++ b/go.sum @@ -191,11 +191,16 @@ golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk= golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.18.0 h1:k8NLag8AGHnn+PHbl7g43CtqZAwG60vZkLqgyZgIHgQ= -golang.org/x/tools v0.18.0/go.mod h1:GL7B4CwcLLeo59yx/9UWWuNOW1n3VZ4f5axWfML7Lcg= -golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -236,5 +241,27 @@ sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMm sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= sigs.k8s.io/structured-merge-diff/v4 v4.4.1 h1:150L+0vs/8DA78h1u02ooW1/fFq/Lwr+sGiqlzvrtq4= sigs.k8s.io/structured-merge-diff/v4 v4.4.1/go.mod h1:N8hJocpFajUSSeSJ9bOZ77VzejKZaXsTtZo4/u7Io08= -sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo= -sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8= +honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= +honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= +k8s.io/api v0.27.2 h1:+H17AJpUMvl+clT+BPnKf0E3ksMAzoBBg7CntpSuADo= +k8s.io/api v0.27.2/go.mod h1:ENmbocXfBT2ADujUXcBhHV55RIT31IIEvkntP6vZKS4= +k8s.io/apimachinery v0.27.2 h1:vBjGaKKieaIreI+oQwELalVG4d8f3YAMNpWLzDXkxeg= +k8s.io/apimachinery v0.27.2/go.mod h1:XNfZ6xklnMCOGGFNqXG7bUrQCoR04dh/E7FprV6pb+E= +k8s.io/client-go v0.27.2 h1:vDLSeuYvCHKeoQRhCXjxXO45nHVv2Ip4Fe0MfioMrhE= +k8s.io/client-go v0.27.2/go.mod h1:tY0gVmUsHrAmjzHX9zs7eCjxcBsf8IiNe7KQ52biTcQ= +k8s.io/component-base v0.27.2 h1:neju+7s/r5O4x4/txeUONNTS9r1HsPbyoPBAtHsDCpo= +k8s.io/component-base v0.27.2/go.mod h1:5UPk7EjfgrfgRIuDBFtsEFAe4DAvP3U+M8RTzoSJkpo= +k8s.io/klog/v2 v2.100.1 h1:7WCHKK6K8fNhTqfBhISHQ97KrnJNFZMcQvKp7gP/tmg= +k8s.io/klog/v2 v2.100.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= +k8s.io/kube-openapi v0.0.0-20230501164219-8b0f38b5fd1f h1:2kWPakN3i/k81b0gvD5C5FJ2kxm1WrQFanWchyKuqGg= +k8s.io/kube-openapi v0.0.0-20230501164219-8b0f38b5fd1f/go.mod h1:byini6yhqGC14c3ebc/QwanvYwhuMWF6yz2F8uwW8eg= +k8s.io/sample-controller v0.27.2 h1:KTdiLknxjf0CB4LTTJTGfzJjnqR5QA/pgUQvXJqyw/I= +k8s.io/sample-controller v0.27.2/go.mod h1:WfiHY1M7OODPfq9OX+6Vc3Df+R5A4yWwctYC2og0hPo= +k8s.io/utils v0.0.0-20230505201702-9f6742963106 h1:EObNQ3TW2D+WptiYXlApGNLVy0zm/JIBVY9i+M4wpAU= +k8s.io/utils v0.0.0-20230505201702-9f6742963106/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= +rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= +rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= diff --git a/internal/discovery/types.go b/internal/discovery/types.go index 7de4a6ca4a..9337e2c7ae 100644 --- a/internal/discovery/types.go +++ b/internal/discovery/types.go @@ -15,93 +15,22 @@ package discovery import ( "fmt" - "sync" - "github.com/prometheus/client_golang/prometheus" "k8s.io/apimachinery/pkg/runtime/schema" ) -type groupVersionKindPlural struct { +// GroupVersionKindPlural is GVK + Plural +type GroupVersionKindPlural struct { schema.GroupVersionKind Plural string } -func (g groupVersionKindPlural) String() string { +func (g GroupVersionKindPlural) String() string { return fmt.Sprintf("%s/%s, Kind=%s, Plural=%s", g.Group, g.Version, g.Kind, g.Plural) } -type kindPlural struct { +// KindPlural is Kind + Plural +type KindPlural struct { Kind string Plural string } - -// CRDiscoverer provides a cache of the collected GVKs, along with helper utilities. -type CRDiscoverer struct { - // m is a mutex to protect the cache. - m sync.RWMutex - // Map is a cache of the collected GVKs. - Map map[string]map[string][]kindPlural - // ShouldUpdate is a flag that indicates whether the cache was updated. - WasUpdated bool - // CRDsAddEventsCounter tracks the number of times that the CRD informer triggered the "add" event. - CRDsAddEventsCounter prometheus.Counter - // CRDsDeleteEventsCounter tracks the number of times that the CRD informer triggered the "remove" event. - CRDsDeleteEventsCounter prometheus.Counter - // CRDsCacheCountGauge tracks the net amount of CRDs affecting the cache at this point. - CRDsCacheCountGauge prometheus.Gauge -} - -// SafeRead executes the given function while holding a read lock. -func (r *CRDiscoverer) SafeRead(f func()) { - r.m.RLock() - defer r.m.RUnlock() - f() -} - -// SafeWrite executes the given function while holding a write lock. -func (r *CRDiscoverer) SafeWrite(f func()) { - r.m.Lock() - defer r.m.Unlock() - f() -} - -// AppendToMap appends the given GVKs to the cache. -func (r *CRDiscoverer) AppendToMap(gvkps ...groupVersionKindPlural) { - if r.Map == nil { - r.Map = map[string]map[string][]kindPlural{} - } - for _, gvkp := range gvkps { - if _, ok := r.Map[gvkp.Group]; !ok { - r.Map[gvkp.Group] = map[string][]kindPlural{} - } - if _, ok := r.Map[gvkp.Group][gvkp.Version]; !ok { - r.Map[gvkp.Group][gvkp.Version] = []kindPlural{} - } - r.Map[gvkp.Group][gvkp.Version] = append(r.Map[gvkp.Group][gvkp.Version], kindPlural{Kind: gvkp.Kind, Plural: gvkp.Plural}) - } -} - -// RemoveFromMap removes the given GVKs from the cache. -func (r *CRDiscoverer) RemoveFromMap(gvkps ...groupVersionKindPlural) { - for _, gvkp := range gvkps { - if _, ok := r.Map[gvkp.Group]; !ok { - continue - } - if _, ok := r.Map[gvkp.Group][gvkp.Version]; !ok { - continue - } - for i, el := range r.Map[gvkp.Group][gvkp.Version] { - if el.Kind == gvkp.Kind { - if len(r.Map[gvkp.Group][gvkp.Version]) == 1 { - delete(r.Map[gvkp.Group], gvkp.Version) - if len(r.Map[gvkp.Group]) == 0 { - delete(r.Map, gvkp.Group) - } - break - } - r.Map[gvkp.Group][gvkp.Version] = append(r.Map[gvkp.Group][gvkp.Version][:i], r.Map[gvkp.Group][gvkp.Version][i+1:]...) - break - } - } - } -} diff --git a/internal/store/builder.go b/internal/store/builder.go index 5c38857d7b..4a34275b66 100644 --- a/internal/store/builder.go +++ b/internal/store/builder.go @@ -48,7 +48,6 @@ import ( metricsstore "k8s.io/kube-state-metrics/v2/pkg/metrics_store" "k8s.io/kube-state-metrics/v2/pkg/options" "k8s.io/kube-state-metrics/v2/pkg/sharding" - "k8s.io/kube-state-metrics/v2/pkg/util" "k8s.io/kube-state-metrics/v2/pkg/watch" ) @@ -195,9 +194,17 @@ func (b *Builder) DefaultGenerateCustomResourceStoresFunc() ksmtypes.BuildCustom // WithCustomResourceStoreFactories returns configures a custom resource stores factory func (b *Builder) WithCustomResourceStoreFactories(fs ...customresource.RegistryFactory) { + deletedCutomResource := map[string]bool{} + for k := range availableStores { + if nonCustomResource(k) { + continue + } + deletedCutomResource[k] = true + } + for i := range fs { f := fs[i] - gvr := util.GVRFromType(f.Name(), f.ExpectedType()) + gvr := customresource.GVRFromType(f.Name(), f.ExpectedType()) var gvrString string if gvr != nil { gvrString = gvr.String() @@ -216,6 +223,13 @@ func (b *Builder) WithCustomResourceStoreFactories(fs ...customresource.Registry b.useAPIServerCache, ) } + deletedCutomResource[f.Name()] = false + } + + for k, v := range deletedCutomResource { + if v { + delete(availableStores, k) + } } } @@ -348,6 +362,45 @@ var availableStores = map[string]func(f *Builder) []cache.Store{ "volumeattachments": func(b *Builder) []cache.Store { return b.buildVolumeAttachmentStores() }, } +var nonCustomResourceStores = map[string]bool{ + "certificatesigningrequests": true, + "clusterroles": true, + "configmaps": true, + "clusterrolebindings": true, + "cronjobs": true, + "daemonsets": true, + "deployments": true, + "endpoints": true, + "endpointslices": true, + "horizontalpodautoscalers": true, + "ingresses": true, + "ingressclasses": true, + "jobs": true, + "leases": true, + "limitranges": true, + "mutatingwebhookconfigurations": true, + "namespaces": true, + "networkpolicies": true, + "nodes": true, + "persistentvolumeclaims": true, + "persistentvolumes": true, + "poddisruptionbudgets": true, + "pods": true, + "replicasets": true, + "replicationcontrollers": true, + "resourcequotas": true, + "roles": true, + "rolebindings": true, + "secrets": true, + "serviceaccounts": true, + "services": true, + "statefulsets": true, + "storageclasses": true, + "validatingwebhookconfigurations": true, + "volumeattachments": true, + "verticalpodautoscalers": true, +} + func resourceExists(name string) bool { _, ok := availableStores[name] return ok @@ -361,6 +414,13 @@ func availableResources() []string { return c } +func nonCustomResource(name string) bool { + if val, ok := nonCustomResourceStores[name]; ok && val { + return true + } + return false +} + func (b *Builder) buildConfigMapStores() []cache.Store { return b.buildStoresFunc(configMapMetricFamilies(b.allowAnnotationsList["configmaps"], b.allowLabelsList["configmaps"]), &v1.ConfigMap{}, createConfigMapListWatch, b.useAPIServerCache) } @@ -553,7 +613,7 @@ func (b *Builder) buildCustomResourceStores(resourceName string, familyHeaders := generator.ExtractMetricFamilyHeaders(metricFamilies) - gvr := util.GVRFromType(resourceName, expectedType) + gvr := customresource.GVRFromType(resourceName, expectedType) var gvrString string if gvr != nil { gvrString = gvr.String() @@ -594,6 +654,65 @@ func (b *Builder) buildCustomResourceStores(resourceName string, return stores } +func (b *Builder) hasResources(resourceName string, expectedType interface{}) bool { + gvr := customresource.GVRFromType(resourceName, expectedType) + if gvr == nil { + return true + } + discoveryClient, err := customresource.CreateDiscoveryClient(b.utilOptions.Apiserver, b.utilOptions.Kubeconfig) + if err != nil { + klog.ErrorS(err, "Failed to create discovery client") + return false + } + g := gvr.Group + v := gvr.Version + r := gvr.Resource + isCRDInstalled, err := discovery.IsResourceEnabled(discoveryClient, schema.GroupVersionResource{ + Group: g, + Version: v, + Resource: r, + }) + if err != nil { + klog.ErrorS(err, "Failed to check if CRD is enabled", "group", g, "version", v, "resource", r) + return false + } + if !isCRDInstalled { + klog.InfoS("CRD is not installed", "group", g, "version", v, "resource", r) + return false + } + // Wait for the resource to come up. + timer := time.NewTimer(ResourceDiscoveryTimeout) + ticker := time.NewTicker(ResourceDiscoveryInterval) + dynamicClient, err := customresource.CreateDynamicClient(b.utilOptions.Apiserver, b.utilOptions.Kubeconfig) + if err != nil { + klog.ErrorS(err, "Failed to create dynamic client") + return false + } + var list *unstructured.UnstructuredList + for range ticker.C { + select { + case <-timer.C: + klog.InfoS("No CRs found for GVR", "group", g, "version", v, "resource", r) + return false + default: + list, err = dynamicClient.Resource(schema.GroupVersionResource{ + Group: g, + Version: v, + Resource: r, + }).List(b.ctx, metav1.ListOptions{}) + if err != nil { + klog.ErrorS(err, "Failed to list objects", "group", g, "version", v, "resource", r) + return false + } + } + if len(list.Items) > 0 { + break + } + } + + return true +} + // startReflector starts a Kubernetes client-go reflector with the given // listWatcher and registers it with the given store. func (b *Builder) startReflector( diff --git a/internal/store/builder_test.go b/internal/store/builder_test.go index 3bb5dea3aa..a381e1ab7e 100644 --- a/internal/store/builder_test.go +++ b/internal/store/builder_test.go @@ -195,4 +195,9 @@ func TestWithAllowAnnotations(t *testing.T) { t.Errorf("Test error for Desc: %s\n Want: \n%+v\n Got: \n%#+v", test.Desc, test.Wanted, resolvedAllowAnnotations) } } +func TestNonCustomResourceName(t *testing.T) { + if len(availableStores) != len(nonCustomResourceStores) { + t.Errorf("Want: %d Got: %d", len(availableStores), len(nonCustomResourceStores)) + } + // TODO: judge each nonCustomResourceStores is inside availableStores. } diff --git a/internal/discovery/discovery.go b/pkg/app/discovery.go similarity index 71% rename from internal/discovery/discovery.go rename to pkg/app/discovery.go index bd764ab701..2b7c9c783b 100644 --- a/internal/discovery/discovery.go +++ b/pkg/app/discovery.go @@ -11,14 +11,15 @@ See the License for the specific language governing permissions and limitations under the License. */ -// Package discovery provides a discovery and resolution logic for GVKs. -package discovery +package app import ( "context" "fmt" + "sync" "time" + "github.com/prometheus/client_golang/prometheus" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/runtime/schema" "k8s.io/client-go/dynamic" @@ -27,16 +28,87 @@ import ( "k8s.io/client-go/tools/cache" "k8s.io/klog/v2" + "k8s.io/kube-state-metrics/v2/internal/discovery" "k8s.io/kube-state-metrics/v2/internal/store" "k8s.io/kube-state-metrics/v2/pkg/customresource" "k8s.io/kube-state-metrics/v2/pkg/metricshandler" "k8s.io/kube-state-metrics/v2/pkg/options" - "k8s.io/kube-state-metrics/v2/pkg/util" ) // Interval is the time interval between two cache sync checks. const Interval = 3 * time.Second +// CRDiscoverer provides a cache of the collected GVKs, along with helper utilities. +type CRDiscoverer struct { + // m is a mutex to protect the cache. + m sync.RWMutex + // Map is a cache of the collected GVKs. + Map map[string]map[string][]discovery.KindPlural + // ShouldUpdate is a flag that indicates whether the cache was updated. + WasUpdated bool + // CRDsAddEventsCounter tracks the number of times that the CRD informer triggered the "add" event. + CRDsAddEventsCounter prometheus.Counter + // CRDsDeleteEventsCounter tracks the number of times that the CRD informer triggered the "remove" event. + CRDsDeleteEventsCounter prometheus.Counter + // CRDsCacheCountGauge tracks the net amount of CRDs affecting the cache at this point. + CRDsCacheCountGauge prometheus.Gauge +} + +// SafeRead executes the given function while holding a read lock. +func (r *CRDiscoverer) SafeRead(f func()) { + r.m.RLock() + defer r.m.RUnlock() + f() +} + +// SafeWrite executes the given function while holding a write lock. +func (r *CRDiscoverer) SafeWrite(f func()) { + r.m.Lock() + defer r.m.Unlock() + f() +} + +// AppendToMap appends the given GVKs to the cache. +func (r *CRDiscoverer) AppendToMap(gvkps ...discovery.GroupVersionKindPlural) { + if r.Map == nil { + r.Map = map[string]map[string][]discovery.KindPlural{} + } + for _, gvkp := range gvkps { + if _, ok := r.Map[gvkp.Group]; !ok { + r.Map[gvkp.Group] = map[string][]discovery.KindPlural{} + } + if _, ok := r.Map[gvkp.Group][gvkp.Version]; !ok { + r.Map[gvkp.Group][gvkp.Version] = []discovery.KindPlural{} + } + r.Map[gvkp.Group][gvkp.Version] = append(r.Map[gvkp.Group][gvkp.Version], discovery.KindPlural{Kind: gvkp.Kind, Plural: gvkp.Plural}) + } +} + +// RemoveFromMap removes the given GVKs from the cache. +func (r *CRDiscoverer) RemoveFromMap(gvkps ...discovery.GroupVersionKindPlural) { + for _, gvkp := range gvkps { + if _, ok := r.Map[gvkp.Group]; !ok { + continue + } + if _, ok := r.Map[gvkp.Group][gvkp.Version]; !ok { + continue + } + for i, el := range r.Map[gvkp.Group][gvkp.Version] { + if el.Kind == gvkp.Kind { + if len(r.Map[gvkp.Group][gvkp.Version]) == 1 { + delete(r.Map[gvkp.Group], gvkp.Version) + if len(r.Map[gvkp.Group]) == 0 { + delete(r.Map, gvkp.Group) + } + break + } + r.Map[gvkp.Group][gvkp.Version] = append(r.Map[gvkp.Group][gvkp.Version][:i], r.Map[gvkp.Group][gvkp.Version][i+1:]...) + break + } + } + } +} + // StartDiscovery starts the discovery process, fetching all the objects that can be listed from the apiserver, every `Interval` seconds. // resolveGVK needs to be called after StartDiscovery to generate factories. func (r *CRDiscoverer) StartDiscovery(ctx context.Context, config *rest.Config) error { @@ -56,7 +128,7 @@ func (r *CRDiscoverer) StartDiscovery(ctx context.Context, config *rest.Config) v := version.(map[string]interface{})["name"].(string) k := objSpec["names"].(map[string]interface{})["kind"].(string) p := objSpec["names"].(map[string]interface{})["plural"].(string) - gotGVKP := groupVersionKindPlural{ + gotGVKP := discovery.GroupVersionKindPlural{ GroupVersionKind: schema.GroupVersionKind{ Group: g, Version: v, @@ -81,7 +153,7 @@ func (r *CRDiscoverer) StartDiscovery(ctx context.Context, config *rest.Config) v := version.(map[string]interface{})["name"].(string) k := objSpec["names"].(map[string]interface{})["kind"].(string) p := objSpec["names"].(map[string]interface{})["plural"].(string) - gotGVKP := groupVersionKindPlural{ + gotGVKP := discovery.GroupVersionKindPlural{ GroupVersionKind: schema.GroupVersionKind{ Group: g, Version: v, @@ -116,7 +188,7 @@ func (r *CRDiscoverer) StartDiscovery(ctx context.Context, config *rest.Config) } // ResolveGVKToGVKPs resolves the variable VKs to a GVK list, based on the current cache. -func (r *CRDiscoverer) ResolveGVKToGVKPs(gvk schema.GroupVersionKind) (resolvedGVKPs []groupVersionKindPlural, err error) { // nolint:revive +func (r *CRDiscoverer) ResolveGVKToGVKPs(gvk schema.GroupVersionKind) (resolvedGVKPs []discovery.GroupVersionKindPlural, err error) { // nolint:revive g := gvk.Group v := gvk.Version k := gvk.Kind @@ -134,7 +206,7 @@ func (r *CRDiscoverer) ResolveGVKToGVKPs(gvk schema.GroupVersionKind) (resolvedG break } } - return []groupVersionKindPlural{ + return []discovery.GroupVersionKindPlural{ { GroupVersionKind: schema.GroupVersionKind{ Group: g, @@ -148,7 +220,7 @@ func (r *CRDiscoverer) ResolveGVKToGVKPs(gvk schema.GroupVersionKind) (resolvedG if hasVersion && !hasKind { kinds := r.Map[g][v] for _, el := range kinds { - resolvedGVKPs = append(resolvedGVKPs, groupVersionKindPlural{ + resolvedGVKPs = append(resolvedGVKPs, discovery.GroupVersionKindPlural{ GroupVersionKind: schema.GroupVersionKind{ Group: g, Version: v, @@ -163,7 +235,7 @@ func (r *CRDiscoverer) ResolveGVKToGVKPs(gvk schema.GroupVersionKind) (resolvedG for version, kinds := range versions { for _, el := range kinds { if el.Kind == k { - resolvedGVKPs = append(resolvedGVKPs, groupVersionKindPlural{ + resolvedGVKPs = append(resolvedGVKPs, discovery.GroupVersionKindPlural{ GroupVersionKind: schema.GroupVersionKind{ Group: g, Version: version, @@ -179,7 +251,7 @@ func (r *CRDiscoverer) ResolveGVKToGVKPs(gvk schema.GroupVersionKind) (resolvedG versions := r.Map[g] for version, kinds := range versions { for _, el := range kinds { - resolvedGVKPs = append(resolvedGVKPs, groupVersionKindPlural{ + resolvedGVKPs = append(resolvedGVKPs, discovery.GroupVersionKindPlural{ GroupVersionKind: schema.GroupVersionKind{ Group: g, Version: version, @@ -216,11 +288,11 @@ func (r *CRDiscoverer) PollForCacheUpdates( // Update the list of enabled custom resources. var enabledCustomResources []string for _, factory := range customFactories { - gvrString := util.GVRFromType(factory.Name(), factory.ExpectedType()).String() + gvrString := customresource.GVRFromType(factory.Name(), factory.ExpectedType()).String() enabledCustomResources = append(enabledCustomResources, gvrString) } // Create clients for discovered factories. - discoveredCustomResourceClients, err := util.CreateCustomResourceClients(opts.Apiserver, opts.Kubeconfig, customFactories...) + discoveredCustomResourceClients, err := customresource.CreateCustomResourceClients(opts.Apiserver, opts.Kubeconfig, customFactories...) if err != nil { klog.ErrorS(err, "failed to update custom resource stores") } diff --git a/internal/discovery/discovery_test.go b/pkg/app/discovery_test.go similarity index 82% rename from internal/discovery/discovery_test.go rename to pkg/app/discovery_test.go index 39d81aebe2..fec68ee391 100644 --- a/internal/discovery/discovery_test.go +++ b/pkg/app/discovery_test.go @@ -11,7 +11,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package discovery +package app import ( "reflect" @@ -19,6 +19,8 @@ import ( "testing" "k8s.io/apimachinery/pkg/runtime/schema" + + "k8s.io/kube-state-metrics/v2/internal/discovery" ) func TestGVKMapsResolveGVK(t *testing.T) { @@ -26,21 +28,21 @@ func TestGVKMapsResolveGVK(t *testing.T) { desc string gvkmaps *CRDiscoverer gvk schema.GroupVersionKind - got []groupVersionKindPlural - want []groupVersionKindPlural + got []discovery.GroupVersionKindPlural + want []discovery.GroupVersionKindPlural } testcases := []testcase{ { desc: "variable version and kind", gvkmaps: &CRDiscoverer{ - Map: map[string]map[string][]kindPlural{ + Map: map[string]map[string][]discovery.KindPlural{ "apps": { "v1": { - kindPlural{ + discovery.KindPlural{ Kind: "Deployment", Plural: "deployments", }, - kindPlural{ + discovery.KindPlural{ Kind: "StatefulSet", Plural: "statefulsets", }, @@ -49,7 +51,7 @@ func TestGVKMapsResolveGVK(t *testing.T) { }, }, gvk: schema.GroupVersionKind{Group: "apps", Version: "*", Kind: "*"}, - want: []groupVersionKindPlural{ + want: []discovery.GroupVersionKindPlural{ { GroupVersionKind: schema.GroupVersionKind{Group: "apps", Version: "v1", Kind: "Deployment"}, Plural: "deployments", @@ -63,20 +65,20 @@ func TestGVKMapsResolveGVK(t *testing.T) { { desc: "variable version", gvkmaps: &CRDiscoverer{ - Map: map[string]map[string][]kindPlural{ + Map: map[string]map[string][]discovery.KindPlural{ "testgroup": { "v1": { - kindPlural{ + discovery.KindPlural{ Kind: "TestObject1", Plural: "testobjects1", }, - kindPlural{ + discovery.KindPlural{ Kind: "TestObject2", Plural: "testobjects2", }, }, "v1alpha1": { - kindPlural{ + discovery.KindPlural{ Kind: "TestObject1", Plural: "testobjects1", }, @@ -85,7 +87,7 @@ func TestGVKMapsResolveGVK(t *testing.T) { }, }, gvk: schema.GroupVersionKind{Group: "testgroup", Version: "*", Kind: "TestObject1"}, - want: []groupVersionKindPlural{ + want: []discovery.GroupVersionKindPlural{ { GroupVersionKind: schema.GroupVersionKind{Group: "testgroup", Version: "v1", Kind: "TestObject1"}, Plural: "testobjects1", @@ -99,20 +101,20 @@ func TestGVKMapsResolveGVK(t *testing.T) { { desc: "variable kind", gvkmaps: &CRDiscoverer{ - Map: map[string]map[string][]kindPlural{ + Map: map[string]map[string][]discovery.KindPlural{ "testgroup": { "v1": { - kindPlural{ + discovery.KindPlural{ Kind: "TestObject1", Plural: "testobjects1", }, - kindPlural{ + discovery.KindPlural{ Kind: "TestObject2", Plural: "testobjects2", }, }, "v1alpha1": { - kindPlural{ + discovery.KindPlural{ Kind: "TestObject1", Plural: "testobjects1", }, @@ -121,7 +123,7 @@ func TestGVKMapsResolveGVK(t *testing.T) { }, }, gvk: schema.GroupVersionKind{Group: "testgroup", Version: "v1", Kind: "*"}, - want: []groupVersionKindPlural{ + want: []discovery.GroupVersionKindPlural{ { GroupVersionKind: schema.GroupVersionKind{Group: "testgroup", Version: "v1", Kind: "TestObject1"}, Plural: "testobjects1", @@ -135,20 +137,20 @@ func TestGVKMapsResolveGVK(t *testing.T) { { desc: "fixed version and kind", gvkmaps: &CRDiscoverer{ - Map: map[string]map[string][]kindPlural{ + Map: map[string]map[string][]discovery.KindPlural{ "testgroup": { "v1": { - kindPlural{ + discovery.KindPlural{ Kind: "TestObject1", Plural: "testobjects1", }, - kindPlural{ + discovery.KindPlural{ Kind: "TestObject2", Plural: "testobjects2", }, }, "v1alpha1": { - kindPlural{ + discovery.KindPlural{ Kind: "TestObject1", Plural: "testobjects1", }, @@ -157,7 +159,7 @@ func TestGVKMapsResolveGVK(t *testing.T) { }, }, gvk: schema.GroupVersionKind{Group: "testgroup", Version: "v1", Kind: "TestObject1"}, - want: []groupVersionKindPlural{ + want: []discovery.GroupVersionKindPlural{ { GroupVersionKind: schema.GroupVersionKind{Group: "testgroup", Version: "v1", Kind: "TestObject1"}, Plural: "testobjects1", diff --git a/pkg/app/server.go b/pkg/app/server.go index aa5b2c916c..ac6bf35dea 100644 --- a/pkg/app/server.go +++ b/pkg/app/server.go @@ -26,10 +26,12 @@ import ( "net/http/pprof" "os" "path/filepath" + "runtime" "strconv" - "strings" "time" + "k8s.io/client-go/rest" + "github.com/oklog/run" "github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus/collectors" @@ -39,11 +41,11 @@ import ( "github.com/prometheus/common/version" "github.com/prometheus/exporter-toolkit/web" "gopkg.in/yaml.v3" + clientset "k8s.io/client-go/kubernetes" _ "k8s.io/client-go/plugin/pkg/client/auth" // Initialize common client auth plugins. "k8s.io/client-go/tools/clientcmd" "k8s.io/klog/v2" - "k8s.io/kube-state-metrics/v2/internal/discovery" "k8s.io/kube-state-metrics/v2/internal/store" "k8s.io/kube-state-metrics/v2/pkg/allowdenylist" "k8s.io/kube-state-metrics/v2/pkg/customresource" @@ -52,8 +54,14 @@ import ( "k8s.io/kube-state-metrics/v2/pkg/metricshandler" "k8s.io/kube-state-metrics/v2/pkg/optin" "k8s.io/kube-state-metrics/v2/pkg/options" - "k8s.io/kube-state-metrics/v2/pkg/util" + "k8s.io/kube-state-metrics/v2/pkg/util/proc" + + crmonitorclientset "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned" + + "strings" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) const ( @@ -61,6 +69,17 @@ const ( healthzPath = "/healthz" ) +type Reconfigure struct { +} + +func (r *Reconfigure) ResolveCustomResourceConfig(opts *options.Options) (customresourcestate.ConfigDecoder, error) { + return resolveCustomResourceConfig(opts) +} + +func (r *Reconfigure) FromConfig(decoder customresourcestate.ConfigDecoder) ([]customresource.RegistryFactory, error) { + return customresourcestate.FromConfig2(decoder) +} + // promLogger implements promhttp.Logger type promLogger struct{} @@ -244,7 +263,7 @@ func RunKubeStateMetrics(ctx context.Context, opts *options.Options) error { proc.StartReaper() storeBuilder.WithUtilOptions(opts) - kubeClient, err := util.CreateKubeClient(opts.Apiserver, opts.Kubeconfig) + kubeClient, ksmCRClient, err := createKubeClient(opts.Apiserver, opts.Kubeconfig) if err != nil { return fmt.Errorf("failed to create client: %v", err) } @@ -268,8 +287,10 @@ func RunKubeStateMetrics(ctx context.Context, opts *options.Options) error { m := metricshandler.New( opts, kubeClient, + ksmCRClient, storeBuilder, opts.EnableGZIPEncoding, + &Reconfigure{}, ) // Run MetricsHandler if config == nil { @@ -285,7 +306,7 @@ func RunKubeStateMetrics(ctx context.Context, opts *options.Options) error { // A nil CRS config implies that we need to hold off on all CRS operations. if config != nil { - discovererInstance := &discovery.CRDiscoverer{ + discovererInstance := &CRDiscoverer{ CRDsAddEventsCounter: crdsAddEventsCounter, CRDsDeleteEventsCounter: crdsDeleteEventsCounter, CRDsCacheCountGauge: crdsCacheCountGauge, @@ -444,6 +465,30 @@ func md5HashAsMetricValue(data []byte) float64 { } func resolveCustomResourceConfig(opts *options.Options) (customresourcestate.ConfigDecoder, error) { + if opts.CustomResourcesKSMCRWatched { + ksmCRClient, err := createKubeCRClient(opts.Apiserver, opts.Kubeconfig) + if err != nil { + return nil, fmt.Errorf("Can not create KSM CR client: %v", err) + } + // TODO(): fetch CRs in all namespaces + crMonitorList, err := ksmCRClient.CustomresourceV1alpha1().CustomResourceMonitors("kube-system").List(context.Background(), metav1.ListOptions{}) + if err != nil { + return nil, fmt.Errorf("Can not list KSM CR: %v", err) + } + var crconfig customresourcestate.Metrics + for _, item := range crMonitorList.Items { + crconfig.Spec.Resources = append(crconfig.Spec.Resources, item.Spec.Resources...) + } + fmt.Printf("Merged custom resource %v \n", crconfig) + var buf strings.Builder + encoder := yaml.NewEncoder(&buf) + err = encoder.Encode(&crconfig) + if err != nil { + return nil, fmt.Errorf("Can encode KSM CR: %v", err) + } + return yaml.NewDecoder(strings.NewReader(buf.String())), nil + } + if s := opts.CustomResourceConfig; s != "" { return yaml.NewDecoder(strings.NewReader(s)), nil } @@ -456,3 +501,61 @@ func resolveCustomResourceConfig(opts *options.Options) (customresourcestate.Con } return nil, nil } + +func createKubeClient(apiserver string, kubeconfig string) (clientset.Interface, crmonitorclientset.Interface, error) { + var config *rest.Config + + var err error + + if config == nil { + config, err = clientcmd.BuildConfigFromFlags(apiserver, kubeconfig) + if err != nil { + return nil, nil, err + } + } + + config.UserAgent = fmt.Sprintf("%s/%s (%s/%s) kubernetes/%s", "kube-state-metrics", version.Version, runtime.GOOS, runtime.GOARCH, version.Revision) + config.AcceptContentTypes = "application/vnd.kubernetes.protobuf,application/json" + config.ContentType = "application/vnd.kubernetes.protobuf" + + kubeClient, err := clientset.NewForConfig(config) + if err != nil { + return nil, nil, err + } + + // Informers don't seem to do a good job logging error messages when it + // can't reach the server, making debugging hard. This makes it easier to + // figure out if apiserver is configured incorrectly. + klog.InfoS("Tested communication with server") + v, err := kubeClient.Discovery().ServerVersion() + if err != nil { + return nil, nil, fmt.Errorf("error while trying to communicate with apiserver: %w", err) + } + klog.InfoS("Run with Kubernetes cluster version", "major", v.Major, "minor", v.Minor, "gitVersion", v.GitVersion, "gitTreeState", v.GitTreeState, "gitCommit", v.GitCommit, "platform", v.Platform) + klog.InfoS("Communication with server successful") + + customResourceMonitorClient, err := crmonitorclientset.NewForConfig(config) + if err != nil { + return nil, nil, err + } + + return kubeClient, customResourceMonitorClient, nil +} + +func createKubeCRClient(apiserver string, kubeconfig string) (crmonitorclientset.Interface, error) { + config, err := clientcmd.BuildConfigFromFlags(apiserver, kubeconfig) + if err != nil { + return nil, err + } + + config.UserAgent = version.Version + config.AcceptContentTypes = "application/vnd.kubernetes.protobuf,application/json" + config.ContentType = "application/vnd.kubernetes.protobuf" + + customResourceMonitorClients, err := crmonitorclientset.NewForConfig(config) + if err != nil { + return nil, err + } + + return customResourceMonitorClients, nil +} diff --git a/pkg/util/utils.go b/pkg/customresource/utils.go similarity index 67% rename from pkg/util/utils.go rename to pkg/customresource/utils.go index 63e5aba650..2a77f7a18e 100644 --- a/pkg/util/utils.go +++ b/pkg/customresource/utils.go @@ -14,14 +14,11 @@ See the License for the specific language governing permissions and limitations under the License. */ -package util +package customresource import ( - "fmt" - "runtime" "strings" - "github.com/prometheus/common/version" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/runtime/schema" "k8s.io/client-go/discovery" @@ -29,10 +26,7 @@ import ( clientset "k8s.io/client-go/kubernetes" "k8s.io/client-go/rest" "k8s.io/client-go/tools/clientcmd" - "k8s.io/klog/v2" testUnstructuredMock "k8s.io/sample-controller/pkg/apis/samplecontroller/v1alpha1" - - "k8s.io/kube-state-metrics/v2/pkg/customresource" ) var config *rest.Config @@ -40,47 +34,8 @@ var currentKubeClient clientset.Interface var currentDiscoveryClient *discovery.DiscoveryClient var currentDynamicClient *dynamic.DynamicClient -// CreateKubeClient creates a Kubernetes clientset and a custom resource clientset. -func CreateKubeClient(apiserver string, kubeconfig string) (clientset.Interface, error) { - if currentKubeClient != nil { - return currentKubeClient, nil - } - - var err error - - if config == nil { - config, err = clientcmd.BuildConfigFromFlags(apiserver, kubeconfig) - if err != nil { - return nil, err - } - } - - config.UserAgent = fmt.Sprintf("%s/%s (%s/%s) kubernetes/%s", "kube-state-metrics", version.Version, runtime.GOOS, runtime.GOARCH, version.Revision) - config.AcceptContentTypes = "application/vnd.kubernetes.protobuf,application/json" - config.ContentType = "application/vnd.kubernetes.protobuf" - - kubeClient, err := clientset.NewForConfig(config) - if err != nil { - return nil, err - } - - // Informers don't seem to do a good job logging error messages when it - // can't reach the server, making debugging hard. This makes it easier to - // figure out if apiserver is configured incorrectly. - klog.InfoS("Tested communication with server") - v, err := kubeClient.Discovery().ServerVersion() - if err != nil { - return nil, fmt.Errorf("error while trying to communicate with apiserver: %w", err) - } - klog.InfoS("Run with Kubernetes cluster version", "major", v.Major, "minor", v.Minor, "gitVersion", v.GitVersion, "gitTreeState", v.GitTreeState, "gitCommit", v.GitCommit, "platform", v.Platform) - klog.InfoS("Communication with server successful") - - currentKubeClient = kubeClient - return kubeClient, nil -} - // CreateCustomResourceClients creates a custom resource clientset. -func CreateCustomResourceClients(apiserver string, kubeconfig string, factories ...customresource.RegistryFactory) (map[string]interface{}, error) { +func CreateCustomResourceClients(apiserver string, kubeconfig string, factories ...RegistryFactory) (map[string]interface{}, error) { // Not relying on memoized clients here because the factories are subject to change. var err error if config == nil { diff --git a/pkg/customresourcemonitor/apis/config/customresource.ksm.io_customresourcemonitors.yaml b/pkg/customresourcemonitor/apis/config/customresource.ksm.io_customresourcemonitors.yaml new file mode 100644 index 0000000000..848c51cfee --- /dev/null +++ b/pkg/customresourcemonitor/apis/config/customresource.ksm.io_customresourcemonitors.yaml @@ -0,0 +1,243 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.10.0 + creationTimestamp: null + name: customresourcemonitors.customresource.ksm.io +spec: + group: customresource.ksm.io + names: + kind: CustomResourceMonitor + listKind: CustomResourceMonitorList + plural: customresourcemonitors + singular: customresourcemonitor + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: CustomResourceMonitor defines monitoring for a set of custom + resources. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: MetricsSpec is the configuration describing the custom resource + state metrics to generate. + properties: + resources: + description: Resources is the list of custom resources to be monitored. + A resource with the same GroupVersionKind may appear multiple times + (e.g., to customize the namespace or subsystem,) but will incur + additional overhead. + items: + description: Resource configures a custom resource for metric generation. + properties: + commonLabels: + additionalProperties: + type: string + description: CommonLabels are added to all metrics. + type: object + errorLogV: + description: ErrorLogV defines the verbosity threshold for errors + logged for this resource. + format: int32 + type: integer + groupVersionKind: + description: GroupVersionKind of the custom resource to be monitored. + properties: + group: + type: string + kind: + type: string + version: + type: string + required: + - group + - kind + - version + type: object + labelsFromPath: + additionalProperties: + items: + type: string + type: array + description: LabelsFromPath adds additional labels where the + value is taken from a field in the resource. + type: object + metricNamePrefix: + description: 'MetricNamePrefix defines a prefix for all metrics + of the resource. If set to "", no prefix will be added. Example: + If set to "foo", MetricNamePrefix will be "foo_".' + type: string + metrics: + description: Metrics are the custom resource fields to be collected. + items: + description: Generator describes a unique metric name. + properties: + commonLabels: + additionalProperties: + type: string + description: CommonLabels are added to all metrics. + type: object + each: + description: Each targets a value or values from the resource. + properties: + gauge: + description: Gauge defines a gauge metric. + properties: + labelFromKey: + description: LabelFromKey adds a label with the + given name if Path is an object. The label value + will be the object key. + type: string + labelsFromPath: + additionalProperties: + items: + type: string + type: array + description: LabelsFromPath adds additional labels + where the value of the label is taken from a + field under Path. + type: object + nilIsZero: + description: NilIsZero indicates that if a value + is nil it will be treated as zero value. + type: boolean + path: + description: Path is the path to to generate metric(s) + for. + items: + type: string + type: array + valueFrom: + description: ValueFrom is the path to a numeric + field under Path that will be the metric value. + items: + type: string + type: array + type: object + info: + description: Info defines an info metric. + properties: + labelFromKey: + description: LabelFromKey adds a label with the + given name if Path is an object. The label value + will be the object key. + type: string + labelsFromPath: + additionalProperties: + items: + type: string + type: array + description: LabelsFromPath adds additional labels + where the value of the label is taken from a + field under Path. + type: object + path: + description: Path is the path to to generate metric(s) + for. + items: + type: string + type: array + type: object + stateSet: + description: StateSet defines a state set metric. + properties: + labelName: + description: LabelName is the key of the label + which is used for each entry in List to expose + the value. + type: string + labelsFromPath: + additionalProperties: + items: + type: string + type: array + description: LabelsFromPath adds additional labels + where the value of the label is taken from a + field under Path. + type: object + list: + description: List is the list of values to expose + a value for. + items: + type: string + type: array + path: + description: Path is the path to to generate metric(s) + for. + items: + type: string + type: array + valueFrom: + description: ValueFrom is the subpath to compare + the list to. + items: + type: string + type: array + required: + - list + type: object + type: + description: Type defines the type of the metric. + type: string + required: + - type + type: object + errorLogV: + description: ErrorLogV defines the verbosity threshold + for errors logged for this metric. Must be non-zero + to override the resource setting. + format: int32 + type: integer + help: + description: Help text for the metric. + type: string + labelsFromPath: + additionalProperties: + items: + type: string + type: array + description: LabelsFromPath adds additional labels where + the value is taken from a field in the resource. + type: object + name: + description: Name of the metric. Subject to prefixing + based on the configuration of the Resource. + type: string + required: + - each + - help + - name + type: object + type: array + resourcePlural: + description: ResourcePlural sets the plural name of the resource. + Defaults to the plural version of the Kind according to flect.Pluralize. + type: string + required: + - groupVersionKind + - metrics + type: object + type: array + required: + - resources + type: object + required: + - spec + type: object + served: true + storage: true diff --git a/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/doc.go b/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/doc.go new file mode 100644 index 0000000000..973b14ecc4 --- /dev/null +++ b/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/doc.go @@ -0,0 +1,21 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// +k8s:deepcopy-gen=package +// +k8s:defaulter-gen=TypeMeta +// +groupName=customresource.ksm.io + +package v1alpha1 diff --git a/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/register.go b/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/register.go new file mode 100644 index 0000000000..cb67f7f9b3 --- /dev/null +++ b/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/register.go @@ -0,0 +1,64 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" +) + +const ( + // GroupName is the group name for CustomResourceMonitor + // TODO(): change ksm.io to another name + GroupName = "customresource.ksm.io" +) + +// SchemeGroupVersion is the group version used to register these objects +var SchemeGroupVersion = schema.GroupVersion{Group: GroupName, Version: "v1alpha1"} + +// Resource takes an unqualified resource and returns a Group qualified GroupResource +func Resource(resource string) schema.GroupResource { + return SchemeGroupVersion.WithResource(resource).GroupResource() +} + +var ( + // SchemeBuilder is the scheme builder with scheme init functions to run for this API package + SchemeBuilder runtime.SchemeBuilder + + // AddToScheme is a global function that registers this API group & version to a scheme + AddToScheme = SchemeBuilder.AddToScheme +) + +func init() { + // We only register manually written functions here. The registration of the + // generated functions takes place in the generated files. The separation + // makes the code compile even when the generated files are missing. + // localSchemeBuilder.Register(addKnownTypes) + SchemeBuilder.Register(addKnownTypes) +} + +// Adds the list of known types to api.Scheme. +func addKnownTypes(scheme *runtime.Scheme) error { + println("## add known types") + scheme.AddKnownTypes(SchemeGroupVersion, + &CustomResourceMonitor{}, + &CustomResourceMonitorList{}, + ) + metav1.AddToGroupVersion(scheme, SchemeGroupVersion) + return nil +} diff --git a/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/types.go b/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/types.go new file mode 100644 index 0000000000..221787bd88 --- /dev/null +++ b/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/types.go @@ -0,0 +1,52 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + "k8s.io/kube-state-metrics/v2/pkg/customresourcestate" +) + +const ( + // CustomResourceMonitorKind is Kind of CustomResourceMonitor + CustomResourceMonitorKind = "CustomResourceMonitor" + // CustomResourceMonitorName is plural Name of CustomResourceMonitor + CustomResourceMonitorName = "customresourcemonitors" +) + +// CustomResourceMonitor defines monitoring for a set of custom resources. +// +genclient +// +k8s:openapi-gen=true +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object +type CustomResourceMonitor struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + // Specification of desired custom resource selection for target discovery by kube-state-metrics. + customresourcestate.Metrics `json:",inline"` +} + +// CustomResourceMonitorList is a list of CustomResourceMonitors. +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object +type CustomResourceMonitorList struct { + metav1.TypeMeta `json:",inline"` + // Standard list metadata + // More info: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#metadata + metav1.ListMeta `json:"metadata,omitempty"` + // List of CustomResourceMonitor + Items []*CustomResourceMonitor `json:"items"` +} diff --git a/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/zz_generated.deepcopy.go b/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/zz_generated.deepcopy.go new file mode 100644 index 0000000000..ff47074305 --- /dev/null +++ b/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1/zz_generated.deepcopy.go @@ -0,0 +1,72 @@ +//go:build !ignore_autogenerated +// +build !ignore_autogenerated + +// Code generated by controller-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/runtime" +) + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CustomResourceMonitor) DeepCopyInto(out *CustomResourceMonitor) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Metrics.DeepCopyInto(&out.Metrics) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CustomResourceMonitor. +func (in *CustomResourceMonitor) DeepCopy() *CustomResourceMonitor { + if in == nil { + return nil + } + out := new(CustomResourceMonitor) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CustomResourceMonitor) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CustomResourceMonitorList) DeepCopyInto(out *CustomResourceMonitorList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]*CustomResourceMonitor, len(*in)) + for i := range *in { + if (*in)[i] != nil { + in, out := &(*in)[i], &(*out)[i] + *out = new(CustomResourceMonitor) + (*in).DeepCopyInto(*out) + } + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CustomResourceMonitorList. +func (in *CustomResourceMonitorList) DeepCopy() *CustomResourceMonitorList { + if in == nil { + return nil + } + out := new(CustomResourceMonitorList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CustomResourceMonitorList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} diff --git a/pkg/customresourcemonitor/client/clientset/versioned/clientset.go b/pkg/customresourcemonitor/client/clientset/versioned/clientset.go new file mode 100644 index 0000000000..c04071933b --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/clientset.go @@ -0,0 +1,121 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package versioned + +import ( + "fmt" + "net/http" + + discovery "k8s.io/client-go/discovery" + rest "k8s.io/client-go/rest" + flowcontrol "k8s.io/client-go/util/flowcontrol" + + customresourcev1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1" +) + +type Interface interface { + Discovery() discovery.DiscoveryInterface + CustomresourceV1alpha1() customresourcev1alpha1.CustomresourceV1alpha1Interface +} + +// Clientset contains the clients for groups. +type Clientset struct { + *discovery.DiscoveryClient + customresourceV1alpha1 *customresourcev1alpha1.CustomresourceV1alpha1Client +} + +// CustomresourceV1alpha1 retrieves the CustomresourceV1alpha1Client +func (c *Clientset) CustomresourceV1alpha1() customresourcev1alpha1.CustomresourceV1alpha1Interface { + return c.customresourceV1alpha1 +} + +// Discovery retrieves the DiscoveryClient +func (c *Clientset) Discovery() discovery.DiscoveryInterface { + if c == nil { + return nil + } + return c.DiscoveryClient +} + +// NewForConfig creates a new Clientset for the given config. +// If config's RateLimiter is not set and QPS and Burst are acceptable, +// NewForConfig will generate a rate-limiter in configShallowCopy. +// NewForConfig is equivalent to NewForConfigAndClient(c, httpClient), +// where httpClient was generated with rest.HTTPClientFor(c). +func NewForConfig(c *rest.Config) (*Clientset, error) { + configShallowCopy := *c + + if configShallowCopy.UserAgent == "" { + configShallowCopy.UserAgent = rest.DefaultKubernetesUserAgent() + } + + // share the transport between all clients + httpClient, err := rest.HTTPClientFor(&configShallowCopy) + if err != nil { + return nil, err + } + + return NewForConfigAndClient(&configShallowCopy, httpClient) +} + +// NewForConfigAndClient creates a new Clientset for the given config and http client. +// Note the http client provided takes precedence over the configured transport values. +// If config's RateLimiter is not set and QPS and Burst are acceptable, +// NewForConfigAndClient will generate a rate-limiter in configShallowCopy. +func NewForConfigAndClient(c *rest.Config, httpClient *http.Client) (*Clientset, error) { + configShallowCopy := *c + if configShallowCopy.RateLimiter == nil && configShallowCopy.QPS > 0 { + if configShallowCopy.Burst <= 0 { + return nil, fmt.Errorf("burst is required to be greater than 0 when RateLimiter is not set and QPS is set to greater than 0") + } + configShallowCopy.RateLimiter = flowcontrol.NewTokenBucketRateLimiter(configShallowCopy.QPS, configShallowCopy.Burst) + } + + var cs Clientset + var err error + cs.customresourceV1alpha1, err = customresourcev1alpha1.NewForConfigAndClient(&configShallowCopy, httpClient) + if err != nil { + return nil, err + } + + cs.DiscoveryClient, err = discovery.NewDiscoveryClientForConfigAndClient(&configShallowCopy, httpClient) + if err != nil { + return nil, err + } + return &cs, nil +} + +// NewForConfigOrDie creates a new Clientset for the given config and +// panics if there is an error in the config. +func NewForConfigOrDie(c *rest.Config) *Clientset { + cs, err := NewForConfig(c) + if err != nil { + panic(err) + } + return cs +} + +// New creates a new Clientset for the given RESTClient. +func New(c rest.Interface) *Clientset { + var cs Clientset + cs.customresourceV1alpha1 = customresourcev1alpha1.New(c) + + cs.DiscoveryClient = discovery.NewDiscoveryClient(c) + return &cs +} diff --git a/pkg/customresourcemonitor/client/clientset/versioned/fake/clientset_generated.go b/pkg/customresourcemonitor/client/clientset/versioned/fake/clientset_generated.go new file mode 100644 index 0000000000..8112e6ef8d --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/fake/clientset_generated.go @@ -0,0 +1,86 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package fake + +import ( + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/watch" + "k8s.io/client-go/discovery" + fakediscovery "k8s.io/client-go/discovery/fake" + "k8s.io/client-go/testing" + + clientset "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned" + customresourcev1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1" + fakecustomresourcev1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/fake" +) + +// NewSimpleClientset returns a clientset that will respond with the provided objects. +// It's backed by a very simple object tracker that processes creates, updates and deletions as-is, +// without applying any validations and/or defaults. It shouldn't be considered a replacement +// for a real clientset and is mostly useful in simple unit tests. +func NewSimpleClientset(objects ...runtime.Object) *Clientset { + o := testing.NewObjectTracker(scheme, codecs.UniversalDecoder()) + for _, obj := range objects { + if err := o.Add(obj); err != nil { + panic(err) + } + } + + cs := &Clientset{tracker: o} + cs.discovery = &fakediscovery.FakeDiscovery{Fake: &cs.Fake} + cs.AddReactor("*", "*", testing.ObjectReaction(o)) + cs.AddWatchReactor("*", func(action testing.Action) (handled bool, ret watch.Interface, err error) { + gvr := action.GetResource() + ns := action.GetNamespace() + watch, err := o.Watch(gvr, ns) + if err != nil { + return false, nil, err + } + return true, watch, nil + }) + + return cs +} + +// Clientset implements clientset.Interface. Meant to be embedded into a +// struct to get a default implementation. This makes faking out just the method +// you want to test easier. +type Clientset struct { + testing.Fake + discovery *fakediscovery.FakeDiscovery + tracker testing.ObjectTracker +} + +func (c *Clientset) Discovery() discovery.DiscoveryInterface { + return c.discovery +} + +func (c *Clientset) Tracker() testing.ObjectTracker { + return c.tracker +} + +var ( + _ clientset.Interface = &Clientset{} + _ testing.FakeClient = &Clientset{} +) + +// CustomresourceV1alpha1 retrieves the CustomresourceV1alpha1Client +func (c *Clientset) CustomresourceV1alpha1() customresourcev1alpha1.CustomresourceV1alpha1Interface { + return &fakecustomresourcev1alpha1.FakeCustomresourceV1alpha1{Fake: &c.Fake} +} diff --git a/pkg/customresourcemonitor/client/clientset/versioned/fake/doc.go b/pkg/customresourcemonitor/client/clientset/versioned/fake/doc.go new file mode 100644 index 0000000000..9b99e71670 --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/fake/doc.go @@ -0,0 +1,20 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +// This package has the automatically generated fake clientset. +package fake diff --git a/pkg/customresourcemonitor/client/clientset/versioned/fake/register.go b/pkg/customresourcemonitor/client/clientset/versioned/fake/register.go new file mode 100644 index 0000000000..1f172e4b5c --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/fake/register.go @@ -0,0 +1,57 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package fake + +import ( + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" + schema "k8s.io/apimachinery/pkg/runtime/schema" + serializer "k8s.io/apimachinery/pkg/runtime/serializer" + utilruntime "k8s.io/apimachinery/pkg/util/runtime" + + customresourcev1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1" +) + +var scheme = runtime.NewScheme() +var codecs = serializer.NewCodecFactory(scheme) + +var localSchemeBuilder = runtime.SchemeBuilder{ + customresourcev1alpha1.AddToScheme, +} + +// AddToScheme adds all types of this clientset into the given scheme. This allows composition +// of clientsets, like in: +// +// import ( +// "k8s.io/client-go/kubernetes" +// clientsetscheme "k8s.io/client-go/kubernetes/scheme" +// aggregatorclientsetscheme "k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset/scheme" +// ) +// +// kclientset, _ := kubernetes.NewForConfig(c) +// _ = aggregatorclientsetscheme.AddToScheme(clientsetscheme.Scheme) +// +// After this, RawExtensions in Kubernetes types will serialize kube-aggregator types +// correctly. +var AddToScheme = localSchemeBuilder.AddToScheme + +func init() { + v1.AddToGroupVersion(scheme, schema.GroupVersion{Version: "v1"}) + utilruntime.Must(AddToScheme(scheme)) +} diff --git a/pkg/customresourcemonitor/client/clientset/versioned/scheme/doc.go b/pkg/customresourcemonitor/client/clientset/versioned/scheme/doc.go new file mode 100644 index 0000000000..7dc3756168 --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/scheme/doc.go @@ -0,0 +1,20 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +// This package contains the scheme of the automatically generated clientset. +package scheme diff --git a/pkg/customresourcemonitor/client/clientset/versioned/scheme/register.go b/pkg/customresourcemonitor/client/clientset/versioned/scheme/register.go new file mode 100644 index 0000000000..0d69cf92aa --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/scheme/register.go @@ -0,0 +1,57 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package scheme + +import ( + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" + schema "k8s.io/apimachinery/pkg/runtime/schema" + serializer "k8s.io/apimachinery/pkg/runtime/serializer" + utilruntime "k8s.io/apimachinery/pkg/util/runtime" + + customresourcev1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1" +) + +var Scheme = runtime.NewScheme() +var Codecs = serializer.NewCodecFactory(Scheme) +var ParameterCodec = runtime.NewParameterCodec(Scheme) +var localSchemeBuilder = runtime.SchemeBuilder{ + customresourcev1alpha1.AddToScheme, +} + +// AddToScheme adds all types of this clientset into the given scheme. This allows composition +// of clientsets, like in: +// +// import ( +// "k8s.io/client-go/kubernetes" +// clientsetscheme "k8s.io/client-go/kubernetes/scheme" +// aggregatorclientsetscheme "k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset/scheme" +// ) +// +// kclientset, _ := kubernetes.NewForConfig(c) +// _ = aggregatorclientsetscheme.AddToScheme(clientsetscheme.Scheme) +// +// After this, RawExtensions in Kubernetes types will serialize kube-aggregator types +// correctly. +var AddToScheme = localSchemeBuilder.AddToScheme + +func init() { + v1.AddToGroupVersion(Scheme, schema.GroupVersion{Version: "v1"}) + utilruntime.Must(AddToScheme(Scheme)) +} diff --git a/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/customresourcemonitor.go b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/customresourcemonitor.go new file mode 100644 index 0000000000..9516bebbb4 --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/customresourcemonitor.go @@ -0,0 +1,179 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + "context" + "time" + + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + types "k8s.io/apimachinery/pkg/types" + watch "k8s.io/apimachinery/pkg/watch" + rest "k8s.io/client-go/rest" + + v1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1" + scheme "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned/scheme" +) + +// CustomResourceMonitorsGetter has a method to return a CustomResourceMonitorInterface. +// A group's client should implement this interface. +type CustomResourceMonitorsGetter interface { + CustomResourceMonitors(namespace string) CustomResourceMonitorInterface +} + +// CustomResourceMonitorInterface has methods to work with CustomResourceMonitor resources. +type CustomResourceMonitorInterface interface { + Create(ctx context.Context, customResourceMonitor *v1alpha1.CustomResourceMonitor, opts v1.CreateOptions) (*v1alpha1.CustomResourceMonitor, error) + Update(ctx context.Context, customResourceMonitor *v1alpha1.CustomResourceMonitor, opts v1.UpdateOptions) (*v1alpha1.CustomResourceMonitor, error) + Delete(ctx context.Context, name string, opts v1.DeleteOptions) error + DeleteCollection(ctx context.Context, opts v1.DeleteOptions, listOpts v1.ListOptions) error + Get(ctx context.Context, name string, opts v1.GetOptions) (*v1alpha1.CustomResourceMonitor, error) + List(ctx context.Context, opts v1.ListOptions) (*v1alpha1.CustomResourceMonitorList, error) + Watch(ctx context.Context, opts v1.ListOptions) (watch.Interface, error) + Patch(ctx context.Context, name string, pt types.PatchType, data []byte, opts v1.PatchOptions, subresources ...string) (result *v1alpha1.CustomResourceMonitor, err error) + CustomResourceMonitorExpansion +} + +// customResourceMonitors implements CustomResourceMonitorInterface +type customResourceMonitors struct { + client rest.Interface + ns string +} + +// newCustomResourceMonitors returns a CustomResourceMonitors +func newCustomResourceMonitors(c *CustomresourceV1alpha1Client, namespace string) *customResourceMonitors { + return &customResourceMonitors{ + client: c.RESTClient(), + ns: namespace, + } +} + +// Get takes name of the customResourceMonitor, and returns the corresponding customResourceMonitor object, and an error if there is any. +func (c *customResourceMonitors) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1alpha1.CustomResourceMonitor, err error) { + result = &v1alpha1.CustomResourceMonitor{} + err = c.client.Get(). + Namespace(c.ns). + Resource("customresourcemonitors"). + Name(name). + VersionedParams(&options, scheme.ParameterCodec). + Do(ctx). + Into(result) + return +} + +// List takes label and field selectors, and returns the list of CustomResourceMonitors that match those selectors. +func (c *customResourceMonitors) List(ctx context.Context, opts v1.ListOptions) (result *v1alpha1.CustomResourceMonitorList, err error) { + var timeout time.Duration + if opts.TimeoutSeconds != nil { + timeout = time.Duration(*opts.TimeoutSeconds) * time.Second + } + result = &v1alpha1.CustomResourceMonitorList{} + err = c.client.Get(). + Namespace(c.ns). + Resource("customresourcemonitors"). + VersionedParams(&opts, scheme.ParameterCodec). + Timeout(timeout). + Do(ctx). + Into(result) + return +} + +// Watch returns a watch.Interface that watches the requested customResourceMonitors. +func (c *customResourceMonitors) Watch(ctx context.Context, opts v1.ListOptions) (watch.Interface, error) { + var timeout time.Duration + if opts.TimeoutSeconds != nil { + timeout = time.Duration(*opts.TimeoutSeconds) * time.Second + } + opts.Watch = true + return c.client.Get(). + Namespace(c.ns). + Resource("customresourcemonitors"). + VersionedParams(&opts, scheme.ParameterCodec). + Timeout(timeout). + Watch(ctx) +} + +// Create takes the representation of a customResourceMonitor and creates it. Returns the server's representation of the customResourceMonitor, and an error, if there is any. +func (c *customResourceMonitors) Create(ctx context.Context, customResourceMonitor *v1alpha1.CustomResourceMonitor, opts v1.CreateOptions) (result *v1alpha1.CustomResourceMonitor, err error) { + result = &v1alpha1.CustomResourceMonitor{} + err = c.client.Post(). + Namespace(c.ns). + Resource("customresourcemonitors"). + VersionedParams(&opts, scheme.ParameterCodec). + Body(customResourceMonitor). + Do(ctx). + Into(result) + return +} + +// Update takes the representation of a customResourceMonitor and updates it. Returns the server's representation of the customResourceMonitor, and an error, if there is any. +func (c *customResourceMonitors) Update(ctx context.Context, customResourceMonitor *v1alpha1.CustomResourceMonitor, opts v1.UpdateOptions) (result *v1alpha1.CustomResourceMonitor, err error) { + result = &v1alpha1.CustomResourceMonitor{} + err = c.client.Put(). + Namespace(c.ns). + Resource("customresourcemonitors"). + Name(customResourceMonitor.Name). + VersionedParams(&opts, scheme.ParameterCodec). + Body(customResourceMonitor). + Do(ctx). + Into(result) + return +} + +// Delete takes name of the customResourceMonitor and deletes it. Returns an error if one occurs. +func (c *customResourceMonitors) Delete(ctx context.Context, name string, opts v1.DeleteOptions) error { + return c.client.Delete(). + Namespace(c.ns). + Resource("customresourcemonitors"). + Name(name). + Body(&opts). + Do(ctx). + Error() +} + +// DeleteCollection deletes a collection of objects. +func (c *customResourceMonitors) DeleteCollection(ctx context.Context, opts v1.DeleteOptions, listOpts v1.ListOptions) error { + var timeout time.Duration + if listOpts.TimeoutSeconds != nil { + timeout = time.Duration(*listOpts.TimeoutSeconds) * time.Second + } + return c.client.Delete(). + Namespace(c.ns). + Resource("customresourcemonitors"). + VersionedParams(&listOpts, scheme.ParameterCodec). + Timeout(timeout). + Body(&opts). + Do(ctx). + Error() +} + +// Patch applies the patch and returns the patched customResourceMonitor. +func (c *customResourceMonitors) Patch(ctx context.Context, name string, pt types.PatchType, data []byte, opts v1.PatchOptions, subresources ...string) (result *v1alpha1.CustomResourceMonitor, err error) { + result = &v1alpha1.CustomResourceMonitor{} + err = c.client.Patch(pt). + Namespace(c.ns). + Resource("customresourcemonitors"). + Name(name). + SubResource(subresources...). + VersionedParams(&opts, scheme.ParameterCodec). + Body(data). + Do(ctx). + Into(result) + return +} diff --git a/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/customresourcemonitor_client.go b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/customresourcemonitor_client.go new file mode 100644 index 0000000000..3ec4ccf91c --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/customresourcemonitor_client.go @@ -0,0 +1,108 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + "net/http" + + rest "k8s.io/client-go/rest" + + v1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1" + "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned/scheme" +) + +type CustomresourceV1alpha1Interface interface { + RESTClient() rest.Interface + CustomResourceMonitorsGetter +} + +// CustomresourceV1alpha1Client is used to interact with features provided by the customresource.ksm.io group. +type CustomresourceV1alpha1Client struct { + restClient rest.Interface +} + +func (c *CustomresourceV1alpha1Client) CustomResourceMonitors(namespace string) CustomResourceMonitorInterface { + return newCustomResourceMonitors(c, namespace) +} + +// NewForConfig creates a new CustomresourceV1alpha1Client for the given config. +// NewForConfig is equivalent to NewForConfigAndClient(c, httpClient), +// where httpClient was generated with rest.HTTPClientFor(c). +func NewForConfig(c *rest.Config) (*CustomresourceV1alpha1Client, error) { + config := *c + if err := setConfigDefaults(&config); err != nil { + return nil, err + } + httpClient, err := rest.HTTPClientFor(&config) + if err != nil { + return nil, err + } + return NewForConfigAndClient(&config, httpClient) +} + +// NewForConfigAndClient creates a new CustomresourceV1alpha1Client for the given config and http client. +// Note the http client provided takes precedence over the configured transport values. +func NewForConfigAndClient(c *rest.Config, h *http.Client) (*CustomresourceV1alpha1Client, error) { + config := *c + if err := setConfigDefaults(&config); err != nil { + return nil, err + } + client, err := rest.RESTClientForConfigAndClient(&config, h) + if err != nil { + return nil, err + } + return &CustomresourceV1alpha1Client{client}, nil +} + +// NewForConfigOrDie creates a new CustomresourceV1alpha1Client for the given config and +// panics if there is an error in the config. +func NewForConfigOrDie(c *rest.Config) *CustomresourceV1alpha1Client { + client, err := NewForConfig(c) + if err != nil { + panic(err) + } + return client +} + +// New creates a new CustomresourceV1alpha1Client for the given RESTClient. +func New(c rest.Interface) *CustomresourceV1alpha1Client { + return &CustomresourceV1alpha1Client{c} +} + +func setConfigDefaults(config *rest.Config) error { + gv := v1alpha1.SchemeGroupVersion + config.GroupVersion = &gv + config.APIPath = "/apis" + config.NegotiatedSerializer = scheme.Codecs.WithoutConversion() + + if config.UserAgent == "" { + config.UserAgent = rest.DefaultKubernetesUserAgent() + } + + return nil +} + +// RESTClient returns a RESTClient that is used to communicate +// with API server by this client implementation. +func (c *CustomresourceV1alpha1Client) RESTClient() rest.Interface { + if c == nil { + return nil + } + return c.restClient +} diff --git a/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/doc.go b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/doc.go new file mode 100644 index 0000000000..df51baa4d4 --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/doc.go @@ -0,0 +1,20 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +// This package has the automatically generated typed clients. +package v1alpha1 diff --git a/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/fake/doc.go b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/fake/doc.go new file mode 100644 index 0000000000..16f4439906 --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/fake/doc.go @@ -0,0 +1,20 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +// Package fake has the automatically generated clients. +package fake diff --git a/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/fake/fake_customresourcemonitor.go b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/fake/fake_customresourcemonitor.go new file mode 100644 index 0000000000..fe06d2aa03 --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/fake/fake_customresourcemonitor.go @@ -0,0 +1,130 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package fake + +import ( + "context" + + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + labels "k8s.io/apimachinery/pkg/labels" + types "k8s.io/apimachinery/pkg/types" + watch "k8s.io/apimachinery/pkg/watch" + testing "k8s.io/client-go/testing" + + v1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1" +) + +// FakeCustomResourceMonitors implements CustomResourceMonitorInterface +type FakeCustomResourceMonitors struct { + Fake *FakeCustomresourceV1alpha1 + ns string +} + +var customresourcemonitorsResource = v1alpha1.SchemeGroupVersion.WithResource("customresourcemonitors") + +var customresourcemonitorsKind = v1alpha1.SchemeGroupVersion.WithKind("CustomResourceMonitor") + +// Get takes name of the customResourceMonitor, and returns the corresponding customResourceMonitor object, and an error if there is any. +func (c *FakeCustomResourceMonitors) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1alpha1.CustomResourceMonitor, err error) { + obj, err := c.Fake. + Invokes(testing.NewGetAction(customresourcemonitorsResource, c.ns, name), &v1alpha1.CustomResourceMonitor{}) + + if obj == nil { + return nil, err + } + return obj.(*v1alpha1.CustomResourceMonitor), err +} + +// List takes label and field selectors, and returns the list of CustomResourceMonitors that match those selectors. +func (c *FakeCustomResourceMonitors) List(ctx context.Context, opts v1.ListOptions) (result *v1alpha1.CustomResourceMonitorList, err error) { + obj, err := c.Fake. + Invokes(testing.NewListAction(customresourcemonitorsResource, customresourcemonitorsKind, c.ns, opts), &v1alpha1.CustomResourceMonitorList{}) + + if obj == nil { + return nil, err + } + + label, _, _ := testing.ExtractFromListOptions(opts) + if label == nil { + label = labels.Everything() + } + list := &v1alpha1.CustomResourceMonitorList{ListMeta: obj.(*v1alpha1.CustomResourceMonitorList).ListMeta} + for _, item := range obj.(*v1alpha1.CustomResourceMonitorList).Items { + if label.Matches(labels.Set(item.Labels)) { + list.Items = append(list.Items, item) + } + } + return list, err +} + +// Watch returns a watch.Interface that watches the requested customResourceMonitors. +func (c *FakeCustomResourceMonitors) Watch(ctx context.Context, opts v1.ListOptions) (watch.Interface, error) { + return c.Fake. + InvokesWatch(testing.NewWatchAction(customresourcemonitorsResource, c.ns, opts)) + +} + +// Create takes the representation of a customResourceMonitor and creates it. Returns the server's representation of the customResourceMonitor, and an error, if there is any. +func (c *FakeCustomResourceMonitors) Create(ctx context.Context, customResourceMonitor *v1alpha1.CustomResourceMonitor, opts v1.CreateOptions) (result *v1alpha1.CustomResourceMonitor, err error) { + obj, err := c.Fake. + Invokes(testing.NewCreateAction(customresourcemonitorsResource, c.ns, customResourceMonitor), &v1alpha1.CustomResourceMonitor{}) + + if obj == nil { + return nil, err + } + return obj.(*v1alpha1.CustomResourceMonitor), err +} + +// Update takes the representation of a customResourceMonitor and updates it. Returns the server's representation of the customResourceMonitor, and an error, if there is any. +func (c *FakeCustomResourceMonitors) Update(ctx context.Context, customResourceMonitor *v1alpha1.CustomResourceMonitor, opts v1.UpdateOptions) (result *v1alpha1.CustomResourceMonitor, err error) { + obj, err := c.Fake. + Invokes(testing.NewUpdateAction(customresourcemonitorsResource, c.ns, customResourceMonitor), &v1alpha1.CustomResourceMonitor{}) + + if obj == nil { + return nil, err + } + return obj.(*v1alpha1.CustomResourceMonitor), err +} + +// Delete takes name of the customResourceMonitor and deletes it. Returns an error if one occurs. +func (c *FakeCustomResourceMonitors) Delete(ctx context.Context, name string, opts v1.DeleteOptions) error { + _, err := c.Fake. + Invokes(testing.NewDeleteActionWithOptions(customresourcemonitorsResource, c.ns, name, opts), &v1alpha1.CustomResourceMonitor{}) + + return err +} + +// DeleteCollection deletes a collection of objects. +func (c *FakeCustomResourceMonitors) DeleteCollection(ctx context.Context, opts v1.DeleteOptions, listOpts v1.ListOptions) error { + action := testing.NewDeleteCollectionAction(customresourcemonitorsResource, c.ns, listOpts) + + _, err := c.Fake.Invokes(action, &v1alpha1.CustomResourceMonitorList{}) + return err +} + +// Patch applies the patch and returns the patched customResourceMonitor. +func (c *FakeCustomResourceMonitors) Patch(ctx context.Context, name string, pt types.PatchType, data []byte, opts v1.PatchOptions, subresources ...string) (result *v1alpha1.CustomResourceMonitor, err error) { + obj, err := c.Fake. + Invokes(testing.NewPatchSubresourceAction(customresourcemonitorsResource, c.ns, name, pt, data, subresources...), &v1alpha1.CustomResourceMonitor{}) + + if obj == nil { + return nil, err + } + return obj.(*v1alpha1.CustomResourceMonitor), err +} diff --git a/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/fake/fake_customresourcemonitor_client.go b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/fake/fake_customresourcemonitor_client.go new file mode 100644 index 0000000000..80470a029f --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/fake/fake_customresourcemonitor_client.go @@ -0,0 +1,41 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package fake + +import ( + rest "k8s.io/client-go/rest" + testing "k8s.io/client-go/testing" + + v1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1" +) + +type FakeCustomresourceV1alpha1 struct { + *testing.Fake +} + +func (c *FakeCustomresourceV1alpha1) CustomResourceMonitors(namespace string) v1alpha1.CustomResourceMonitorInterface { + return &FakeCustomResourceMonitors{c, namespace} +} + +// RESTClient returns a RESTClient that is used to communicate +// with API server by this client implementation. +func (c *FakeCustomresourceV1alpha1) RESTClient() rest.Interface { + var ret *rest.RESTClient + return ret +} diff --git a/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/generated_expansion.go b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/generated_expansion.go new file mode 100644 index 0000000000..5405868b2d --- /dev/null +++ b/pkg/customresourcemonitor/client/clientset/versioned/typed/customresourcemonitor/v1alpha1/generated_expansion.go @@ -0,0 +1,21 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package v1alpha1 + +type CustomResourceMonitorExpansion interface{} diff --git a/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor/interface.go b/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor/interface.go new file mode 100644 index 0000000000..58ce6c0f85 --- /dev/null +++ b/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor/interface.go @@ -0,0 +1,46 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by informer-gen. DO NOT EDIT. + +package customresourcemonitor + +import ( + v1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor/v1alpha1" + internalinterfaces "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/informers/externalversions/internalinterfaces" +) + +// Interface provides access to each of this group's versions. +type Interface interface { + // V1alpha1 provides access to shared informers for resources in V1alpha1. + V1alpha1() v1alpha1.Interface +} + +type group struct { + factory internalinterfaces.SharedInformerFactory + namespace string + tweakListOptions internalinterfaces.TweakListOptionsFunc +} + +// New returns a new Interface. +func New(f internalinterfaces.SharedInformerFactory, namespace string, tweakListOptions internalinterfaces.TweakListOptionsFunc) Interface { + return &group{factory: f, namespace: namespace, tweakListOptions: tweakListOptions} +} + +// V1alpha1 returns a new v1alpha1.Interface. +func (g *group) V1alpha1() v1alpha1.Interface { + return v1alpha1.New(g.factory, g.namespace, g.tweakListOptions) +} diff --git a/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor/v1alpha1/customresourcemonitor.go b/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor/v1alpha1/customresourcemonitor.go new file mode 100644 index 0000000000..39f8e5685d --- /dev/null +++ b/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor/v1alpha1/customresourcemonitor.go @@ -0,0 +1,91 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by informer-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + "context" + time "time" + + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" + watch "k8s.io/apimachinery/pkg/watch" + cache "k8s.io/client-go/tools/cache" + + customresourcemonitorv1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1" + versioned "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned" + internalinterfaces "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/informers/externalversions/internalinterfaces" + v1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/listers/customresourcemonitor/v1alpha1" +) + +// CustomResourceMonitorInformer provides access to a shared informer and lister for +// CustomResourceMonitors. +type CustomResourceMonitorInformer interface { + Informer() cache.SharedIndexInformer + Lister() v1alpha1.CustomResourceMonitorLister +} + +type customResourceMonitorInformer struct { + factory internalinterfaces.SharedInformerFactory + tweakListOptions internalinterfaces.TweakListOptionsFunc + namespace string +} + +// NewCustomResourceMonitorInformer constructs a new informer for CustomResourceMonitor type. +// Always prefer using an informer factory to get a shared informer instead of getting an independent +// one. This reduces memory footprint and number of connections to the server. +func NewCustomResourceMonitorInformer(client versioned.Interface, namespace string, resyncPeriod time.Duration, indexers cache.Indexers) cache.SharedIndexInformer { + return NewFilteredCustomResourceMonitorInformer(client, namespace, resyncPeriod, indexers, nil) +} + +// NewFilteredCustomResourceMonitorInformer constructs a new informer for CustomResourceMonitor type. +// Always prefer using an informer factory to get a shared informer instead of getting an independent +// one. This reduces memory footprint and number of connections to the server. +func NewFilteredCustomResourceMonitorInformer(client versioned.Interface, namespace string, resyncPeriod time.Duration, indexers cache.Indexers, tweakListOptions internalinterfaces.TweakListOptionsFunc) cache.SharedIndexInformer { + return cache.NewSharedIndexInformer( + &cache.ListWatch{ + ListFunc: func(options v1.ListOptions) (runtime.Object, error) { + if tweakListOptions != nil { + tweakListOptions(&options) + } + return client.CustomresourceV1alpha1().CustomResourceMonitors(namespace).List(context.TODO(), options) + }, + WatchFunc: func(options v1.ListOptions) (watch.Interface, error) { + if tweakListOptions != nil { + tweakListOptions(&options) + } + return client.CustomresourceV1alpha1().CustomResourceMonitors(namespace).Watch(context.TODO(), options) + }, + }, + &customresourcemonitorv1alpha1.CustomResourceMonitor{}, + resyncPeriod, + indexers, + ) +} + +func (f *customResourceMonitorInformer) defaultInformer(client versioned.Interface, resyncPeriod time.Duration) cache.SharedIndexInformer { + return NewFilteredCustomResourceMonitorInformer(client, f.namespace, resyncPeriod, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc}, f.tweakListOptions) +} + +func (f *customResourceMonitorInformer) Informer() cache.SharedIndexInformer { + return f.factory.InformerFor(&customresourcemonitorv1alpha1.CustomResourceMonitor{}, f.defaultInformer) +} + +func (f *customResourceMonitorInformer) Lister() v1alpha1.CustomResourceMonitorLister { + return v1alpha1.NewCustomResourceMonitorLister(f.Informer().GetIndexer()) +} diff --git a/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor/v1alpha1/interface.go b/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor/v1alpha1/interface.go new file mode 100644 index 0000000000..a5345b7481 --- /dev/null +++ b/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor/v1alpha1/interface.go @@ -0,0 +1,45 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by informer-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + internalinterfaces "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/informers/externalversions/internalinterfaces" +) + +// Interface provides access to all the informers in this group version. +type Interface interface { + // CustomResourceMonitors returns a CustomResourceMonitorInformer. + CustomResourceMonitors() CustomResourceMonitorInformer +} + +type version struct { + factory internalinterfaces.SharedInformerFactory + namespace string + tweakListOptions internalinterfaces.TweakListOptionsFunc +} + +// New returns a new Interface. +func New(f internalinterfaces.SharedInformerFactory, namespace string, tweakListOptions internalinterfaces.TweakListOptionsFunc) Interface { + return &version{factory: f, namespace: namespace, tweakListOptions: tweakListOptions} +} + +// CustomResourceMonitors returns a CustomResourceMonitorInformer. +func (v *version) CustomResourceMonitors() CustomResourceMonitorInformer { + return &customResourceMonitorInformer{factory: v.factory, namespace: v.namespace, tweakListOptions: v.tweakListOptions} +} diff --git a/pkg/customresourcemonitor/client/informers/externalversions/factory.go b/pkg/customresourcemonitor/client/informers/externalversions/factory.go new file mode 100644 index 0000000000..93828eee80 --- /dev/null +++ b/pkg/customresourcemonitor/client/informers/externalversions/factory.go @@ -0,0 +1,252 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by informer-gen. DO NOT EDIT. + +package externalversions + +import ( + reflect "reflect" + sync "sync" + time "time" + + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" + schema "k8s.io/apimachinery/pkg/runtime/schema" + cache "k8s.io/client-go/tools/cache" + + versioned "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned" + customresourcemonitor "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/informers/externalversions/customresourcemonitor" + internalinterfaces "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/informers/externalversions/internalinterfaces" +) + +// SharedInformerOption defines the functional option type for SharedInformerFactory. +type SharedInformerOption func(*sharedInformerFactory) *sharedInformerFactory + +type sharedInformerFactory struct { + client versioned.Interface + namespace string + tweakListOptions internalinterfaces.TweakListOptionsFunc + lock sync.Mutex + defaultResync time.Duration + customResync map[reflect.Type]time.Duration + + informers map[reflect.Type]cache.SharedIndexInformer + // startedInformers is used for tracking which informers have been started. + // This allows Start() to be called multiple times safely. + startedInformers map[reflect.Type]bool + // wg tracks how many goroutines were started. + wg sync.WaitGroup + // shuttingDown is true when Shutdown has been called. It may still be running + // because it needs to wait for goroutines. + shuttingDown bool +} + +// WithCustomResyncConfig sets a custom resync period for the specified informer types. +func WithCustomResyncConfig(resyncConfig map[v1.Object]time.Duration) SharedInformerOption { + return func(factory *sharedInformerFactory) *sharedInformerFactory { + for k, v := range resyncConfig { + factory.customResync[reflect.TypeOf(k)] = v + } + return factory + } +} + +// WithTweakListOptions sets a custom filter on all listers of the configured SharedInformerFactory. +func WithTweakListOptions(tweakListOptions internalinterfaces.TweakListOptionsFunc) SharedInformerOption { + return func(factory *sharedInformerFactory) *sharedInformerFactory { + factory.tweakListOptions = tweakListOptions + return factory + } +} + +// WithNamespace limits the SharedInformerFactory to the specified namespace. +func WithNamespace(namespace string) SharedInformerOption { + return func(factory *sharedInformerFactory) *sharedInformerFactory { + factory.namespace = namespace + return factory + } +} + +// NewSharedInformerFactory constructs a new instance of sharedInformerFactory for all namespaces. +func NewSharedInformerFactory(client versioned.Interface, defaultResync time.Duration) SharedInformerFactory { + return NewSharedInformerFactoryWithOptions(client, defaultResync) +} + +// NewFilteredSharedInformerFactory constructs a new instance of sharedInformerFactory. +// Listers obtained via this SharedInformerFactory will be subject to the same filters +// as specified here. +// Deprecated: Please use NewSharedInformerFactoryWithOptions instead +func NewFilteredSharedInformerFactory(client versioned.Interface, defaultResync time.Duration, namespace string, tweakListOptions internalinterfaces.TweakListOptionsFunc) SharedInformerFactory { + return NewSharedInformerFactoryWithOptions(client, defaultResync, WithNamespace(namespace), WithTweakListOptions(tweakListOptions)) +} + +// NewSharedInformerFactoryWithOptions constructs a new instance of a SharedInformerFactory with additional options. +func NewSharedInformerFactoryWithOptions(client versioned.Interface, defaultResync time.Duration, options ...SharedInformerOption) SharedInformerFactory { + factory := &sharedInformerFactory{ + client: client, + namespace: v1.NamespaceAll, + defaultResync: defaultResync, + informers: make(map[reflect.Type]cache.SharedIndexInformer), + startedInformers: make(map[reflect.Type]bool), + customResync: make(map[reflect.Type]time.Duration), + } + + // Apply all options + for _, opt := range options { + factory = opt(factory) + } + + return factory +} + +func (f *sharedInformerFactory) Start(stopCh <-chan struct{}) { + f.lock.Lock() + defer f.lock.Unlock() + + if f.shuttingDown { + return + } + + for informerType, informer := range f.informers { + if !f.startedInformers[informerType] { + f.wg.Add(1) + // We need a new variable in each loop iteration, + // otherwise the goroutine would use the loop variable + // and that keeps changing. + informer := informer + go func() { + defer f.wg.Done() + informer.Run(stopCh) + }() + f.startedInformers[informerType] = true + } + } +} + +func (f *sharedInformerFactory) Shutdown() { + f.lock.Lock() + f.shuttingDown = true + f.lock.Unlock() + + // Will return immediately if there is nothing to wait for. + f.wg.Wait() +} + +func (f *sharedInformerFactory) WaitForCacheSync(stopCh <-chan struct{}) map[reflect.Type]bool { + informers := func() map[reflect.Type]cache.SharedIndexInformer { + f.lock.Lock() + defer f.lock.Unlock() + + informers := map[reflect.Type]cache.SharedIndexInformer{} + for informerType, informer := range f.informers { + if f.startedInformers[informerType] { + informers[informerType] = informer + } + } + return informers + }() + + res := map[reflect.Type]bool{} + for informType, informer := range informers { + res[informType] = cache.WaitForCacheSync(stopCh, informer.HasSynced) + } + return res +} + +// InternalInformerFor returns the SharedIndexInformer for obj using an internal +// client. +func (f *sharedInformerFactory) InformerFor(obj runtime.Object, newFunc internalinterfaces.NewInformerFunc) cache.SharedIndexInformer { + f.lock.Lock() + defer f.lock.Unlock() + + informerType := reflect.TypeOf(obj) + informer, exists := f.informers[informerType] + if exists { + return informer + } + + resyncPeriod, exists := f.customResync[informerType] + if !exists { + resyncPeriod = f.defaultResync + } + + informer = newFunc(f.client, resyncPeriod) + f.informers[informerType] = informer + + return informer +} + +// SharedInformerFactory provides shared informers for resources in all known +// API group versions. +// +// It is typically used like this: +// +// ctx, cancel := context.Background() +// defer cancel() +// factory := NewSharedInformerFactory(client, resyncPeriod) +// defer factory.WaitForStop() // Returns immediately if nothing was started. +// genericInformer := factory.ForResource(resource) +// typedInformer := factory.SomeAPIGroup().V1().SomeType() +// factory.Start(ctx.Done()) // Start processing these informers. +// synced := factory.WaitForCacheSync(ctx.Done()) +// for v, ok := range synced { +// if !ok { +// fmt.Fprintf(os.Stderr, "caches failed to sync: %v", v) +// return +// } +// } +// +// // Creating informers can also be created after Start, but then +// // Start must be called again: +// anotherGenericInformer := factory.ForResource(resource) +// factory.Start(ctx.Done()) +type SharedInformerFactory interface { + internalinterfaces.SharedInformerFactory + + // Start initializes all requested informers. They are handled in goroutines + // which run until the stop channel gets closed. + Start(stopCh <-chan struct{}) + + // Shutdown marks a factory as shutting down. At that point no new + // informers can be started anymore and Start will return without + // doing anything. + // + // In addition, Shutdown blocks until all goroutines have terminated. For that + // to happen, the close channel(s) that they were started with must be closed, + // either before Shutdown gets called or while it is waiting. + // + // Shutdown may be called multiple times, even concurrently. All such calls will + // block until all goroutines have terminated. + Shutdown() + + // WaitForCacheSync blocks until all started informers' caches were synced + // or the stop channel gets closed. + WaitForCacheSync(stopCh <-chan struct{}) map[reflect.Type]bool + + // ForResource gives generic access to a shared informer of the matching type. + ForResource(resource schema.GroupVersionResource) (GenericInformer, error) + + // InternalInformerFor returns the SharedIndexInformer for obj using an internal + // client. + InformerFor(obj runtime.Object, newFunc internalinterfaces.NewInformerFunc) cache.SharedIndexInformer + + Customresource() customresourcemonitor.Interface +} + +func (f *sharedInformerFactory) Customresource() customresourcemonitor.Interface { + return customresourcemonitor.New(f, f.namespace, f.tweakListOptions) +} diff --git a/pkg/customresourcemonitor/client/informers/externalversions/generic.go b/pkg/customresourcemonitor/client/informers/externalversions/generic.go new file mode 100644 index 0000000000..7c4a901d58 --- /dev/null +++ b/pkg/customresourcemonitor/client/informers/externalversions/generic.go @@ -0,0 +1,63 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by informer-gen. DO NOT EDIT. + +package externalversions + +import ( + "fmt" + + schema "k8s.io/apimachinery/pkg/runtime/schema" + cache "k8s.io/client-go/tools/cache" + + v1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1" +) + +// GenericInformer is type of SharedIndexInformer which will locate and delegate to other +// sharedInformers based on type +type GenericInformer interface { + Informer() cache.SharedIndexInformer + Lister() cache.GenericLister +} + +type genericInformer struct { + informer cache.SharedIndexInformer + resource schema.GroupResource +} + +// Informer returns the SharedIndexInformer. +func (f *genericInformer) Informer() cache.SharedIndexInformer { + return f.informer +} + +// Lister returns the GenericLister. +func (f *genericInformer) Lister() cache.GenericLister { + return cache.NewGenericLister(f.Informer().GetIndexer(), f.resource) +} + +// ForResource gives generic access to a shared informer of the matching type +// TODO extend this to unknown resources with a client pool +func (f *sharedInformerFactory) ForResource(resource schema.GroupVersionResource) (GenericInformer, error) { + switch resource { + // Group=customresource.ksm.io, Version=v1alpha1 + case v1alpha1.SchemeGroupVersion.WithResource("customresourcemonitors"): + return &genericInformer{resource: resource.GroupResource(), informer: f.Customresource().V1alpha1().CustomResourceMonitors().Informer()}, nil + + } + + return nil, fmt.Errorf("no informer found for %v", resource) +} diff --git a/pkg/customresourcemonitor/client/informers/externalversions/internalinterfaces/factory_interfaces.go b/pkg/customresourcemonitor/client/informers/externalversions/internalinterfaces/factory_interfaces.go new file mode 100644 index 0000000000..e213970b36 --- /dev/null +++ b/pkg/customresourcemonitor/client/informers/externalversions/internalinterfaces/factory_interfaces.go @@ -0,0 +1,41 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by informer-gen. DO NOT EDIT. + +package internalinterfaces + +import ( + time "time" + + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" + cache "k8s.io/client-go/tools/cache" + + versioned "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned" +) + +// NewInformerFunc takes versioned.Interface and time.Duration to return a SharedIndexInformer. +type NewInformerFunc func(versioned.Interface, time.Duration) cache.SharedIndexInformer + +// SharedInformerFactory a small interface to allow for adding an informer without an import cycle +type SharedInformerFactory interface { + Start(stopCh <-chan struct{}) + InformerFor(obj runtime.Object, newFunc NewInformerFunc) cache.SharedIndexInformer +} + +// TweakListOptionsFunc is a function that transforms a v1.ListOptions. +type TweakListOptionsFunc func(*v1.ListOptions) diff --git a/pkg/customresourcemonitor/client/listers/customresourcemonitor/v1alpha1/customresourcemonitor.go b/pkg/customresourcemonitor/client/listers/customresourcemonitor/v1alpha1/customresourcemonitor.go new file mode 100644 index 0000000000..f5a77098d2 --- /dev/null +++ b/pkg/customresourcemonitor/client/listers/customresourcemonitor/v1alpha1/customresourcemonitor.go @@ -0,0 +1,100 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by lister-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/labels" + "k8s.io/client-go/tools/cache" + + v1alpha1 "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/apis/customresourcemonitor/v1alpha1" +) + +// CustomResourceMonitorLister helps list CustomResourceMonitors. +// All objects returned here must be treated as read-only. +type CustomResourceMonitorLister interface { + // List lists all CustomResourceMonitors in the indexer. + // Objects returned here must be treated as read-only. + List(selector labels.Selector) (ret []*v1alpha1.CustomResourceMonitor, err error) + // CustomResourceMonitors returns an object that can list and get CustomResourceMonitors. + CustomResourceMonitors(namespace string) CustomResourceMonitorNamespaceLister + CustomResourceMonitorListerExpansion +} + +// customResourceMonitorLister implements the CustomResourceMonitorLister interface. +type customResourceMonitorLister struct { + indexer cache.Indexer +} + +// NewCustomResourceMonitorLister returns a new CustomResourceMonitorLister. +func NewCustomResourceMonitorLister(indexer cache.Indexer) CustomResourceMonitorLister { + return &customResourceMonitorLister{indexer: indexer} +} + +// List lists all CustomResourceMonitors in the indexer. +func (s *customResourceMonitorLister) List(selector labels.Selector) (ret []*v1alpha1.CustomResourceMonitor, err error) { + err = cache.ListAll(s.indexer, selector, func(m interface{}) { + ret = append(ret, m.(*v1alpha1.CustomResourceMonitor)) + }) + return ret, err +} + +// CustomResourceMonitors returns an object that can list and get CustomResourceMonitors. +func (s *customResourceMonitorLister) CustomResourceMonitors(namespace string) CustomResourceMonitorNamespaceLister { + return customResourceMonitorNamespaceLister{indexer: s.indexer, namespace: namespace} +} + +// CustomResourceMonitorNamespaceLister helps list and get CustomResourceMonitors. +// All objects returned here must be treated as read-only. +type CustomResourceMonitorNamespaceLister interface { + // List lists all CustomResourceMonitors in the indexer for a given namespace. + // Objects returned here must be treated as read-only. + List(selector labels.Selector) (ret []*v1alpha1.CustomResourceMonitor, err error) + // Get retrieves the CustomResourceMonitor from the indexer for a given namespace and name. + // Objects returned here must be treated as read-only. + Get(name string) (*v1alpha1.CustomResourceMonitor, error) + CustomResourceMonitorNamespaceListerExpansion +} + +// customResourceMonitorNamespaceLister implements the CustomResourceMonitorNamespaceLister +// interface. +type customResourceMonitorNamespaceLister struct { + indexer cache.Indexer + namespace string +} + +// List lists all CustomResourceMonitors in the indexer for a given namespace. +func (s customResourceMonitorNamespaceLister) List(selector labels.Selector) (ret []*v1alpha1.CustomResourceMonitor, err error) { + err = cache.ListAllByNamespace(s.indexer, s.namespace, selector, func(m interface{}) { + ret = append(ret, m.(*v1alpha1.CustomResourceMonitor)) + }) + return ret, err +} + +// Get retrieves the CustomResourceMonitor from the indexer for a given namespace and name. +func (s customResourceMonitorNamespaceLister) Get(name string) (*v1alpha1.CustomResourceMonitor, error) { + obj, exists, err := s.indexer.GetByKey(s.namespace + "/" + name) + if err != nil { + return nil, err + } + if !exists { + return nil, errors.NewNotFound(v1alpha1.Resource("customresourcemonitor"), name) + } + return obj.(*v1alpha1.CustomResourceMonitor), nil +} diff --git a/pkg/customresourcemonitor/client/listers/customresourcemonitor/v1alpha1/expansion_generated.go b/pkg/customresourcemonitor/client/listers/customresourcemonitor/v1alpha1/expansion_generated.go new file mode 100644 index 0000000000..116343e1eb --- /dev/null +++ b/pkg/customresourcemonitor/client/listers/customresourcemonitor/v1alpha1/expansion_generated.go @@ -0,0 +1,27 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by lister-gen. DO NOT EDIT. + +package v1alpha1 + +// CustomResourceMonitorListerExpansion allows custom methods to be added to +// CustomResourceMonitorLister. +type CustomResourceMonitorListerExpansion interface{} + +// CustomResourceMonitorNamespaceListerExpansion allows custom methods to be added to +// CustomResourceMonitorNamespaceLister. +type CustomResourceMonitorNamespaceListerExpansion interface{} diff --git a/pkg/customresourcestate/config.go b/pkg/customresourcestate/config.go index 873591372f..257da99cb6 100644 --- a/pkg/customresourcestate/config.go +++ b/pkg/customresourcestate/config.go @@ -28,7 +28,6 @@ import ( "k8s.io/kube-state-metrics/v2/internal/discovery" "k8s.io/kube-state-metrics/v2/pkg/customresource" - "k8s.io/kube-state-metrics/v2/pkg/util" ) // customResourceState is used to prefix the auto-generated GVK labels as well as an appendix for the metric itself @@ -36,11 +35,13 @@ import ( const customResourceState string = "customresource" // Metrics is the top level configuration object. +// +k8s:deepcopy-gen=true type Metrics struct { Spec MetricsSpec `yaml:"spec" json:"spec"` } // MetricsSpec is the configuration describing the custom resource state metrics to generate. +// +k8s:deepcopy-gen=true type MetricsSpec struct { // Resources is the list of custom resources to be monitored. A resource with the same GroupVersionKind may appear // multiple times (e.g., to customize the namespace or subsystem,) but will incur additional overhead. @@ -48,24 +49,29 @@ type MetricsSpec struct { } // Resource configures a custom resource for metric generation. +// +k8s:deepcopy-gen=true type Resource struct { // MetricNamePrefix defines a prefix for all metrics of the resource. // If set to "", no prefix will be added. // Example: If set to "foo", MetricNamePrefix will be "foo_". + // +optional MetricNamePrefix *string `yaml:"metricNamePrefix" json:"metricNamePrefix"` // GroupVersionKind of the custom resource to be monitored. GroupVersionKind GroupVersionKind `yaml:"groupVersionKind" json:"groupVersionKind"` // Labels are added to all metrics. If the same key is used in a metric, the value from the metric will overwrite the value here. + // +optional Labels `yaml:",inline" json:",inline"` // Metrics are the custom resource fields to be collected. Metrics []Generator `yaml:"metrics" json:"metrics"` // ErrorLogV defines the verbosity threshold for errors logged for this resource. + // +optional ErrorLogV klog.Level `yaml:"errorLogV" json:"errorLogV"` // ResourcePlural sets the plural name of the resource. Defaults to the plural version of the Kind according to flect.Pluralize. + // +optional ResourcePlural string `yaml:"resourcePlural" json:"resourcePlural"` } @@ -100,10 +106,13 @@ func (gvk GroupVersionKind) String() string { } // Labels is common configuration of labels to add to metrics. +// +k8s:deepcopy-gen=true type Labels struct { // CommonLabels are added to all metrics. + // +optional CommonLabels map[string]string `yaml:"commonLabels" json:"commonLabels"` // LabelsFromPath adds additional labels where the value is taken from a field in the resource. + // +optional LabelsFromPath map[string][]string `yaml:"labelsFromPath" json:"labelsFromPath"` } @@ -131,6 +140,7 @@ func (l Labels) Merge(other Labels) Labels { } // Generator describes a unique metric name. +// +k8s:deepcopy-gen=true type Generator struct { // Name of the metric. Subject to prefixing based on the configuration of the Resource. Name string `yaml:"name" json:"name"` @@ -140,13 +150,16 @@ type Generator struct { Each Metric `yaml:"each" json:"each"` // Labels are added to all metrics. Labels from Each will overwrite these if using the same key. + // +optional Labels `yaml:",inline" json:",inline"` // json will inline because it is already tagged // ErrorLogV defines the verbosity threshold for errors logged for this metric. Must be non-zero to override the resource setting. + // +optional ErrorLogV klog.Level `yaml:"errorLogV" json:"errorLogV"` } // Metric defines a metric to expose. // +union +// +k8s:deepcopy-gen=true type Metric struct { // Type defines the type of the metric. // +unionDiscriminator @@ -168,8 +181,13 @@ type ConfigDecoder interface { Decode(v interface{}) (err error) } +// GVKToGVKP interface implements ResolveGVKToGVKPs +type GVKToGVKP interface { + ResolveGVKToGVKPs(gvk schema.GroupVersionKind) (resolvedGVKPs []discovery.GroupVersionKindPlural, err error) +} + // FromConfig decodes a configuration source into a slice of `customresource.RegistryFactory` that are ready to use. -func FromConfig(decoder ConfigDecoder, discovererInstance *discovery.CRDiscoverer) (func() ([]customresource.RegistryFactory, error), error) { +func FromConfig(decoder ConfigDecoder, discovererInstance GVKToGVKP) (func() ([]customresource.RegistryFactory, error), error) { var customResourceConfig Metrics factoriesIndex := map[string]bool{} @@ -204,7 +222,7 @@ func FromConfig(decoder ConfigDecoder, discovererInstance *discovery.CRDiscovere if err != nil { return nil, fmt.Errorf("failed to create metrics factory for %s: %w", resource.GroupVersionKind, err) } - gvrString := util.GVRFromType(factory.Name(), factory.ExpectedType()).String() + gvrString := customresource.GVRFromType(factory.Name(), factory.ExpectedType()).String() if _, ok := factoriesIndex[gvrString]; ok { klog.InfoS("reloaded factory", "GVR", gvrString) } @@ -227,3 +245,26 @@ func configOverrides(config *Metrics) { } } } + +// FromConfig decodes a configuration source into a slice of customresource.RegistryFactory that are ready to use. +// FromConfig2 decodes a configuration source into a slice of customresource.RegistryFactory that are ready to use. +func FromConfig2(decoder ConfigDecoder) ([]customresource.RegistryFactory, error) { + var crconfig Metrics + var factories []customresource.RegistryFactory + factoriesIndex := map[string]bool{} + if err := decoder.Decode(&crconfig); err != nil { + return nil, fmt.Errorf("failed to parse Custom Resource State metrics: %w", err) + } + for _, resource := range crconfig.Spec.Resources { + factory, err := NewCustomResourceMetrics(resource) + if err != nil { + return nil, fmt.Errorf("failed to create metrics factory for %s: %w", resource.GroupVersionKind, err) + } + if _, ok := factoriesIndex[factory.Name()]; ok { + return nil, fmt.Errorf("found multiple custom resource configurations for the same resource %s", factory.Name()) + } + factoriesIndex[factory.Name()] = true + factories = append(factories, factory) + } + return factories, nil +} diff --git a/pkg/customresourcestate/config_metrics_types.go b/pkg/customresourcestate/config_metrics_types.go index 25d63bfb69..84494dd8f8 100644 --- a/pkg/customresourcestate/config_metrics_types.go +++ b/pkg/customresourcestate/config_metrics_types.go @@ -17,43 +17,55 @@ limitations under the License. package customresourcestate // MetricMeta are variables which may used for any metric type. +// +k8s:deepcopy-gen=true type MetricMeta struct { // LabelsFromPath adds additional labels where the value of the label is taken from a field under Path. + // +optional LabelsFromPath map[string][]string `yaml:"labelsFromPath" json:"labelsFromPath"` // Path is the path to to generate metric(s) for. + // +optional Path []string `yaml:"path" json:"path"` } // MetricGauge targets a Path that may be a single value, array, or object. Arrays and objects will generate a metric per element. // Ref: https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#gauge +// +k8s:deepcopy-gen=true type MetricGauge struct { MetricMeta `yaml:",inline" json:",inline"` // ValueFrom is the path to a numeric field under Path that will be the metric value. + // +optional ValueFrom []string `yaml:"valueFrom" json:"valueFrom"` // LabelFromKey adds a label with the given name if Path is an object. The label value will be the object key. + // +optional LabelFromKey string `yaml:"labelFromKey" json:"labelFromKey"` // NilIsZero indicates that if a value is nil it will be treated as zero value. + // +optional NilIsZero bool `yaml:"nilIsZero" json:"nilIsZero"` } // MetricInfo is a metric which is used to expose textual information. // Ref: https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#info +// +k8s:deepcopy-gen=true type MetricInfo struct { MetricMeta `yaml:",inline" json:",inline"` // LabelFromKey adds a label with the given name if Path is an object. The label value will be the object key. + // +optional LabelFromKey string `yaml:"labelFromKey" json:"labelFromKey"` } // MetricStateSet is a metric which represent a series of related boolean values, also called a bitset. // Ref: https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#stateset +// +k8s:deepcopy-gen=true type MetricStateSet struct { MetricMeta `yaml:",inline" json:",inline"` // List is the list of values to expose a value for. List []string `yaml:"list" json:"list"` // LabelName is the key of the label which is used for each entry in List to expose the value. + // +optional LabelName string `yaml:"labelName" json:"labelName"` // ValueFrom is the subpath to compare the list to. + // +optional ValueFrom []string `yaml:"valueFrom" json:"valueFrom"` } diff --git a/pkg/customresourcestate/zz_generated.deepcopy.go b/pkg/customresourcestate/zz_generated.deepcopy.go new file mode 100644 index 0000000000..f5452804ed --- /dev/null +++ b/pkg/customresourcestate/zz_generated.deepcopy.go @@ -0,0 +1,255 @@ +//go:build !ignore_autogenerated +// +build !ignore_autogenerated + +// Code generated by controller-gen. DO NOT EDIT. + +package customresourcestate + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *Generator) DeepCopyInto(out *Generator) { + *out = *in + in.Each.DeepCopyInto(&out.Each) + in.Labels.DeepCopyInto(&out.Labels) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Generator. +func (in *Generator) DeepCopy() *Generator { + if in == nil { + return nil + } + out := new(Generator) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *Labels) DeepCopyInto(out *Labels) { + *out = *in + if in.CommonLabels != nil { + in, out := &in.CommonLabels, &out.CommonLabels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.LabelsFromPath != nil { + in, out := &in.LabelsFromPath, &out.LabelsFromPath + *out = make(map[string][]string, len(*in)) + for key, val := range *in { + var outVal []string + if val == nil { + (*out)[key] = nil + } else { + in, out := &val, &outVal + *out = make([]string, len(*in)) + copy(*out, *in) + } + (*out)[key] = outVal + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Labels. +func (in *Labels) DeepCopy() *Labels { + if in == nil { + return nil + } + out := new(Labels) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *Metric) DeepCopyInto(out *Metric) { + *out = *in + if in.Gauge != nil { + in, out := &in.Gauge, &out.Gauge + *out = new(MetricGauge) + (*in).DeepCopyInto(*out) + } + if in.StateSet != nil { + in, out := &in.StateSet, &out.StateSet + *out = new(MetricStateSet) + (*in).DeepCopyInto(*out) + } + if in.Info != nil { + in, out := &in.Info, &out.Info + *out = new(MetricInfo) + (*in).DeepCopyInto(*out) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Metric. +func (in *Metric) DeepCopy() *Metric { + if in == nil { + return nil + } + out := new(Metric) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *MetricGauge) DeepCopyInto(out *MetricGauge) { + *out = *in + in.MetricMeta.DeepCopyInto(&out.MetricMeta) + if in.ValueFrom != nil { + in, out := &in.ValueFrom, &out.ValueFrom + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MetricGauge. +func (in *MetricGauge) DeepCopy() *MetricGauge { + if in == nil { + return nil + } + out := new(MetricGauge) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *MetricInfo) DeepCopyInto(out *MetricInfo) { + *out = *in + in.MetricMeta.DeepCopyInto(&out.MetricMeta) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MetricInfo. +func (in *MetricInfo) DeepCopy() *MetricInfo { + if in == nil { + return nil + } + out := new(MetricInfo) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *MetricMeta) DeepCopyInto(out *MetricMeta) { + *out = *in + if in.LabelsFromPath != nil { + in, out := &in.LabelsFromPath, &out.LabelsFromPath + *out = make(map[string][]string, len(*in)) + for key, val := range *in { + var outVal []string + if val == nil { + (*out)[key] = nil + } else { + in, out := &val, &outVal + *out = make([]string, len(*in)) + copy(*out, *in) + } + (*out)[key] = outVal + } + } + if in.Path != nil { + in, out := &in.Path, &out.Path + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MetricMeta. +func (in *MetricMeta) DeepCopy() *MetricMeta { + if in == nil { + return nil + } + out := new(MetricMeta) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *MetricStateSet) DeepCopyInto(out *MetricStateSet) { + *out = *in + in.MetricMeta.DeepCopyInto(&out.MetricMeta) + if in.List != nil { + in, out := &in.List, &out.List + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ValueFrom != nil { + in, out := &in.ValueFrom, &out.ValueFrom + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MetricStateSet. +func (in *MetricStateSet) DeepCopy() *MetricStateSet { + if in == nil { + return nil + } + out := new(MetricStateSet) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *Metrics) DeepCopyInto(out *Metrics) { + *out = *in + in.Spec.DeepCopyInto(&out.Spec) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Metrics. +func (in *Metrics) DeepCopy() *Metrics { + if in == nil { + return nil + } + out := new(Metrics) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *MetricsSpec) DeepCopyInto(out *MetricsSpec) { + *out = *in + if in.Resources != nil { + in, out := &in.Resources, &out.Resources + *out = make([]Resource, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MetricsSpec. +func (in *MetricsSpec) DeepCopy() *MetricsSpec { + if in == nil { + return nil + } + out := new(MetricsSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *Resource) DeepCopyInto(out *Resource) { + *out = *in + if in.MetricNamePrefix != nil { + in, out := &in.MetricNamePrefix, &out.MetricNamePrefix + *out = new(string) + **out = **in + } + out.GroupVersionKind = in.GroupVersionKind + in.Labels.DeepCopyInto(&out.Labels) + if in.Metrics != nil { + in, out := &in.Metrics, &out.Metrics + *out = make([]Generator, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Resource. +func (in *Resource) DeepCopy() *Resource { + if in == nil { + return nil + } + out := new(Resource) + in.DeepCopyInto(out) + return out +} diff --git a/pkg/metricshandler/metrics_handler.go b/pkg/metricshandler/metrics_handler.go index 955a6a84bb..46190aefa6 100644 --- a/pkg/metricshandler/metrics_handler.go +++ b/pkg/metricshandler/metrics_handler.go @@ -26,6 +26,7 @@ import ( "strconv" "strings" "sync" + "time" "github.com/prometheus/common/expfmt" @@ -37,14 +38,29 @@ import ( "k8s.io/klog/v2" ksmtypes "k8s.io/kube-state-metrics/v2/pkg/builder/types" + "k8s.io/kube-state-metrics/v2/pkg/customresource" + "k8s.io/kube-state-metrics/v2/pkg/customresourcestate" + + crmonitorclientset "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/clientset/versioned" + crinformers "k8s.io/kube-state-metrics/v2/pkg/customresourcemonitor/client/informers/externalversions" + metricsstore "k8s.io/kube-state-metrics/v2/pkg/metrics_store" "k8s.io/kube-state-metrics/v2/pkg/options" ) +// Reconfigure provides two functions +type Reconfigure interface { + // ResolveCustomResourceConfig update custom resource stores + ResolveCustomResourceConfig(opts *options.Options) (customresourcestate.ConfigDecoder, error) + // FromConfig construct customresource.RegistryFactory from ConfigDecoder + FromConfig(decoder customresourcestate.ConfigDecoder) ([]customresource.RegistryFactory, error) +} + // MetricsHandler is a http.Handler that exposes the main kube-state-metrics // /metrics endpoint. It allows concurrent reconfiguration at runtime. type MetricsHandler struct { opts *options.Options + ksmCRMonitorClient crmonitorclientset.Interface kubeClient kubernetes.Interface storeBuilder ksmtypes.BuilderInterface enableGZIPEncoding bool @@ -56,16 +72,20 @@ type MetricsHandler struct { metricsWriters metricsstore.MetricsWriterList curShard int32 curTotalShards int + reconfigure Reconfigure } // New creates and returns a new MetricsHandler with the given options. -func New(opts *options.Options, kubeClient kubernetes.Interface, storeBuilder ksmtypes.BuilderInterface, enableGZIPEncoding bool) *MetricsHandler { +func New(opts *options.Options, kubeClient kubernetes.Interface, crMonitorClient crmonitorclientset.Interface, storeBuilder ksmtypes.BuilderInterface, enableGZIPEncoding bool, reconfigure Reconfigure) *MetricsHandler { + return &MetricsHandler{ opts: opts, kubeClient: kubeClient, + ksmCRMonitorClient: crMonitorClient, storeBuilder: storeBuilder, enableGZIPEncoding: enableGZIPEncoding, mtx: &sync.RWMutex{}, + reconfigure: reconfigure, } } @@ -89,13 +109,44 @@ func (m *MetricsHandler) ConfigureSharding(ctx context.Context, shard int32, tot m.curTotalShards = totalShards } +// ReconfigureCustomResourceMetrics reconfigures customresource stores. +func (m *MetricsHandler) ReconfigureCustomResourceMetrics(ctx context.Context, opts *options.Options) error { + m.mtx.Lock() + defer m.mtx.Unlock() + + if m.cancel != nil { + m.cancel() + } + + ctx, m.cancel = context.WithCancel(ctx) + m.storeBuilder.WithContext(ctx) + + config, err := m.reconfigure.ResolveCustomResourceConfig(opts) + if err != nil { + return err + } + + var factories []customresource.RegistryFactory + + if config != nil { + factories, err = m.reconfigure.FromConfig(config) + if err != nil { + return fmt.Errorf("Parsing from Custom Resource State Metrics file failed: %v", err) + } + } + m.storeBuilder.WithCustomResourceStoreFactories(factories...) + m.metricsWriters = m.storeBuilder.Build() + + return nil +} + // Run configures the MetricsHandler's sharding and if autosharding is enabled // re-configures sharding on re-sharding events. Run should only be called // once. func (m *MetricsHandler) Run(ctx context.Context) error { autoSharding := len(m.opts.Pod) > 0 && len(m.opts.Namespace) > 0 - if !autoSharding { + if !autoSharding && !m.opts.CustomResourcesKSMCRWatched { klog.InfoS("Autosharding disabled") m.ConfigureSharding(ctx, m.opts.Shard, m.opts.TotalShards) // Wait for context to be done, metrics will be served until then. @@ -103,77 +154,113 @@ func (m *MetricsHandler) Run(ctx context.Context) error { return ctx.Err() } - klog.InfoS("Autosharding enabled with pod", "pod", klog.KRef(m.opts.Namespace, m.opts.Pod)) - klog.InfoS("Auto detecting sharding settings") - ss, err := detectStatefulSet(m.kubeClient, m.opts.Pod, m.opts.Namespace) - if err != nil { - return fmt.Errorf("detect StatefulSet: %w", err) - } - statefulSetName := ss.Name - - labelSelectorOptions := func(o *metav1.ListOptions) { - o.LabelSelector = fields.SelectorFromSet(ss.Labels).String() - } - - i := cache.NewSharedIndexInformer( - cache.NewFilteredListWatchFromClient(m.kubeClient.AppsV1().RESTClient(), "statefulsets", m.opts.Namespace, labelSelectorOptions), - &appsv1.StatefulSet{}, 0, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc}, - ) - i.AddEventHandler(cache.ResourceEventHandlerFuncs{ - AddFunc: func(o interface{}) { - ss := o.(*appsv1.StatefulSet) - if ss.Name != statefulSetName { - return - } - - shard, totalShards, err := shardingSettingsFromStatefulSet(ss, m.opts.Pod) - if err != nil { - klog.ErrorS(err, "Detected sharding settings from StatefulSet") - return - } - - m.mtx.RLock() - shardingUnchanged := m.curShard == shard && m.curTotalShards == totalShards - m.mtx.RUnlock() - - if shardingUnchanged { - return - } - - m.ConfigureSharding(ctx, shard, totalShards) - }, - UpdateFunc: func(oldo, curo interface{}) { - old := oldo.(*appsv1.StatefulSet) - cur := curo.(*appsv1.StatefulSet) - if cur.Name != statefulSetName { - return - } - - if old.ResourceVersion == cur.ResourceVersion { - return - } + allInformers := []cache.SharedIndexInformer{} + if !autoSharding { + klog.InfoS("Autosharding disabled") + m.ConfigureSharding(ctx, m.opts.Shard, m.opts.TotalShards) + } else { + klog.InfoS("Autosharding enabled with pod", "pod", klog.KRef(m.opts.Namespace, m.opts.Pod)) + klog.InfoS("Auto detecting sharding settings") + ss, err := detectStatefulSet(m.kubeClient, m.opts.Pod, m.opts.Namespace) + if err != nil { + return fmt.Errorf("detect StatefulSet: %w", err) + } + statefulSetName := ss.Name - shard, totalShards, err := shardingSettingsFromStatefulSet(cur, m.opts.Pod) - if err != nil { - klog.ErrorS(err, "Detected sharding settings from StatefulSet") - return - } + labelSelectorOptions := func(o *metav1.ListOptions) { + o.LabelSelector = fields.SelectorFromSet(ss.Labels).String() + } - m.mtx.RLock() - shardingUnchanged := m.curShard == shard && m.curTotalShards == totalShards - m.mtx.RUnlock() + i := cache.NewSharedIndexInformer( + cache.NewFilteredListWatchFromClient(m.kubeClient.AppsV1().RESTClient(), "statefulsets", m.opts.Namespace, labelSelectorOptions), + &appsv1.StatefulSet{}, 0, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc}, + ) + i.AddEventHandler(cache.ResourceEventHandlerFuncs{ + AddFunc: func(o interface{}) { + ss := o.(*appsv1.StatefulSet) + if ss.Name != statefulSetName { + return + } + + shard, totalShards, err := shardingSettingsFromStatefulSet(ss, m.opts.Pod) + if err != nil { + klog.ErrorS(err, "Detected sharding settings from StatefulSet") + return + } + + m.mtx.RLock() + shardingUnchanged := m.curShard == shard && m.curTotalShards == totalShards + m.mtx.RUnlock() + + if shardingUnchanged { + return + } + + m.ConfigureSharding(ctx, shard, totalShards) + }, + UpdateFunc: func(oldo, curo interface{}) { + old := oldo.(*appsv1.StatefulSet) + cur := curo.(*appsv1.StatefulSet) + if cur.Name != statefulSetName { + return + } + + if old.ResourceVersion == cur.ResourceVersion { + return + } + + shard, totalShards, err := shardingSettingsFromStatefulSet(cur, m.opts.Pod) + if err != nil { + klog.ErrorS(err, "Detected sharding settings from StatefulSet") + return + } + + m.mtx.RLock() + shardingUnchanged := m.curShard == shard && m.curTotalShards == totalShards + m.mtx.RUnlock() + + if shardingUnchanged { + return + } + + m.ConfigureSharding(ctx, shard, totalShards) + }, + }) + go i.Run(ctx.Done()) + allInformers = append(allInformers, i) + } - if shardingUnchanged { - return - } + if m.opts.CustomResourcesKSMCRWatched { + informerFactory := crinformers.NewSharedInformerFactory(m.ksmCRMonitorClient, time.Second*60) + ksmCRMonitorInformer := informerFactory.Customresource().V1alpha1().CustomResourceMonitors().Informer() + + ksmCRMonitorInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{ + AddFunc: func(o interface{}) { + fmt.Println("## CR added") + m.ReconfigureCustomResourceMetrics(ctx, m.opts) + }, + UpdateFunc: func(oldo, curo interface{}) { + fmt.Println("## CR updated") + // TODO: only keep real update and skip re-sync updates + // m.ReconfigureCustomResourceMetrics(ctx, m.opts) + }, + DeleteFunc: func(o interface{}) { + fmt.Println("## CR deleted") + m.ReconfigureCustomResourceMetrics(ctx, m.opts) + }, + }) + go ksmCRMonitorInformer.Run(ctx.Done()) + allInformers = append(allInformers, ksmCRMonitorInformer) + } - m.ConfigureSharding(ctx, shard, totalShards) - }, - }) - go i.Run(ctx.Done()) - if !cache.WaitForCacheSync(ctx.Done(), i.HasSynced) { + hasSynced := []cache.InformerSynced{} + for _, x := range allInformers { + hasSynced = append(hasSynced, x.HasSynced) + } + if !cache.WaitForCacheSync(ctx.Done(), hasSynced...) { return errors.New("waiting for informer cache to sync failed") } + <-ctx.Done() return ctx.Err() } diff --git a/pkg/options/options.go b/pkg/options/options.go index 09adf8939a..d431014740 100644 --- a/pkg/options/options.go +++ b/pkg/options/options.go @@ -17,44 +17,52 @@ limitations under the License. package options import ( + // "context" "flag" "fmt" "os" + + // "path/filepath" "strings" "github.com/prometheus/common/version" "github.com/spf13/cobra" + + /* "gopkg.in/yaml.v2" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/client-go/tools/clientcmd" */ "k8s.io/klog/v2" ) // Options are the configurable parameters for kube-state-metrics. type Options struct { - AnnotationsAllowList LabelsAllowList `yaml:"annotations_allow_list"` - Apiserver string `yaml:"apiserver"` - CustomResourceConfig string `yaml:"custom_resource_config"` - CustomResourceConfigFile string `yaml:"custom_resource_config_file"` - CustomResourcesOnly bool `yaml:"custom_resources_only"` - EnableGZIPEncoding bool `yaml:"enable_gzip_encoding"` - Help bool `yaml:"help"` - Host string `yaml:"host"` - Kubeconfig string `yaml:"kubeconfig"` - LabelsAllowList LabelsAllowList `yaml:"labels_allow_list"` - MetricAllowlist MetricSet `yaml:"metric_allowlist"` - MetricDenylist MetricSet `yaml:"metric_denylist"` - MetricOptInList MetricSet `yaml:"metric_opt_in_list"` - Namespace string `yaml:"namespace"` - Namespaces NamespaceList `yaml:"namespaces"` - NamespacesDenylist NamespaceList `yaml:"namespaces_denylist"` - Node NodeType `yaml:"node"` - Pod string `yaml:"pod"` - Port int `yaml:"port"` - Resources ResourceSet `yaml:"resources"` - Shard int32 `yaml:"shard"` - TLSConfig string `yaml:"tls_config"` - TelemetryHost string `yaml:"telemetry_host"` - TelemetryPort int `yaml:"telemetry_port"` - TotalShards int `yaml:"total_shards"` - UseAPIServerCache bool `yaml:"use_api_server_cache"` + AnnotationsAllowList LabelsAllowList `yaml:"annotations_allow_list"` + Apiserver string `yaml:"apiserver"` + CustomResourceConfig string `yaml:"custom_resource_config"` + CustomResourceConfigFile string `yaml:"custom_resource_config_file"` + CustomResourcesOnly bool `yaml:"custom_resources_only"` + CustomResourcesKSMCRWatched bool `yaml:"custom_resources_ksm_cr_watched"` + EnableGZIPEncoding bool `yaml:"enable_gzip_encoding"` + Help bool `yaml:"help"` + Host string `yaml:"host"` + Kubeconfig string `yaml:"kubeconfig"` + LabelsAllowList LabelsAllowList `yaml:"labels_allow_list"` + MetricAllowlist MetricSet `yaml:"metric_allowlist"` + MetricDenylist MetricSet `yaml:"metric_denylist"` + MetricOptInList MetricSet `yaml:"metric_opt_in_list"` + Namespace string `yaml:"namespace"` + Namespaces NamespaceList `yaml:"namespaces"` + NamespacesDenylist NamespaceList `yaml:"namespaces_denylist"` + Node NodeType `yaml:"node"` + Pod string `yaml:"pod"` + Port int `yaml:"port"` + Resources ResourceSet `yaml:"resources"` + Shard int32 `yaml:"shard"` + TLSConfig string `yaml:"tls_config"` + TelemetryHost string `yaml:"telemetry_host"` + TelemetryPort int `yaml:"telemetry_port"` + TotalShards int `yaml:"total_shards"` + UseAPIServerCache bool `yaml:"use_api_server_cache"` Config string @@ -120,6 +128,7 @@ func (o *Options) AddFlags(cmd *cobra.Command) { autoshardingNotice := "When set, it is expected that --pod and --pod-namespace are both set. Most likely this should be passed via the downward API. This is used for auto-detecting sharding. If set, this has preference over statically configured sharding. This is experimental, it may be removed without notice." o.cmd.Flags().BoolVar(&o.CustomResourcesOnly, "custom-resource-state-only", false, "Only provide Custom Resource State metrics (experimental)") + o.cmd.Flags().BoolVar(&o.CustomResourcesKSMCRWatched, "custom-resource-ksm-cr-watched", false, "Watch KSM CR which can monitor custom resource metrics") o.cmd.Flags().BoolVar(&o.EnableGZIPEncoding, "enable-gzip-encoding", false, "Gzip responses when requested by clients via 'Accept-Encoding: gzip' header.") o.cmd.Flags().BoolVarP(&o.Help, "help", "h", false, "Print Help text") o.cmd.Flags().BoolVarP(&o.UseAPIServerCache, "use-apiserver-cache", "", false, "Sets resourceVersion=0 for ListWatch requests, using cached resources from the apiserver instead of an etcd quorum read.") diff --git a/scripts/update-codegen.sh b/scripts/update-codegen.sh new file mode 100644 index 0000000000..c96b677dac --- /dev/null +++ b/scripts/update-codegen.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash + +set -o errexit +set -o nounset +set -o pipefail + +set -x +set -e + +go get sigs.k8s.io/controller-tools/cmd/controller-gen@v0.10.0 +$(which controller-gen) object paths=./pkg/customresourcestate +$(which controller-gen) crd:crdVersions=v1 paths=./... output:crd:dir=./pkg/customresourcestate/apis/config + diff --git a/tests/e2e/discovery_test.go b/tests/e2e/discovery_test.go index d34c932daf..cba679bc37 100644 --- a/tests/e2e/discovery_test.go +++ b/tests/e2e/discovery_test.go @@ -29,12 +29,12 @@ import ( "k8s.io/klog/v2" "k8s.io/kube-state-metrics/v2/internal" - "k8s.io/kube-state-metrics/v2/internal/discovery" "k8s.io/kube-state-metrics/v2/pkg/options" ) // PopulateTimeout is the timeout on populating the cache for the first time. const PopulateTimeout = 10 * time.Second +const Interval = 3 * time.Second func TestVariableVKsDiscoveryAndResolution(t *testing.T) {