[RHOAIENG-11850] Update manifests

[mholder6]

Motivation

Modifications

Updated the manifests to include resource requests and limits.

Result

PR checklist

Checklist items below are applicable for development targeted to both fast and stable branches/tags

• Unit tests pass locally
• FVT tests pass locally
• If the PR adds a new container image or updates the tag of an existing image (not build within cpaas), is the corresponding change made in live-builder and cpaas-midstream to add/update the image tag in the operator CSV? Link the PRs if applicable

Checklist items below are applicable for development targeted to both fast and stable branches/tags

• Tested modelmesh serving deployment with odh-manifests and ran odh-manifests-e2e tests locally

[Jooho]

@mholder6
The root cause of the issue is that there are no resource specifications for the pod. If no resources are specified with a resourceQuota, the pod won't start. Therefore, we should first check if any of the manifests in the model serving components (modelmesh/kserve/odh-model-controller) are missing resource specifications. If any are missing, we need to add them.

However, the values for each pod (CPU, memory) should be calculated properly. I’m not sure where the values in the PR came from, but it would be a good idea to determine the correct values for each pod.

Here’s the suggested approach:

• Deploy all components and leave them running for 1 hour.
• Check the CPU and memory usage metrics. This can be used to set the request values.
• Create and delete several InferenceService (ISVC) instances multiple times.
• Check the metrics again and compare the resource usage before and after ISVC instances are created. Based on this, we can calculate the required resources and set appropriate limits for CPU and memory.

@israel-hdez @spolti any other ideas?

[spolti]

Sounds good @Jooho that would be a better idea than just adding limits to the yamls. Plus, this would need to be configurable assuring that the values configured can be changed if the defaults are not good.
by the way, I suggested @mholder6 take these values from an estimation after deploying some modelmesh models and watch the etcd resource consumption.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mholder6, spolti

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment