Need to put all the helm charts in one place

[Tomcli]

We need to put all the helm charts in one place so it's easy for users to access. Also, refer to IBM/FfDL#139 for how we can expose our helm charts via github-pages.

[fplk]

What about putting each chart into its corresponding Github repository, since the helm repository can serve as the central location. If someone just wants to set it up, he or she should just use the helm repository rather than messing with repositories, whereas developers should be able to focus on and modify their microservice in isolation while only looking at a single repository. Does that make sense?

[sboagibm]

@Tomcli said:

We need to put all the helm charts in one place so it's easy for users to access.

Can you elaborate on this? Note that our internal ICP deploy is using a single dedicated repo for the helm charts. The dev on that repo states:

because i don’t see the benefit of having the trainer chart in the trainer repo. Any sort of changes would require 3+ PRs. The microservice charts are logically separate but can’t actually be separated, you can’t run a trainer chart standalone

On the other hand, I often make changes to one service that would require a change to the charts for just that repo, and in that case, having a separate repo would really be painful.

Also, refer to IBM/FfDL#139
You changed 53 files in that PR, which I had not seen until now (my bad). Perhaps you can walk us through it on a high level?

for how we can expose our helm charts via github-pages.

And also elaborate on this, as I'm understanding what you're referring to in the PR exactly.

[fplk]

I already pointed that out to Michael - if we can't run services in isolation, I consider that a bug and not a feature or natural occurrence, since microservices need to be isolated and that indicates a lack of decoupling to me. I was also actually able to run the current trainer charts in isolation and the lcm, tds etc. ones. Not sure how much my order mattered - I assume they won't deploy in any order. The helm --debug --dry-run output is already pretty big when running it in isolation, the more stuff we add the more difficult it is to track down errors. Finally, since AI Sphere strives to be a full ecosystem, it seems suboptimal to me to have one giant helm chart for the entire thing. If I want to swap out one component for another, this will be messy, esp. if that component is external and if we cover traditional ML etc. at some point that would only exacerbate the size issues, wouldn't you agree?

[Tomcli]

My Idea in IBM/FfDL#139 is to have 3 main Helm charts. One for installing the s3 driver driver which currently is at AISphere/ffdl-community.

The second Helm chart would be all the third party microservices. In my IBM/FfDL#139 PR, I put all the etcd/localstack/Prometheus microservices in this chart along with any PVC and local PV we want to setup.

Then the last chart will include all the core-microservices. Although in my PR I put all the microservices into a single chart because I did this PR a while ago, I'm open to have each core microservice to have its own subcharts where the global charts can pull them over.