Explicitly allow cross-namespace references per source

[jonasbadstuebner]

Background

My team is running a company-internal multi-tenant Kubernetes cluster.

We currently have all HelmRepositories, GitRepositories etc. deployed to the flux-system namespace and referenced from there.
We recently learned that this is not the best practice for multi-tenancy and wanted to move the resources to the namespaces they are referenced from, so we could set --no-cross-namespace-refs=true on the controllers. This would make the Repositories more secure for our tenants.

But since we reuse some HelmRepositories (for example for having kube-state-metrics in multiple different namespaces) we would prefer to explicitly allow referencing some of the sources from other namespaces.
So the source for kube-state-metrics (and others) could stay in the flux-system namespace and we could still reference it from any other namespace (or an explicit list of namespaces).

Solution ideas

The idea that came to mind was having a resource where cross namespace references could be explicitly allowed, similar to ReferenceGrants in the Gateway API.
But we would be fine with other solutions to this as well, as long as you could allow cross-namespace references on a per-source level. An annotation on the source would also be a viable option for us.

Do you see a possibility of something like this being added to Flux?
We could support with the implementation.

[matheuscscp]

I'd rather solve this with RBAC. .spec.chartRef could have a .serviceAccountName for impersonation. Same in Kustomization, .spec.sourceRef.serviceAccountName.

[stefanprodan]

This can be solved very easy with Kubernetes builtin CEL admission. You allow cross-namespace from the Flux side, then with a CEL policy you lookup .spec.sourceRef and allow it to have .spec.sourceRef.namespace only when the name matches your allowlist.