I just ran "pkg install llvm50" on FreeBSD. Easy, quick, and noninvasive; it's not like installing another compiler is going to trash your system, you still have to take action to use it in place of the default. Likewise on Linux, you've always been able to install multiple compilers, and I've never had a problem with that in the two decades I've been doing it.
I can see containers are sometimes useful and/or a convenience, but you're right it is currently some weird cargo-cult. I don't find it that lightweight either; it can waste serious amounts of space storing all the different images and intermediate states in Dockerfiles. At least on FreeBSD I can use jails with ZFS snapshots (and dataset compression) to keep space wastage to a minimum, and have it quickly freed when I'm done with them without any extra effort.
I find your example of installing a package to be somewhat amusing.
The fact that one can install a package in some OS means that somewhere, one or more people sat down and wrote a recipe for building and packaging it, and then hopefully performed some minimal QA on the final product to make sure it actually works.
This is not trivial, particularly not if multiple versions of a compiler may need to coexist (because of other packages depending on a particular version), and it needs to be done for each combination of compiler version and OS.
To me, examples like: "just run <insert package installation command here>", expecting a package to just magically install itself on some OS are the symptoms of a cargo-cult.
Rather, I think that using containers is more like outfitting a cargo container with a fully stocked and staffed workshop, then shipping the container and personnel to where the client needs it to be. The client then only needs to supply the storage space for the container and the goods that need processing (a shared directory in this particular case) and wait for the finished result.
Sure, some of the workers may not speak your language, they may use different tools, but until (if?) the workshop gets rebuilt and staffed on "native soil", the inconvenience should be acceptable.
I can see containers are sometimes useful and/or a convenience, but you're right it is currently some weird cargo-cult. I don't find it that lightweight either; it can waste serious amounts of space storing all the different images and intermediate states in Dockerfiles. At least on FreeBSD I can use jails with ZFS snapshots (and dataset compression) to keep space wastage to a minimum, and have it quickly freed when I'm done with them without any extra effort.