iPhone12 Disabling auto connect for Bluetooth speakers

I have multiple stereos/speakers for different parts of my home that I like to Bluetooth to at my choice. I have multiple iPhones/users in my home. And the phones auto connect by who is closest to whichever speaker first automatically, and it’s irritating. Unpairing to avoid this every time is also inconvenient. We both have smart watches and thus disabling Bluetooth is not a practical option. Is there a way to prevent “auto-connect” for specific Bluetooth devices (without un-pairing) on an iPhone? Please Apple, make this an option if not. I’ve researched and it seems to be a common conflict in this connected world.