Reading between the lines at Use the Playing Next queue in Music on Mac - Apple Support:
Turn Autoplay on or off: Click the Autoplay button at the top of the queue to turn Autoplay on; click it again to turn it off.
When Autoplay is on, the button changes
Note: When you turn off Autoplay on a device that uses your Apple ID — your iPhone, for example — Autoplay is turned off on all other devices that use the same Apple ID.
it would seem that the autoplay state should be a function of your Apple ID. Once you turn it off it should stay off, and be that way across any devices using your iCloud Music Library that supports this feature. Testing here it seems my iPhone is better at picking up a change in state than Music. Closing and reopening Music picks up on a change I made on the iPhone. Once turned off it appears to stay turned off for me.
Note that even with this feature off the default queueing method when you select something to play is to play that item and then anything else that is checked in the current view that follows it, or anything in the current view if shuffle is enabled. Using the filter field (View > Show Filter Field), column browser, or album/artist views may be useful in narrowing down what is visible for playback.
tt2