Watch Ultra Series 2 delivering notification alerts when locked

After upgrading to Watch OS 26.1, my ultra 2 is delivering push notifications from my phone - with haptics and sounds - even when locked and not on my wrist (stored away elsewhere). It's getting really annoying and sometimes even startling hearing my watch go off out of nowhere.