I do something similar. Over 90% of my YouTube consumption is with Alfred workflows which use mpv and yt-dlp under the hood. I just press a keyboard shortcut and the frontmost tab closes in the browser and starts playing in mpv.

The remaining percentage is still annoying, as it happens from the phone.