Reliability is pretty much the same.
What you're asking is like asking whether you should buy a sports coupe or a pickup truck. They have vastly different uses.
Your real question should be whether you need the MacBook Pro's portability and usage without an extra display or. not.
If you just plan in using it on a desk always and will not travel or need to take it anywhere else, the Mac mini M4 Pro is fine.
Also, unless there's a very specific reason you want the Studio Display, it's honestly an overpriced and under featured display. There are much better displays out there, for a fraction of the Studio display cost with better features and performance.
https://www.digitaltrends.com/computing/monitors-you-should-buy-instead-of-apple-studio-display/
For $5000 the Mac mini is selling yourself short. Even the top of the line Mini will not exceed $2000 unless you add in Final Cut Pro and Logic Pro.
The options are extremely disparate and it's pretty hard to make a comparison.
If a 5K retina has served you well, the a MacBook Pro M4 Max is way overkill.
The maxed out Mac mini M4 Pro should be much more than enough in that case.