HomePod refusing to connect to Apple TV

Hi all.

This explanation will be kind of long. I have a brand new HomePod that I'm trying to use in conjunction with my Apple TV. Upon set up using my iPhone, I can connect no problem. The HomePod shows up in my Apple TV audio output options and all is well. About two or three hours later, it will stop playing whatever I'm watching, the loading wheel will spin and eventually, the Apple TV gives up and starts playing my audio through the television speakers. After this happens, the HomePod is still listed under Audio output on the Apple TV but it will not connect at all. When I look at the HomePod in my Home app, it also cannot locate the HomePod any longer. When I try to ask Siri a question, she says "I'm sorry I'm having trouble connecting with the internet."

I have no internet issues. I have 8 other devices connected to my internet and they all work splendidly with a speed of 900 MBPS on my last speed test (And I live alone). In fact, the Apple TV works fine as soon as it loses connection with the HomePod and streaming continues with no problem.

When I unplug the HomePod and plug it back in, it works fine for another 2-3 hours and then it does it again. I've done a reset on the HomePod multiple times, including deleting the Home from my Home app altogether and redoing my entire Home.

I got the HomePod 2-3 days ago. It came straight home and has no physical damage.

It's also not my Apple TV because it has the same issues with my Apple TV in my bedroom as well (The one it is usually connected with is my Living Room).

Is there something maybe in the settings of my internet that I need to change? Although, I don't understand how that could be it since it works fine for 2-3 hours before it disconnects.

Also, the HomePod IS updated to the latest update, as is my iPhone that was used to set the HomePod up.

Any help anyone could offer would be much appreciated.