I've been using an M1 Mini for over a year and don't see anything slow down.
I zipped a 330MB folder and it took a few seconds. It unzipped almost instantly.
If you are copying to the same volume on your startup drive, it should be instantaneous as it would be using copy-on-write. The copy is just a reference until you edit it. At that point it copies to a new storage location.
Are you moving and copying to an external drive? What format is the drive and how is it connected? If USB3, are you really getting USB3 from the connection? Some hubs will downgrade the entire bus to USB2 if there are any USB2 or USB1 devices connected.
You can view that in System Information (hold option key when selecting Apple menu). Select USB and look for the bus your drive is connected. Does it show Up to 5Gb/s or 480Mb/s?