How do I reset my (manager) side of ARD fully? (Authentication fails)

I am having trouble adding a remote system to my ARD. There are two systems there, one I can manage, the other fails on authentication. One the one that fails I can (if turned on) ssh into, and the ARD ports are reachable. But authentication fails. I've done everything I could think of on that side (reboot, remote management off/on, remote management users remove/add, remove as much of the settings on the remote system I could think of) but nothing helps. ARD sees the other machine.

Now, tyhis system is a repurposed system that was first used differently and known differently to my ARD. So I wondered, might my own ARD keep some information alive that it somehow applies to that client? So, what I would like to try now is to remove all of my local ARD information, really everything, and see if that solves the issue.

Other tips are welcome, I'm out of ideas.