Can you invert a magnetic mask in Final Cut Pro?

You are correct . . . click on the name Magnetic Mask and the small invert icon (circled) will appear to the right.

Try this:

Add a Draw Mask and Draw the mask around the entire frame.

Click the Draw Mask title bar and drag it up over the Magnetic Mask (or the effect using it).

Select the Invert Mask option in the Magnetic Mask.

... If that's possible (like I said, I do not have a version of Final Cut that has Magnetic Mask available.)

If that doesn't work...

Duplicate the clip - top version with Magnetic Mask, but bottom one without.

Set the Compositing > Blend Mode of the Top clip to Silhouette Alpha.

As FCP is not behaving normally it may be time to delete the preferences.

Mouse over the Magnetic Mask title bar, a small icon should appear looking like a circle inside a rectangle. Click on that icon and select Invert Masks.

That said, I don't have a version of FCP that has magnetic masks... so somebody else might need to "chime in" here.