Hey there Kärnhus,
Thanks for reaching out to Apple Support Communities. I understand you’re seeing an unexpected blank slide with the same page number. I’ll be happy to help with this.
Removing a slide can be done using the steps from the following link:
Keynote for Mac: Add or delete slides
If these are showing the same slide number, are these transitions from a blank page to the info on the slide? If so, you can use the steps from this link to help you remove the transition:
Keynote for Mac: Add transitions between slides
If you’re still having issues after this, you may want to reach out to Apple Support using the following link. They can look further into this with you:
Contact Apple Support
Cheers