Remove Music Album Ratings (Grey Stars)

Does anyone know how to disable Apple automatically adding an album rating? I have smart playlists that only include tracks rated 4* and above. For some unknown reason if I rate just a single track as 4, sometimes the whole album will get rated as 4 and every other song within that album completely skewing all my playlists. I want to talk to the genius who thought this was a rational idea.

I tried using some custom scripts from @turingtest but they haven't done anything. Any help would be much appreciated.