Would I be right in thinking that the problem with non-heritage panels is that they don't fit as well as heritage?
Yes that tends to be the case. The non-heritage/pattern panels also tend to be thinner too and the coating on them can sometimes react with whatever paint is used to then spray them with, so it's usually safest to sand pattern panels back to bare metal. It does of course depend upon the particular non genuine panel in question though, some are much better than others and there have also been cases of some batches of genuine Heritage panels not being particularly good fit either.
It's also fairly widely recognised that certain types of pattern panels tend to be a bigger problem than others. Non heritage/pattern wings and scuttle panels have a reputation of being a notoriously poor fit, where as many people report little difference between genuine and pattern A-panels and front panels for instance.