I agree, there's nothing worse than skittish animals in general. I work with foals and yearlings and handle them from day one, I try to encourage them to be confident and inquisitive.
The more you do early on the easier the job will be when the time comes to sitting on them for the very first time, it makes all the effort worth while and it's much more pleasant for both jokey and horse as the horse isn't frightened to death.
Do you not find that when a foal is tamed from a young age makes them wicked as they have no fear of humans