You are not far away from my own conclusion.
Some high value animals, I have no wish to shoot in a fence. Like Roan, Sable, or Lechwe.
I have seen all of them in fence, and had no urge to hunt them. In the same time, I have no problem hunting springbok, impala or blesbock in the fence...