Well, I started this thread and have thoroughly enjoyed the discussion. However, I have sat back and not shared my own list. I will add one more qualifier to the question. Assume that the animal that you are going to pass up is healthy. There isn't anything that I wouldn't shoot if it needed to be put down because it was sick or injured.
OK, so now on to my list of apparently healthy animals that I don't think I would shoot. I'm not saying anyone else should the same preferences as me. As long as it's legal, go ahead and shoot what you want. Also, I know better than to say "I'll never shoot.", these are just my no-go animals as of today, as I sit at home.
My now list: honey badger (for me, just too amazing to shoot), aardvark (seen a couple in daylight and got pictures - that's good enough), steenbok, and probably all of the tiny ten. I've shot enough blue wildebeest and gemsbok, so unless an animal needed to be culled, I'd pass. Beyond those two, there are plenty of others that don't get me excited (like giraffe, springbok, etc.), but that's not the same as saying "I wouldn't shoot..."
I think that even if one of the animals on my list was huge, I would pass. I wouldn't appreciate it (at least as I imagine the situation). Better to leave it for someone who would be thrilled (even my PH if he wanted it). That was my rationale when I took a Shiras moose 4 years ago. I had a boatload of Colorado points for moose. I finally decided to apply. Most likely, with that many points, I could have drawn a bull tag on my first try. I decided to apply for a cow instead. My reasoning was that I have an absolutely huge Alaska/Yukon moose over the fireplace in my trophy room. Even if I took a monster Shiras, it would pale in comparison to the one that I have. I figured that it was better to leave the bull tag for someone who would be thrilled get one. So, I drew a cow tag and killed one. I checked off a second moose specie and got some excellent meat for my freezer. That seemed to me like an all around good outcome.