It's the whole package working together.... or not. For the best accuracy: The bullet has to be good. The bullet has to be correct for the twist and geometry of the lands and grooves and diameters. The bore/barrel has to be straight. The bedding has to be right. The action has to be stress free and in line with the bore. The shooter has to shoot straight. Also, for cartridge guns after the first shot with new ammo or new brass, the reloading dies have to correctly fit the chamber. The reloading specs have to be specifically set up for the individual rifle and chamber.
Obviously if you see unusually large or odd groups something is not right. Many times I've watched and listened to shooters at the range talking about their loads, rifles and subsequent targets. But IMO over half the time they are completely misinterpreting what the target is "saying". Unless they ask for a second opinion, I just agree or talk about something else.

Some shooters do have a preconceived notions about things.
In order to compare apples to apples, here are a few targets shot by a variety of my rifles in different calibers all at 50 yards. For years I've watched new or wanna-be muzzleloader BP shooters struggle with Minie' conicals out of new, reproduction rifle muskets in perfect condition. After struggling, most throw in the towel and resort to something like a modern fast twist ML using pellets, sabot'd jacketed bullets and scopes

Just as with cartridge guns, the load, components and technique have to match the gun! Most of the time when they first try shooting Minies' the groups are large patterns with some key holing

Old style MLs shooting cast lead bullets and BP have long had a rep for being inherently less than accurate. I added a couple of targets examples below, shot by original rifle muskets shooting hollow base Minies' to counter that misconception.
Full power load, Montana M99, 450 Watts, 450 gr GS flat point solid, 2 1/2 X Leupold, 3 shots, 50 yards
45-110, C Sharps, 460 gr GG bullet, 2F blackpowder lightly compressed, globe front and tang aperture, 5 shots, 50 yards
Original M1854 Mississippi, 54 cal Minie', 55gr 2F blackpowder, open sights, 5 shots, 50 yards
Original M1863 Springfield rifle musket, 58 cal Minie', 60 gr 2F blackpowder, open sights, 3 shots, 50 yards