The problem with shooting much less than 5 shots is that you don't really have a statistically significant sample. The practical problem here is if you are trying to figure out if a change you made in a load has made it shoot better or worse, if you shoot 3 shot groups, the smaller group, if it's not much smaller, could actually be a less accurate load. You would find that out if you shot more rounds into the group. If you are shooting a hotter load in a "pencil barrel" rifle, you may want to shoot 3 shots, let it cool, then fire 3 more. You can overlay targets so you get the 2, 3 shot groups preserved, but the composite of 6 rounds actually tells you more.
Also worth considering if the group is round, or do you have vertical stringing, or horizontal displacement?
Then consider bedding issues, or how a lever action carbine will sometimes "walk" shots lower on target if the front barrel band is tight on both the magazine tube and the barrel, plus how they will react if you shoot from the magazine, or if you keep the magazine full, or empty, or whatever you want to use to replicate how you would actually use the gun in the field.