Are you fairly sure A is better than B?

  • 936 Views
  • Last Post 31 October 2016
John Alexander posted this 24 October 2016

      I believe one of the many factors hindering CB shooters from making as much progress as we might make otherwise it that we draw conclusions about whether a tested “improvement” is really better without shooting enough groups.

  For example, if we want to find out if 16 grains of Old Blaster powder (Load X is better than our standard load of 15 grains (load A). We might load up ten rounds of each and shoot a couple of ten shot groups.  If the A group measures 1.3” and the X group measures 1.6” (23% bigger) it is tempting to conclude that we had better stick with A because X is clearly worse. Joe Brennan has told us that this is a shaky conclusion which we shouldn't have much confidence in based on statistics.  But statistics are hard to love.    However, yesterday in the thread “Testing Bullet Lubes at 3,000 fps” mtngun presented a lot of date on actual shooting results that lets shooters see why Joe is right and the conclusion that A is better than X really is shaky. And it can be seen without opening a statistic book. At least any shooter can see why it is shaky, who will really look at the range of results of the ten 10-shot groups fired, keep an open mind, and do a little thinking.

 Even if we eliminate group three of mtngun's groups because the flier may have been caused by the scope adjustment, the other nine groups ranged from .98” to 1.95” (99% bigger).  There is no logical reason to believe that if the shooter above testing A and X fired eight more groups with each Load A and Load X that those groups wouldn't vary in a similar way. Expecting to make a valid decision about A vs. X with just the first two groups fired is fairly likely to be wrong.   I am not claiming that it takes ten 10-shot groups with load A and X. (Certainly if group X had been twice as big as group A, the shooter's conclusion that load A was better would have been on solider ground.) But I am saying that we are kidding ourselves a bit that the original 20 shot experiment allowed a solid conclusion. When this is considered in conjunction with the fact that from reports here and in the Fouling Shot it is clear that many shooters are making such decisions based on two 5-shot groups. So it isn't surprising that we sometimes seem to be chasing our tail in the pursuit of improving CB shooting.    John  

Attached Files

Order By: Standard | Newest | Votes
John Carlson posted this 24 October 2016

One of the basic tenets of any research project is that the results must be repeatable. For instance when I thought I had identified a trend where my third group of the day was always the worst I tried cleaning the bore after every other group.

Nope, that wasn't it.:thinking:

John Carlson. CBA Director of Military Competition.

Attached Files

RicinYakima posted this 25 October 2016

Posssible reasons your third group is always larger: blatter is full sun is higher in the sky wind changed hungry HTH,  document.write('/images/emoticons/cool.gif');

Attached Files

RicinYakima posted this 25 October 2016

John, I am following all of mtngun's work because Joe B. has convinced me that statistics are real, at least most of the time. And since shooting is an art and not a science, I am following all the trends folks are posting. Ric

Attached Files

45 2.1 posted this 25 October 2016

Oh, statistics are real, but not based on what Joe thinks they are. Take a college class in statistics and they will tell you it is based on “an infinite sample". Anything done here is a very small sample and my not be what is proposed. The older crowd of shooters in the early 1900's preferred 30 to 50 shot groups to tell them what to really expect.

Attached Files

David Reiss posted this 25 October 2016

The issue as I see it,, is that CB shooters are always looking for the “holy grail” of groups. Some trudge along trying what they thinks is best, while others try what Mergatroid is doing and yet still others are doing nothing at all spectacular, but getting the best groups. Some of us are leaders, some are followers and the rest are just lost. No one has obtained the “holy grail” as of today because by the nature of the game we are tinkerers. We seldom leave any aspect of the game alone long enough to work, because the grass is always greener on the other side. So when it comes down to it you have to ask yourself, “Are you a ballistician or a statistician, or are you a tinkerer of everything"

David Reiss - NRA Life Member & PSC Range Member Retired Police Firearms Instructor/Armorer
-Services: Wars Fought, Uprisings Quelled, Bars Emptied, Revolutions Started, Tigers Tamed, Assassinations Plotted, Women Seduced, Governments Run, Gun Appraisals, Lost Treasure Found.
- Also deal in: Land, Banjos, Nails, Firearms, Manure, Fly Swatters, Used Cars, Whisky, Racing Forms, Rare Antiquities, Lead, Used Keyboard Keys, Good Dogs, Pith Helmets & Zulu Headdresses. .

Attached Files

Eutectic posted this 25 October 2016

Sorry Rick, shooting is a science! If it is an art then I am going to take art classes and forget all this fun. Then I can paint a peace sign on my rifle stock and wear a tie-dyed shirt to the range.

The problem is when the difference is small then a LOT of shooting is required.

John Zemanek wanted an answer about bevel base vs plain base cast bullets. The answer required over 400 6 shot groups (6 shot revolvers). This was to prove what was an 18% difference in group size. Most of us would have quit a lot sooner.

I take that back, offhand shooting may be an art. That may explain my offhand groups.

Steve

Attached Files

RicinYakima posted this 25 October 2016

Steve, I respectfully disagree. Art is also: a skill at doing a specified thing, typically one acquired through practice i.e. “the art of conversation". John Zemanek used the “scientific method” to acquire information on his hypothesis: do bevel bases bullets shoot smaller groups than flat base bullets? His results were for his equipment under his conditions and his methods. If it were accepted as a theory, there would only be one type of bullet. Was there peer review that also supported his hypothesis? Those of us who reload, balancing and choosing from multiple factors to create ammunition for a variety of guns, and then shoot them, are artists. As are you!   Ric

Attached Files

45 2.1 posted this 25 October 2016

RicinYakima wrote: Those of us who reload, balancing and choosing from multiple factors to create ammunition for a variety of guns, and then shoot them, are artists. Up until the time you know what variables cause what, how, why and when.... and transfer to the other calibers and cartridges with some very small unbelievable groups. Then it's science....... and very few people seem to know or understand those things.

Attached Files

tturner53 posted this 25 October 2016

Eutectic wrote: Sorry Rick, shooting is a science! If it is an art then I am going to take art classes and forget all this fun. Then I can paint a peace sign on my rifle stock and wear a tie-dyed shirt to the range.

The problem is when the difference is small then a LOT of shooting is required.

John Zemanek wanted an answer about bevel base vs plain base cast bullets. The answer required over 400 6 shot groups (6 shot revolvers). This was to prove what was an 18% difference in group size. Most of us would have quit a lot sooner.

I take that back, offhand shooting may be an art. That may explain my offhand groups.

SteveI have to know, which was better? And, wouldn't there be around 2 million variables that could change the results? Different gun, for instance. Or alloy. I think for every rule we discover there's many exceptions. It's voodoo.

Attached Files

Eutectic posted this 25 October 2016

There ARE 2 million variables but usually only a few major ones. The trick is to try to keep the major ones constant.

Bevel base was better, this was target loads with wadcutter bullets. Matched H&G molds except for the base, same load, same alloy, etc. etc. John was very careful, and he was a suburb technician with a machine rest.

Want to try magnum loads? It might be different, the longer base band might be important.

This does not detract from the usefulness, I use bevel base bullets in my target loads.

Attached Files

Ken Campbell Iowa posted this 25 October 2016

the meaning of the word * art * isn't very * scientific * ...

ken

Attached Files

Wineman posted this 26 October 2016

I work in an industry where “art” and “science” are intermingled that we don't really know which is which. The difference between one batch of spoiled grape juice and another can be hundreds of dollars. Many people will tell you winemaking is an art but until Louis Pasteur did his science 200 years ago, it was just another tub of spoiled grape juice. If you were lucky it tasted pretty good and because of pH, and alcohol it will never kill you unless the quantity impairs your judgment or overwhelms your anatomy. Take yeast for instance, 30 years ago when I started the number of commercial strains was a dozen or so. What do you want a yeast to do? You immerse this poor single cell fungi in a batch of sugar, and to stay alive they use it up and live in their own waste stream (alcohol and CO2). Finish the job (no sugar left) don't make any stinky aromas, not cost too much, and make the stuff taste good is really all you want. However, today there are hundreds of yeast strains (think of them as local variations) and all of them are claimed to “shoot 1 MOA or less". Will you go wrong using A over B? Not likely, science has weeded out the bad apples but the artists are looking for an edge where “B” claims to do magical things to your wine and win all sorts of awards etc. I guess what I am trying to say is that I agree the the number of variables is almost infinite, but there are probably less than 10 which account for most of the variance. Alloy matched to velocity, good barrel and bedding, bullet fit in the throat, mild conditions and firing enough shots only changing one thing at a time goes a long way to accuracy.

Good discussion, let's hear more.

Dave

Attached Files

Eutectic posted this 26 October 2016

"Will you go wrong using A over B? Not likely, science has weeded out the bad apples but the artists are looking for an edge where “B” claims to do magical things” Dave

So we change one thing at a time. You have a good load with lube A to compare to lube B. It is easy, just lube up some identical bullets and load them the same. For lube you can substitute different brand of gas check etc.

Then the big problem is controlled shooting conditions. Wind and weather increase dispersion. When the difference is small, it does not take much dispersion to increase the standard deviation to where small differences are hidden.

So we move to a shorter indoor range. The groups are smaller, and so is the difference. At the indoor range the shooter on your right is whacking away with a 50 Action Express and to your left a SWAT wannabe with a short barrel AR-15 is emptying 20 round magazines as fast as the range safety officer will allow. Dispersion is still a problem,compounded by a headache despite plugs AND muffs.

CBA match shooting has weeded out the bad apples. We know the basics, cast bullet shooting is a mature science, art, sport. Winning groups at matches have been the same for years. The improvements we are trying to make are small, and may be gun specific. To make choices, we need controlled shooting conditions. I think this is currently the limiting factor for most shooters.
Steve

Magic is just another mans technology. Robert Heinlein

Attached Files

Bud Hyett posted this 26 October 2016

When shooting at the Marston Municipal Range while living in Illinois, Ed Doonan taught me to shoot ten-shot groups at three targets aligned on the backer. The biggest part is to call the shot and keep notes on the result. 

In sequence:<>Shoot the first group <>Pull the first target, leaving second and third target.<>Put a fresh target on the board and proceed to shoot the second ten-shot group.<>Pull the first and second target<>Put a fresh target on the board and proceed to shoot the third ten-shot group.<>Pull both targets  You now have:<>Three ten-shot group targets with notes that reflect the changing conditions<>One twenty-shot group target reflecting changing wind and light conditions, shooter fatigue, etc. shot at the midpoint of the testing. <*>One thirty-shot target that is collective and can be compared for flyers with preceding targets.This takes an evening's preparation the week before and concentration at the range for several hours. This should be done twice or thrice, to give a better overall picture. The duration is good practice for a match and helps build concentration stamina. 

Farm boy from Illinois, living in the magical Pacific Northwest

Attached Files

norm posted this 27 October 2016

"Concentration stamina” I like that term.

Attached Files

David Reiss posted this 31 October 2016

I know I am outgunned in this fight, since most of you have much more CB competition experience than I have. 

So from the outside looking in, the CB game is both an art & science. The putting together of the components is science. Changing them when experience tells you too is art. Pulling the trigger at just the right time is art. Reading the environment is an art. Statistics play into all of these things as much as you allow it. 

Success is based on repetition, whether in science when you are trying to prove or disprove a hypothesis or in art when you become a better painter by practice. Yes inherent skill plays a big part in painting, but it does so in shooting too.   If we solely use statistics to guide us to becoming a better CB shooter, then we are chasing our tail as John said. Statistics should be used as another tool on our bench, but how much is up to each individual.

David Reiss - NRA Life Member & PSC Range Member Retired Police Firearms Instructor/Armorer
-Services: Wars Fought, Uprisings Quelled, Bars Emptied, Revolutions Started, Tigers Tamed, Assassinations Plotted, Women Seduced, Governments Run, Gun Appraisals, Lost Treasure Found.
- Also deal in: Land, Banjos, Nails, Firearms, Manure, Fly Swatters, Used Cars, Whisky, Racing Forms, Rare Antiquities, Lead, Used Keyboard Keys, Good Dogs, Pith Helmets & Zulu Headdresses. .

Attached Files

mtngun posted this 31 October 2016

Eutectic wrote:The problem is when the difference is small then a LOT of shooting is required.

John Zemanek wanted an answer about bevel base vs plain base cast bullets. The answer required over 400 6 shot groups Exactly.   And most people don't have either the time, the money, or the inclination to do that much testing.   I keep hoping NASA will give me a grant to study cast bullets but so far it hasn't happened.  :D

Zemanek did some good work, and he always had so much data that you couldn't argue with his results.

Attached Files

mtngun posted this 31 October 2016

To address John's original point -- yes, statistics are often required to determine if “A” is really better than “B", unless the difference is night and day.   

And even with statistics and gobs of shooting, we'll never be able to “prove” a 5% improvement.   

A student's t-test is useful for evaluating simple experiments, and there are free online versions that crunch the numbers for you.   But .... if you mess around with t-tests, you'll soon learn that it's next to impossible to prove anything with 5 groups unless the difference is night and day.    10 groups are much better, but still thin from a statistician's point of view.   Sigh.

If you persevere and actually prove that “A” is better than “B” in one gun, that doesn't prove that “A” is better than “B” as a general rule in all loads and guns.     So you need to repeat the test in different guns and different loads. 

This would be much easier if NASA would award research grants for cast bullet research so we could quit our day jobs and do shooting experiments full time.  :D  :D  :D 

Attached Files

Close