Measuring Statistical Precision of Your Data

Or: How do you know how well you know what you know?

One of the greatest challenges of science is knowing how reliable our discoveries are. Ultimately, we want to say not only what we understand about the universe, but also the certainty or uncertainty of that understanding.

If it walks like a duck and talks like a duck, with how much statistical precision can we conclude that it’s a duck?

The way that we do that is to find the statistical precision of our measurements. When we make a new measurement, we always associate an uncertainty value with that measurement. This is done in different ways in different sciences, depending on the goal of the study and the measurement instruments used. For example, when you measure a length with a ruler, you would record the smallest measurement the ruler could make as the precision of that ruler. If a ruler had markings as small as 1 millimeter, then the precision of that ruler would be 1 millimeter. 

This ruler, which is measuring the tooth of an extinct giant shark, has markings as small as 1 millimeter.

In the study of variable stars, the precision of a measurement depends on whether you are observing the star visually or photometrically. For visual observations, the human eye can measure the brightness of a variable star by comparing it to the known magnitude of a comparison star with a precision of about 0.1 magnitude. For more on magnitudes, visit our What is a Magnitude page.

Click image to enlarge. A finder chart for the star Epsilon Aurigae. The chart has Epsilon Aurigae at the center and several other stars around it.

For photometric observations, the light from the star is turned into electrical signals with a precision of about 0.003 magnitudes. But fear not! Even though visual observation is not as precise as photometric observation, visual observation is still very important to the study of variable stars. The problem with photometric observations is that there are not many of them, and their magnitudes have not been measured very often. There are often gaps of weeks or months between photometric observations. On the other hand, visual observations have been taken much more frequently, with few long gaps between observations. These observations allow estimates of the magnitudes of the variable stars to be made with great precision over long periods of time. One final distinction to keep in mind is between precision and accuracy. Accuracy is the degree to which repeated measurements of the same quantity return the true value of that quantity. Precision is the degree to which repeated measurements of the same quantity return the same results over and over again. A good analogy for accuracy and precision is target shooting. Shots that go all over the target but average out to the middle are accurate, but not precise.