Index/Instrument Error Correction in Sextant Measurements

Quote of the Day

No scientific discovery is named after its original discoverer.

— Stigler's Law of Eponymy

Introduction

Figure 1: Illustration of Index Error.

Sextants make measurements that are subject to systematic errors – all instruments are subject to systematic errors. Alas, much of my career has been spent on calibrating out systematic instrument errors. Even with all my efforts, residual systematic errors remain.

Figure 1 illustrates a common situation.
Ideally, an angle measurement from my sextant would exactly equal the actual altitude of the celestial object I am observing (green line). Unfortunately, there is always some error intrinsic to the sextant itself. In the case of a sextant, the errors are usually modeled as being composed as a fixed offset plus a angle offset that is a function of the angle measurement. This type of error modeling is common – I basically use the same error modeling approach with the angle settings on my compound miter saw.

In my opinion, the index and instrument errors are the easiest of all the altitude measurement errors to understand. In this post, I will define the terms and show how these errors are used.

Background

Error Model

The errors related to the sextant itself are usually modeled as shown in Equation 1. Some people also add a term called Personal Error (PE), which models errors introduced by the sextant operator. The PE usually not known, but can be determined. I will ignore it for this post.

Eq. 1

where

• TE is the total sextant measurement error attributable to the instrument itself.
• I is the fixed instrument error – it is intrinsic to the instrument as it was constructed. This error is documented by the sextant manufacturer on a calibration certificate shipped with the sextant. In Figure 1, I is represented by the gold-colored line. Figure 3 shows two sextant calibration certificates. As you can see in Figure 3, the fixed error varies with the angle being measured.
• IE is the called the Index Error, which consists of the sum of a number of correctable (adjustable) instrument error sources that add an offset to any angle measured (see Figure 1). A properly calibrated sextant should have an IE of 0. However, it can be time-consuming to adjust this error to exactly 0°, so people often use a computation correction for small IEs. It is measured by observing the horizon, bringing the reflected and direct images of the horizon into coincidence, and then reading the sextant angle – which ideally should be zero, but usually isn't.

Definitions

Correctable Instrument Error Sources

An error source is said the be correctable when the navigator can adjust his sextant to eliminate them. Here are the key sources of error that generally can be removed by making adjustments to the instrument.

Error of perpendicularity

The index mirror must be perpendicular to the sextant frame, otherwise an error is introduced.

Side Error

The horizon glass must be perpendicular to the frame of the sextant, otherwise an error is introduced.

Index Error

The horizon glass must be parallel to the index mirror when the sextant is set on zero, otherwise an error is introduced.

Collimation Error

The telescope must be parallel to the frame, otherwise an error is introduced.

The sum of these errors is usually referred to as the index error – technically, this is not correct but it is common practice.

These sum of these errors results in a measurement that is biased by a fixed value. Figure 2 shows two views of a horizon with a sextant set to 0° – Figure 2(a) shows the horizon through a well-calibrated sextant, and Figure 2(b) shows the horizon through a sextant with a small index error.

Figure 2(a): Sextant Horizon View with No Index Error. Sextant is reading 0° (Source).	Figure 2(b): Sextant Horizon View with Index Error. Sextant is reading 0° (Source).

Fixed Instrument Error Sources

An instrument error source is uncorrectable when the sextant provides no means of adjusting the instrument to eliminate the error source. The sextant manufacturer must measure the sum of these errors and inform the navigator of the error values, which they do with a calibration certificate (see Figure 3). I see these sorts of certificates on instruments all the time – I jokingly call them birth certificates.

Graduation Error

Imperfections introduced in the manufacture of the graduations of the sextant's angle scale. As a side note, one of the most interesting books I have read is a history of graduating sextants and surveying instruments called The Divided Circle. I was amazed at how much history there is on the subject.

Prismatic Error

The error introduced when the planes of a mirror are not exactly parallel.

Centering Error

The error introduced when the index arm does not pivoted at the exact center of arc's curvature.

Figure 3 shows two examples of sextant calibration certificates. Note how the errors can be positive or negative.

Figure 3(a): Sextant Horizon View with No Index Error. Sextant is reading 0° (Source).	Figure 3(b): Sextant Horizon View with Index Error. Sextant is reading 0° (Source).

Conclusion

The nature of the sextants instrument and index errors are such that they just end up being corrections directly applied to any measurement taken by a specific sextant – pretty straightforward.