## Introduction

I frequently hear people make statements that somehow do not seem right. Today I heard a good example. During a discussion of laser levels, the topic of required accuracy came up. I heard a contractor make the following statement:

Who cares if the laser level is accurate within an 1/16th of inch at 100 feet? The Earth curves away from a horizontal line by 1/8th of inch for every 100 feet of horizontal distance. The Earth's curvature will swamp out your instrument error in less than 100 feet.

The statement about the curvature of Earth got me thinking. How much does the Earth's surface deviate from a horizontal line over a distance of 100 feet? The contractor's number intuitively seemed wrong because the Earth is round and the deviation from horizontal should be a function of distance. A little math will give me the answer. For consistency's sake, I will perform all computations in US customary units.

## Analysis

Figure 1 illustrates the situation and contains the derivation of both an exact and a approximate solution. The triangle formed by *x*, *R + δ*, and *R* is a right triangle, which means that the Pythagorean theorem can be used to produce an exact solution. In addition, a simple approximation for *δ* is also developed assuming *R* >> *x* and using a linear approximation for the square root. In Appendix A, I give examples of the computations in Mathcad.

Given the situation shown in Figure 1, we can compute the deviation from horizontal as follows.

Eq. 1 |

where

*R*is the radius of the Earth (3963.2 miles)*x*is the horizontal distance of interest (100 ft)

## Conclusion

The contractor had stated that the curvature of the Earth causes level to deviate from horizontal by an 1/8th of an inch (125 thousandths of inch) for 100 feet of horizontal distance. The actual deviation is ~2.9 thousandths of an inch for 100 feet of horizontal distance, which is almost 45 times less than the contractor claimed. So it is meaningful to buy a laser level that is accurate to 1/16th of an inch over 100 feet, i.e. the laser level error is not swamped by the curvature of the Earth.

Why did the contractor make the claim that the Earth's curvature is 1/8th inch over 100 feet? He made a simple mistake. He did not understand that the deviation from horizontal for short distances is given by a square-law relationship, shown in Equation 2. In Equation 2, I include an approximation that is only valid when *R* is much greater than *x*, which is true in typical construction problems.

Eq. 2 |

If we use Equation 1 or Equation 2 to calculate the deviation from horizontal at 1 mile, we get 8 inches. This value is quoted in a number of references on surveying (e.g. here is one, here is another). What the contractor did was erroneously assume that the deviation varied linearly with distance, which would mean that a deviation of 8 inches at 1 mile is equivalent to an 1/8th of inch at 100 feet.

For those of you who may be interested in the related question of the error in horizontal distances caused by living on a spherical planet, see this blog post.

Aside: Here is an interesting discussion that references this web page.

## Appendix A: Computation Examples

Figure 2 shows a few computation examples. Normally, I let Mathcad do the unit conversion, but I do show one example with explicit unit conversion.

hello, I was confused by your quotient because you state the distance in kilometres and the elevation in feet, and then at the end you quote the distance in miles and the elevation in inches. this is crazy and confusing! Using 4 units of measurement with only one equation, when there should only be one. it doesn't say anything about the conversion in the equation, so I figure it could be metric, then when I use metric in the calculation, the result is different from what you say. I will actually have to look on another website! for the solution! Cheers! happy Christmas!

Dimensional analysis can help you keep things straight. I have added a small appendix that illustrates the calculations. Normally, I let my computer algebra system do the unit conversions for me, so I never actually worry about them. In the appendix, I illustrate how to perform the unit conversion. I hope that helps.

Merry Christmas.

Mathscinotes

I like your web page but I have a question. Since it's been over 40 years since I took algebra could you explain equation 2? When I use smaller numbers, like 3 for x, 4 for r I get different answers between the first step & the second step. 1 for the first step & 1.25 for the second step. Thanks

My article was not clear on the approximation and I have added some explanatory text. The approximation is only valid when R is much much bigger than x. This is true when R is the radius of the Earth and x is a typical construction distance. The approximation fails when when R and x are similar in size.

Hopefully, my clarification in the text will help. I am so used to doing these sorts of approximations that I do not even notice when I do them -- very bad habit.

Mathscinotes

Hi, Thanks for your calculation for the vertical error. I am however interested in figuring out how much the distance between two points vary with curvature of earth (as opposed to just using trigonometry). Could you help me in figuring this calculation out? Thanks

Hi Hanna,

I would like to try to help, but I do not completely understand your question. Are you trying to determine the distance between two points on the Earth's surface? If so, I can help. Generally distances on the Earth's surface are expressed in terms of the great circle distance. There are numerous web sites that address this calculation, like this one. If you are interested in a proof of the formula used on this web page, see this reference. If you decide that you need an Excel version of the calculation and are having trouble putting it together, I can pull something together in less than a minute.

If you have a different question, just give me a bit more detailed version of your question and I will try to help.

mathscinotes

Hi. For some reason I never saw your reply. Thanks for getting back to me. I am observing animals from a 93-96m high sea cliff (tidal variation) with a theodolite. I get angles from reference point that allow me to calculate the (diagonal) distance of the animal in the water from the theodolite on the cliff top. As I know the height at that moment, I can easily calculate the horizontal distance from the theodolite to the animal. However I would like to know what is the actual distance of the animal from the foot of the cliff, if the curvature of the earth is taken into account. I have tried using the moveable typescript calculations but am not sure if I can do it correctly. Assuming the angle from is 45degrees

Hi Hanna,

Let's take a quick look and see if I am thinking of your problem correctly. To keep things very simple, let's assume that your theodolite is 100 m above the ocean and you are observing something in the water at a 45 °C angle. Using "flat Earth" methods, your subject is 100 m away -- correct? Let's work the same problem and determine the arc length of your subject from a point directly below your theodolite.

Here is my drawing that illustrates this situation.

I can convert this situation into a geometry problem as shown in the following figure.

I can analyze this geometry as shown below.

As you would expect, the arc distance is not much different than the distance you would compute using simple angles.

Is this what you were looking for? I can put it into Excel if you want to play with it.

Mathscinotes

Pingback: Whale Math | Math Encounters Blog

Je peux vous dire que c'est continuellement du bonheur de passer sur

votre blog

I am taking big issue with this article.

Here you said there is a 2.9 thousandths of an inch curvature for each 100 feet of horizontal distance.....(heheh).

I hate imperial so please allow me to convert it to metric.

0.07366 mm = 30.480m

Multiply all of this up by 1000 =

73mm fall for every 30,000km. Are you mad? One would have nearly gone around the whole circumference by then - for what a 73mm fall in curve? When the diamerter is ?

Those that come up with 8inches a mile are much closer to the truth.

Correction: "When the diameter is"........12,742km? The observor has curved through thousands of km not less than 1 centremeter you madman.

I can assure you the author is not mad. You, on the other hand, I am not so sure about. You take 1 number from the post, and then completely miss the point of the post (was that deliberate?) and abuse the number in the most absolutely ridiculous way possible to draw a completely wrong conclusion. Then you delude yourself into thinking that is evidence the author is mad???

Your most amusing part is that you seem to suggest that "8 inches a mile" is about correct. You should try applying your same abuse to this number, and you will again assume the author is mad. Trust me, the author is not the madman.

If you want to understand this, you should read the section titled "Conclusion" and pay particular attention to "square-law relationship" and "What the contractor did was erroneously assume that the deviation varied linearly with distance". A mistake you made as well.

Thank you. Very nicely put.

mathscinotes

So, this begs the question: At what point would the curvature of the earth "swamp out" the instrument error?

This is a good question and it all comes down to how you define "swamped". Let's assume that a measurement becomes useless when the error of the instrument equals the curvature of the Earth. My laser level has an error of 1/8 inch per 100 yards (i.e. an angular error). With respect to curvature, this means that the instrument is useful out to about 484 yards. The calculations are as shown below.