Quote of the Day
I mean the word proof not in the sense of the lawyers, who set two half proofs equal to a whole one, but in the sense of a mathematician, where half a proof is zero, and it is demanded for proof that every doubt becomes impossible.
— Karl Friedrich Gauss, quoted from Calculus Gems (1992) by George F. Simmons
Introduction
When you build an electronic product, at some point you will be accused of building a product that is interfering with someone else's product. In this particular story, a little bit of math quickly showed that the problem could not be with our product. Given this knowledge, we then embarked on a field trip to determine exactly what was the cause of the problem. The actual problem and its solution ended up being a surprise for everyone.
This particular story illustrates a number of points about solving field problems:
- The first word from the field is always in error.
- Get the right person on the problem right away or bad things are going to happen.
- When bad things begin to happen, your ability to contain them is nil.
The story ends well, but solving the problem ended up being MUCH more painful than it needed to be. If one or two things had been different right up front, the problem would have been solved with little distress.
To make the rest of this discussion clear, I should mention that we make a product that mounts on the side of a home and that is deployed by local telephone and cable companies. These devices provide provide Internet access using Ethernet over fiber optic cable. Hundreds of thousands of these units have been deployed with no interference complaints — until now.
The Problem
The problem began for me with a phone call from a manager who handles our relations with a government regulator. A customer had complained to the regulator that one of our products was interfering with fire department communications. This is a very serious charge. However we have shipped these products for years with no interference issues of any sort. Something did not seem right. The government regulator had said that we needed to meet with this customer and resolve his issue. If the problem was not resolved, we would need to stop shipping product.
I was floored. Thankfully it would turn out that NOTHING we had heard from the field would turn out to be correct. However it was my job to prove that we were not the cause of the problem.
Information Gathering
The first thing I did was contact the customer with the complaint. Let's call him "Bill," which is not his real name. Bill was mad and wanted to make sure that I knew he was mad. Here is how Bill saw the problem.
- He had received a number of complaints from his customers that our hardware was interfering with local fire department radio communications.
- He had contacted our customer service and had been given the run-around.
- He had contacted our product manager and had received some advice, but nothing they told him to do solved his problem
- This had been going on for months.
- He was sick and tired of the situation and had reported us to the government, which he felt would get our attention.
Bill certainly was correct about getting our attention. My approach in these situations is to play "Mr. Spock" – you know, the Vulcan from Star Trek. In an absolute deadpan tone of voice, I just keep asking good questions and eventually the angry customer will realize that you know what you are doing and that you want to help. I told him that I had just been informed of the problem and that I was confident we would get to the bottom of the issue. He began to cool down once I began to ask questions. Here is what my interview told me.
- The interference was at 155.52 MHz – a sub-harmonic of GPON, a fiber optic transport. This relationship can be seen by noting that .
- Their emergency communications were assigned to this frequency by the FCC.
- He had not observed the interference himself, but had only heard reports of it.
- The fire department did not have any technical knowledge of their communications system. A local consulting engineer managed the system for them.
I asked him to contact the consulting engineer and get a first-person assessment of the interference. Meanwhile, I would check with the FCC and find out what I could about their emergency communication system.
Things Begin to Clear
Once I had their radio call sign, I could use the FCC radio web site to find out information about their emergency communications system. The information would be useful in my mathematical analysis.
- The transmitter was licensed for 100 W of effective radiated power (ERP).
- The antenna was on a hill about one 1 km out of town (it is a small town). When I actually got on-site, I saw that the antenna could be seen from all points within the town.
- The communication system was designed to transmit a signal that would tell scanners that it was a communication channel.
Bill had also got busy and contacted the consulting engineer in charge of their emergency communication system. Bill soon got back to me and was sounding pretty sheepish. The consulting engineer told him that the fire department had not reported ANY interference. What had been reported was that one citizen who owns a scanner was complaining that our product was causing his scanner to work differently than it had before. Prior to using our product, his scanner would scan the emergency band for fire department communications. The scanner would only stop when there was an emergency transmission. When our product had been deployed, the scanner now would constantly stop on that frequency because it would detect the low-level GPON emission from our product. Since the GPON emission from our product is no different in level from that made by a normal PC's Ethernet at , the scanner would have trouble with or without our product being there.
The interesting thing that the consultant said was that the fire department LIKED the situation. Their communications were working fine and they consider the local citizens who monitor their communications to be a nuisance. These local folks had the nasty habit of showing up at every call the local fire department made.
While on the FCC website, I noticed that some emergency communications in my state used the same frequency as Bill's fire department. These local communities also used our product, but I had never heard any complaints from these communities. I called the folks in charge of these communications and they confirmed that they had received NO reports of interference. Now I am really confused.
What was happening here? Why was Bill's fire department having no trouble at all and the local "ambulance chasers" were unhappy? Why had no one in my state with the same situation complained at all?
A little math soon shed some light on the situation.
Analysis
Determining the Field Strength of an Isotropic Radiator
While a detailed study of the field pattern of a real antenna is complicated, we really only need an "order of magnitude" analysis to illustrate that our product could not be the source of the problem. Let's begin by deriving an expression for the field strength from an omni antenna – an omni antenna radiates equally in all directions. This will be close enough for what we are doing here. Equation 1 gives us the power radiated in all directions from an isotropic radiator.
Eq. 1 |
|
Where
- σP is the power per steradian (solid angle unit)
- r is the radial distance from the antenna
- PT is the Effective Isotropic Radiative Power (EIRP)
- GT is the antenna gain (= 1 for an isotropic radiator)
We can use Poynting's Theorem (Equation 2) to relate the radiated power to electric field strength and magnetic intensity.
Eq. 2 |
|
Where
- E is the electric field strength
- H is the magnetic intensity
In Equation 3, we use the impedance of free space to relate H to E.
Eq. 3 |
|
We can substitute Equations 2 and 3 into 1 to get Equation 4.
Eq. 4 |
|
We can solve Equation 4 to for E to get Equation 5, an expression for the radiated electric field strength from an omni antenna.
Eq. 5 |
|
We can use Equation 5 to compare the field strength from our product to that from Bill's fire department's antenna.
Product Radiated Power Versus Fire Department Radiated Power
Field Configuation
I had been told that that problem was occuring in town. I wanted to understand how the radio signal from our product compared to that of the fire department in town. While still in the office, I used some information from Bill and Google Earth to determine that:
- Our product was usually mounted about 30 meters from the scanner antenna
- The fire department's antenna was about 1 km from the scanner antenna
Estimating Product Emission Level
Our product emission levels at 155.2 MHz were measured at a government-approved facility to be 30 dBµV/m at 3 meters. Since field strength varies linearly with distance from the source, the field strength from our product at 30 meters must be 1/10th the value at 3 meters or 3 dBµV/m.
Estimating Emergency Transmission Field Strengths
We can use Equation 5 to compute the field strength from the fire department's antenna. The calculation is illustrated in Equation 6.
Eq. 6 |
|
This means that the signal from the fire department is more than 17,000 times stronger than the emissions from our product. There is no way that our product is interfering with their reception. It turns out that this analysis was absolutely correct.
My Field Trip Results
When I visited Bill, things got interesting. Here is a quick summary of what I discovered on my field trip.
- Our test gear showed that our math estimates were right on. In the town, the fire department antenna produced a monster signal compared to our product.
- We identified emissions from our product and similar emissions from every computer, router, and switch in town. All emissions measured were within FCC limits.
- I purchased both an analog and digital scanner so that I could see the problem myself. The analog scanner, when set at maximum sensitivity (minimum squelch), would stop scanning at 155.2 MHz when near our product. It would also stop at Ethernet emission frequencies, like 156.25 MHz, when near a PC, router, or switch. Newer digital scanners had no issue at all. Digital scanners filter out unwanted interference by identifying a special code that emergency communications systems put out. They will never lock on our product's emissions because we do not emit that special code by adjusting their squelch level up slightly.
- It turned out that no person in Bill's community had reported any interference issues. Only one person could be found that actually had a problem and he was many kilometers outside of the city. He had an old analog receiver that was set for maximum sensitivity. This gentleman said that he needed his receiver to be at maximum sensitivity because he could just barely receive the fire department transmissions under the best of circumstances. Of course, our product had been mounted in the worst possible location with respect to interference (right next to his receiver input — 1 meter instead of 30 meters).
I now had all the information I needed to state with confidence that I had identified the root cause of the problem.
Conclusion
When I returned home I contacted my state emergency communication coordinator who informed me that my state had converted to digital emergency communications many years before. Therefore, the scanners users near my home had converted to digital scanners in the years before our products came out. There were either no more analog receivers in my area or their squelch levels had been turned up slightly to ignore the interference from any Ethernet-based product. These same squelch settings would eliminate problems with GPON emissions, too.
What happened was that Bill had one customer way out of town with an analog receiver. At his distance, the emission from our product was smaller than that of the fire department's antenna, but still large enough to be detected by his scanner (it was set at maximum gain). We informed this customer that he could try adjusting his squelch, but he refused to do that because he said increasing the squelch level would reduce his receiver's sensitivity. He felt he needed all the gain he could get to receive the fire department's signal clearly because it was weak at this distance. After reviewing his specific situation, our product had been mounted in the worst possible place relative to his antenna. We were confident that if we moved our product a few feet away from his receiver that we would eliminate the interference. Bill agreed to discuss this with the gentleman, but he was concerned that his customer may have imposed limits on where he could mount our product.
It turned out that people in town had long ago adjusted their squelch levels up to avoid interference from local computers and other Ethernet-based equipment. In fact, many of the local scanner operators had recently switched to digital scanners when their state switched to digital emergency communications. The scanner code used by the fire department was published on the web and anyone could program their digital scanner to stop only on fire department transmissions.
I told Bill that I felt that we had met our regulatory obligations and that the one person with a complaint could easily deal with the problem by moving the position of the product. Bill agreed and said that he would withdraw his complaint with the government regulator. The case was closed.
Clearly, this situation had been blown way out of proportion. Because the right people had not been put on the problem right away, actually solving the problem was taking forever and Bill had grown exasperated. In a desperate attempt to get some effective help and satisfy this one unhappy customer, Bill complained to the government. I guess his approach worked because he got me on site. As in the game "Telephone," the magnitude of the problem had grown with each retelling. By the time it got to me, it sounded like their fire department communications were totally jammed and emergency services were completely disrupted. In fact, it was an extremely minor problem with one very vocal person.
As I said in my introduction, this story illustrates my three points on dealing with field problems.
- The first word from the field is always in error.
- Get the right person on the problem right away or bad things are going to happen.
- When bad things begin to happen, your ability to contain them is nil.