Two Kinds of Bad Data

When scientists mistrust their data, they do they same thing you do when you think your watch is off: they check another clock. To diagnose the problem in the Atlantic, Willis needed to compare ocean temperature measurements from multiple sources. The first source he turned to was sea level data from satellite altimeters.

Because water expands when it absorbs heat, and contracts when it cools, sea level is physically connected to heat content in the upper ocean. Satellite altimeters measure sea surface height with radar. The radar sends a pulse of energy toward the Earth’s surface and listens for the echo. The time delay and intensity of the echo reveal the altitude of the sea surface.

Willis also had ocean-based data sets, including temperature profiles from the Argo robot fleet as well as from expendable bathythermographs, called “XBTs” for short. XBTs are the equivalent of a disposable razor. A temperature sensor is spooled out behind a ship by thin copper wire. It sinks, making measurements at increasing depths, transmitting them back to the ship via the wire until the line snaps and the sensor sinks to the bottom of the ocean, discarded.

Artist's impression of an XBT.

An XBT may look like a rocket, but it’s more like a fishing weight: a heavy zinc nose houses a thermistor (to measure temperature) attached to a spool of copper wire. The XBT is launched from a ship, then falls through the water at a constant rate. Temperature measurements are sent back to the ship through the wire until the entire length of wire is unspooled (up to 1,500 meters), at which point the connection breaks and the XBT falls to the ocean floor. (Render by Robert Simmon, NASA Goddard Space Flight Center.)

The devices are manufactured to free-fall through the water at a known rate; scientists infer the depth of the temperature measurements by the time lapsed after the sensor hits the water. They have been used by the U.S. Navy and oceanographers since the 1960s.

“Basically, I used the sea level data as a bridge to the in situ [ocean-based] data,” explains Willis, comparing them to one another figuring out where they didn’t agree. “First, I identified some new Argo floats that were giving bad data; they were too cool compared to other sources of data during the time period. It wasn’t a large number of floats, but the data were bad enough, so that when I tossed them, most of the cooling went away. But there was still a little bit, so I kept digging and digging.”

The digging led him to the data from the expendable temperature sensors, the XBTs. A month before, Willis had seen a paper by Viktor Gouretski and Peter Koltermann that showed a comparison of XBT data collected over the past few decades to temperatures obtained in the same ocean areas by more accurate techniques, such as bottled water samples collected during research cruises. Compared to more accurate observations, the XBTs were too warm. The problem was more pronounced at some points in time than others.

The Gouretski paper hadn’t rung any alarm bells right away, explains Willis, “because I knew from the earlier analysis that there was a big cooling signal in Argo all by itself. It was there even if I didn’t use the XBT data. That’s part of the reason that we thought it was real in the first place,” explains Willis.

But when he factored the too-warm XBT measurements into his ocean warming time series, the last of the ocean cooling went away. Later, Willis teamed up with Susan Wijffels of Australia’s Commonwealth Scientific and Industrial Organization (CSIRO) and other ocean scientists to diagnose the XBT problems in detail and come up with a way to correct them.

Maps of 2004 through 2006 ocean temperature change at a depth of 500 meters comparing corrected to uncorrected data.

Willis’ map of ocean temperature change from 2004 to 2006 originally showed drops of over 1.5° Celsius in the Atlantic Ocean. The apparent large drop in temperature was due to bad data from the Argo floats and XBTs, and it disappeared when errors in these data sets were corrected. (The remaining large swings in temperature visible in these maps are due to shifting positions of ocean currents.) (Maps by Robert Simmon, based on data from Josh Willis and John Lyman.)

“So the new Argo data were too cold, and the older XBT data were too warm, and together, they made it seem like the ocean had cooled,” says Willis. The February evening he discovered the mistake, he says, is “burned into my memory.” He was supposed to fly to Colorado that weekend to give a talk on “ocean cooling” to prominent climate researchers. Instead, he’d be talking about how it was all a mistake.