OFJ Field Journal from Robert Gounley - 10/11/95

WHEN A PAGER GOES OFF

That's the way it usually happens. The evening of Wednesday, 11 October would be different.

My friend and I were driving a freeway to Pasadena on our way to dinner and a lecture at Caltech. When my pager began its obnoxious chirping, I threatened, as on many previous occasions, to throw it out the window of my moving car if it was yet another false alarm. This time, I recognized the phone number. It was the Galileo Mission Support Area, the facility where data from the spacecraft is processed. When they page me, usually there's a problem on the spacecraft. This didn't feel like a false alarm.

The previous day, the spacecraft had snapped some pictures of Jupiter, the first of our approach images. Today we were to play the tape recorder back, buffering short segments of the picture and downlinking the raw bits to the ground. With our current software and data-rate, it would take several weeks to download the pictures, but before leaving the Galileo project in November I was looking forward to having a full-color picture of Jupiter to hang on my wall. Had something gone wrong?

Within a few minutes, I had reached a nearby supermarket and was talking to Jose' about the beeper message. It didn't sound good. Earlier that day, the tape recorded had been commanded to rewind back to the beginning of the tape in preparation for playback. This should move the tape at high speed until sensors in the recorder detect the transparent leader at the end; this signals the recorder to automatically stop the tape. The telemetry from the spacecraft showed that the capstan, the wheel that drives the tape, was turning at the expected rate. However, it should only have taken a few minutes to reach the beginning of tape. The capstan had kept spinning.

No one quite knew what this meant. Had the tape broken? Was the telemetry playing tricks on us? The Anomaly Recovery Team was being called and I would have to pull a story together. Dinner with my friend was out.

When I got to JPL, people had already begun to stream in. Jose' informed me that the tape recorder motor was still running. (At least it was when the radio signals with this information left the spacecraft 45 minutes earlier.) Several people were on telephone talking to hardware experts about what they were seeing. Everyone was very intent on what they were doing. The spacecraft was in trouble less than two months before the most important part of its mission.

By all indications, the tape recorder motors looked like they would go on indefinitely waiting for that signal that the end of tape had been reached. The program on the spacecraft was supposed to command the recorder to play back sections of the tape, but the recorder was rejecting these commands because it hadn't finished its last task. We would have to radio a command to the spacecraft to tell the recorder to stop unconditionally.

Time was short. From the tracking station listening to the spacecraft (Goldstone, out in California's Mojave desert), Galileo was a setting star in the sky. Soon it would be too low on the horizon for the signal from the tracking station to get through. Ordinarily, we could call on the tracking station in Canberra, Australia to send the commands since Galileo would then be rising in its sky. Unfortunately, the transmitter at the station had failed only the week before and was down for repair. The station would be able to listen to Galileo, but not talk to it. The next station, in Madrid, Spain, wouldn't be in view of the spacecraft for nearly 12 more hours.

No commanding had been planned during this time, so the Goldstone transmitter had been left off in order to improve reception of Galileo's weak signal. Getting the transmitter operational would take nearly an hour to warm up the high-power electronics and tune the signal so that the spacecraft could get the message clearly. Meanwhile, while the folks in Goldstone were busy getting their equipment ready, we here at JPL would prepare the commands for transmission, making certain that the instructions we send couldn't possibly make the situation any worse.

Soon I found myself in the Project conference room, briefing all assembled on the state of things. While speaking as clearly as possible, my words were going as quickly as they could go. The Project Manager understood and approved of the plan to stop the recorder and several engineers bolted out of the room to send the commands. They returned ten minutes later, glum and shaken. The transmitter at Goldstone had resisted attempts to turn it on quickly. We had missed our opportunity!

That same night, other engineers where running a simulation of the Jupiter encounter on the Galileo Testbed. The Testbed is built from computers and other components identical to the ones on the spacecraft. We use it to be certain that the commands we send to Galileo will perform exactly as we expect. By some incredible stroke of fate, the testbed engineers discovered they were having a problem also. The tape recorder they were using, identical to the one flying to Jupiter, seemed to be moving without reaching the end of tape! Whatever the problem was, we knew that at least the recorder on the ground could be opened up and examined for any clues that might explain our problems in flight.

That night, we made ready the commands that would be sent to the spacecraft from Madrid the following morning. None of us knew whether we would have a working tape recorder when this was over. No one talked about it, but continued in their preparations to be sure that everything would go smoothly tomorrow.

I drove home around midnight after first stopping at an all-night hamburger stand for that dinner I had missed earlier. A half billion miles away, the spacecraft I had worked on for nearly 13 years was suffering from a major glitch and there was nothing anyone could do but wait.

EPILOG:

The commands sent to Galileo the following morning stopped the recorder. Later, it was found that the problems with the tape recorder on the testbed only superficially matched those on the spacecraft. The parts that had failed on it could not have caused the flight equipment to behave in quite the way that it did.

Tape experts from the manufacturer and JPL identified several theories to explain the problem -- some of them recoverable and some not. About a week after the original problem, we sent a series of commands to the spacecraft to see if the tape could be made to move. It worked and the tape advanced exactly as instructed.

The anomaly will cause us to use the tape recorder more cautiously, but it now appears probable that we will still be able to use it to collect and store data for the life of the mission.