OFJ Field Journal from Jim Erickson - 10/9/95
BURGER INTERRUPTUS: A SPACECRAFT IN TROUBLEThe week started out like any other week. Tuesday we approved a high level (not very detailed) science plan for the Ganymede encounter on our eighth orbit (in May 1997). Our minds were focused ahead to the orbital tour phase of the mission. We looked at the science plan for observations, when we would do orbit trim maneuvers to target us for the ninth encounter, and the myriad of engineering events to support getting the science data.
The next day we switched to final approval of the last Jupiter approach sequence, and the actual sequence which will perform the Io flyby, recording of the probe's transmitted data, and the critical main engine burn that will place us into orbit as an artificial satellite of Jupiter. All of that. What a switch from the previous day! We went from thinking a year and a half into the future, to getting ready for one month from now. Having this sequence development concluded was a giant effort, now successfully off of our task checklist.
Unfortunately, we could only relax for a few minutes. Shortly after picking up burgers for the family, we were called back in for a serious spacecraft anomaly, or problem.
What happened? After taking an image of Jupiter, the tape recorder had been commanded to rewind so that it could begin playing back the image. Like most tape recorders, the recorder is supposed to stop when it senses that it has reached the start of the tape. This time, the spacecraft was reporting that the tape recorder was stuck in the rewind mode, and was attempting to slew (or move) continuously.
After analysis, commands were planned to safely stop the recorder. Due to a previous failure of the high power ground transmitter at the Canberra (Australia) Deep Space Antenna site, we have to wait until Thursday morning to send the tape recorder commands. It's *really* hard to sit and wait while you know that your spacecraft is in trouble. In what may be an amazing coincidence, or a possible hint of what the problem is, a similar problem has occurred on the ground simulator (a hardware and software identical copy of the spacecraft computer systems). Analysis of the ground failure may lead to understanding of the flight problem.
Thursday - The commands have stopped the tape recorder successfully. We have organized the effort to find what the problem is and how to fix or work around it. If you've ever had to figure out why your computer isn't working, you already have a good idea about how we do this--you check out everything you can. One group is searching for any hardware problem which could cause the "signature" we see in telemetry (the signature refers to the apparent symptoms of the problem). Another group is looking for any possible spacecraft software problem that could cause the symptoms. Other groups are looking in other directions.
I'm assigned to coordinate a group to look to see if it's possible that the way we have arranged a sequence of spacecraft commands could have caused the problem. With a long history of successful commanding, behind us, we're looking for *anything* we have done differently between the past successful encounters and this sequence (a sequence is a series of spacecraft commands loaded into the computer on board, "clocking out" [running] at the appropriate time). All possible differences are being looked at, even if no one believes they could be relevant.
Daily meetings are being held to give the results found that day. Today, it's mainly descriptions of the plans people have to do their job. The hardware testers have a tough job. They have to be careful that their tests don't cause further damage, and that their tests don't accidentally make the problem go away (on the ground simulator) before we can find out what is causing it.
Friday - Today's anomaly status meeting concentrated on analyzing the failure of the *ground* tape recorder. We saw some plots of the telemetry produced by the ground recorder. The particular failure possibilities were laid out, and there was some discussion about what tests should be done to decide which failure possibility was the real one. It's still not clear that the failure of the ground tape recorder and the spacecraft tape recorder are due to the same problem, but it's a reasonable thing to investigate. It's also much easier for us to examine the tape recorder that's downstairs in the testbed than the one that's near Jupiter. The tests were already in progress during the meeting, and were scheduled to continue over the weekend.
The analysis of the ground software and sequence identified three activities that were linked in one way or another to the period when the tape recorder problem arose. None of these activities seems like a likely cause of the anomaly , but further investigation is needed to make sure.