Estimating the Reliability of the Shuttle Reaction Control System

STATISTICAL ANALYSIS OF FAILURE DATA in the reaction control system (RCS) of the space shuttle is an important step in improving the reliability of the system. Past studies have suggested that the two variables "soak time" and "cycle time" are major explanatory variables for predicting reliability. "Soak time" refers to the amount of time the oxidizer valve in the RCS is under pressure in a N₂O₄ environment, while "cycle time" is the number of cycles of operation of the valve. The reliability of the system is expressed by the survival function R(s,t), the probability that the system is operating when the soak time is s and the cycle time is t. In a method previously used to estimate reliability, the two variables are treated separately (that is, only the marginal survival functions are considered) and Weibull models are assumed for them. A result of this procedure is that valves appear to wear out with increasing soak time but to improve with increasing cycle time. Its chief drawback is that it does not account for dependencies between soak time and cycle time. Figure 1 shows quantile-quantile plots of uncensored failure times with fitted Weibull distributions. The estimated parameter values of 2.9 and 0.737 bear out the observations mentioned above about the wear of oxidizer valves. The shapes of the plots indicate that the Weibull model is certainly reasonable, at least for the marginal distributions.

Fig. 1. The Q-Q plot and the CDF plot for soak time and cycle time for uncensored observations.

Fig. 2. The Q-Q plot and the CDF plot for soak time for all observations.

The data of this study consisted of 259 observations, the majority of which were censored by cessation of data collection. That is, units were still operating or were routinely replaced before failure and we can say only that their soak and cycle times to failure were greater than the times since their installation. Figure 2 shows that the Weibull model fits poorly when the censored data is included. An application of the goodness-of-fit test developed by Akritas and Clogg¹ confirms this. We have considered three possible improvements of the basic Weibull model:

1. A modified Weibull model for the marginal survival functions. For either soak time or cycle time, the observable variables are (X,d), where X = min(T,U), T is the time to failure, U is the censoring variable (time elapsed since the valve was installed), and d is an indicator of whether or not the observation is uncensored. If T £ U then d = 1; otherwise d = 0. We assume that X has an unconditional Weibull distribution and also that its conditional distribution given that d = 1 is also Weibull. Associated with these two distributions are four unknown parameters for which maximum likelihood estimates were obtained numerically. From the estimated parameters, the cumulative hazard function

   

   and the survival function R(t) = exp(-L(t)) could be estimated. Theorem 2.1 of Chen et al² ensures strong uniform consistency of the estimator of A.

    

2. An alternative to the Marshall-Olkin distribution. The Marshall-Olkin bivariate exponential survival function P[S > s;T > t] = exp(-Ls - mt - rmax(s,t)) describes a system with two independent exponential life lengths U₁. and U₂, both of which are censored by a third exponential variable U₃.³ Thus S = min(U₁,U₃) and T = min(U₂,U₃). If we assume Weibull instead of exponential distributions for U₁, U₂, and U₃ we obtain a large family of bivariate survival functions potentially useful for modeling RCS failure times. Unfortunately, the marginal and conditional survival functions from this family are not necessarily Weibull. Since we have good Weibull fits to observed failure times, this approach is not completely satisfactory. A simple modification of the family leads to Weibull marginal and conditional distributions.

   P[S > s ; T > t] = exp{-(ls)^a - (mt) - r(ls)^a(mt)^b}

   Not only is this desirable for the sake of agreement with observations but also the family may be derived from more basic considerations. Maximum likelihood estimators of the parameters have the expected asymptotic normality and efficiency properties,² although they are of questionable applicability in the present situation. The new model was fitted by maximum likelihood to 54 observed failure times in the data set and the results were compared to the results of separate marginal fits. The estimated shape parameters a and b were 2.95 and .683, compared to 2.93 and .682 and the estimated scale parameters l and m were .00049 and .00127, compared to .00049 and .00128. Thus, the bivariate model and separately treated marginals gave almost identical marginal distributions. The estimated value of the interaction parameter r was .165. Figure 3 shows the marginal quantile-quantile plots and the estimated and empirical distribution functions for the bivariate model. When the censored data is included in the estimation procedure, the parameter estimates become a = 5.09, b = .50, l = .00041, m = .00032, r = -6.66. This is a considerable change, and the fit between the model marginals and the empirical marginals for uncensored observations becomes worse. We do not conclude that the bivariate model is unsuccessful, however, because it is not intended merely to fit the empirical marginal distributions.

   

   Fig. 3. The Q-Q plot and the CDF plot for soak time and cycle time for marginal distributions of the bivariate Weibull model with uncensored observations.

    

3. A logistic regression approach. Let p(s,t) denote the probability that the system has failed when the soak time is s and the cycle time is t. We considered additive models in which this function is of logistic form:

   log((p(s,t))/(1 - p(s,t))) = g₁(s) + g₂(t),

   where g₁ and g₂ are functions to be determined. A preliminary spline fit indicated that they might be cubic polynomials. Under that assumption, we found that neither variable could be eliminated from the model, although the estimated coefficients for cycle time are not highly significant individually. There are no interaction terms in the model; however, this does not imply independence of S and T. Since failed valves show little correlation between soak and cycle times, there is no prior reason to suppose that interaction terms must exist. A stepwise procedure based on the AIC criterion was applied to a model with interaction terms and seemed to indicate that they should not be included. Because of the sparseness of the data, that conclusion is tentative.

A simple comparison of the predictive power of the bivariate Weibull model and the logistic model was obtained by using each to predict the status of 226 valves in the study, rejecting as outliers 33 observations, all with soak time 1663. Fitting both models with the entire data set, excluding outliers, we estimated the probability of survival at the recorded soak and cycle times. Either model predicts survival if the estimated survival probability is greater than .5. Table 1 shows predicted survival versus actual survival.

Table 1. Predictions made by Logistic and Weibull Models

It is clear that by this standard the logistic model works better than the Weibull model for this data. We do not conclude, however, that it is better for all purposes or for other data sets. The Weibull model might well be preferable for gaining an understanding of the wear processes affecting the system.

Footnotes
¹M. Akritas and C. Clogg. "Tests of Independence for Bivariate Data with Random Censoring: A Contingency Table Approach," Biometrics 47 (1991): 1339-54.
²H. S. Chen, R. Chhikara, R. Heydorn, and C. Peters. "Modeling the Shuttle Reaction Control System Data," (1998). (In preparation.)
³A. W. Marshall and I. Olkin. "A Generalized Bivariate Exponential Distribution," J. of Applied Probability 4 (1967): 291-302.

References
G. Campbell. "Nonparametric Bivariate Estimation with Randomly Censored Data," Biometrika 68 (1981): 417-22.
D. M. Dabrowska. "Kaplan-Meier Estimate on the Plane," Annals of Statistics 16 (1988): 1475-89.
J. Liu and G. Bhattacharyya. "Some New Constructions of Bivariate Weibull Models," Ann. Inst. Statist. Math. 42 (1990): 543-59.
R. L. Prentice and J. Cai. "Covariance and Survival Function Estimation Using Censored Multivariate Failure Time Data," Biometrika 79 (1992): 495-512.
R. D. Siddhartha, B. F. Edward, and H. Bruce. "Risk Analysis of the Space Shuttle: Pre-Challenger Prediction of Failure," J. of the American Statistical Association 84 (1989): 945-57.
W.-Y. Tsai, S. Leurgans, and J. Crowley. "Nonparametric Estimation of a Bivariate Survival Function in the Presence of Censoring," Annals of Statistics 14 (1986): 1351-65.

Contents
ISSO -- Institute for Space Systems Operations
1997-1998 Annual Report