Chapter 9 Maintenance and Replacement

1. Chapter 9 Maintenance and Replacement The problem of determining the lifetime of an asset or an activity simultaneously with its management during that lifetime is an important problem in practice. The most typical example is the problem of optimal maintenance and replacement of a machine; see Rapp(1974) and Pierskalla and Voelker(1976). Other examples occur in forest management such as in Naslund(1969), Clark(1976), and Heaps(1984), and in advertising copy management as in Pekelman and Sethi(1978).

2. 9.1 A Simple Maintenance and Replacement Model 9.1 A Simple Maintenance and Replacement Model T = the sale date of the machine to be determined,  = the constant discount rate, x(t) = the resale value of machine in dollars at time t ; let x(0) = x0 , u(t) = the preventive maintenance rate at time t (maintenance here means money spend over and above the minimum required for necessary repairs), g(t) = the maintenance effectiveness function at time t (measured in dollars added to the resale value per dollar spent on preventive maintenance) ,

3. d(t) = the obsolescence function at time t (measured in terms of dollars subtracted from x at time t),  = the constant production rate in dollars per unit time per unit resale value; assume  >  or else it does not pay to produce. It is assumed that g(t) is a nonincreasing function of time and d(t) is a nondecreasing function of time, and that for all t where U is a positive constant.

4. The state variable x is affected by the obsolescence factor, the amount of preventive maintenance, and the maintenance effectiveness function. Thus, In the interests of realism we assume that This implies that the resale value of machine can’t increase.

5. 9.1.1 Solution by the Maximum Principle The standard Hamiltonian as formulated in Section 2.2 is where the adjoint variable satisfies Since T is unspecified, the additional terminal condition of (3.14) becomes which must hold on the optimal path at time T . The adjoint variable  can be easily obtained by integrating (9.6), i.e.,

6. The interpretation of (t) is as follows. It gives in present value terms, the marginal profit per dollar of gain in resale value at time t. The first term represents the value of one dollar of additional salvage value at T brought about by one dollar of additional resale value at the current time t. The second term represent the represents the present value of incremental production from t to T bought about by the extra productivity of the machine due to the additional one dollar of resale value at time t .

7. To find how the optimal control switches, we need to examine the switching function in (9.9). Rewriting it as It is clear that the optimal control in (9.9) can now be rewritten as Note that all the above calculations were made on the assumption that T was fixed, i.e., without imposing condition (9.7). On an optimal path, this condition, which uses (9.5), (9.7), and (9.8), can be restated as

8. This means that when , we have and when , we have Since d(t) is nondecreasing, g(t) is nonincreasing, and x(t) is nonincreasing, equation (9.13) or equation (9.14), whichever the case may be, has a solution for T*.

9. 9.1.2 A Numerical Example Suppose U=1, x(0)=100, d(t)=2,  =0.1, =0.05, and g(t)= 2/(1+t)1/2. Let the unit of time be one month. First, we write the condition on ts by equating (9.10) to 0, which gives In doing so, we have assumed that the solution of (9.15) lies in the open interval (0,T). As we shall indicate later, special care needs to be exercised if this is not the case. Substituting the data in (9.15) we have which simplifies to

10. Then, integrating (9.3), we find and hence Since we have assumed 0< ts <T, we substitute x(T) into (9.13), and obtain which simplifies to We must solve (9.16) and (9.17) simultaneously. Substituting (9.17) into (9.16), we find that ts must be a zero of the function

11. A simple binary search program was written to solve this equation, which obtained the value ts =10.6. Substitution of this into (9.17) yields T=34.8. Since this satisfies our supposition that 0< ts <T , we can conclude our computations. Thus, the optimal solution is to preventive maintenance at the maximum rate during the first 10.6 months, and thereafter not at all. The sale date is at 34.8 months after purchase. Figure 9.1 gives the functions x(t) and u(t) for this optimal maintenance and sale date policy.

12. Figure 9.1: Optimal Maintenance and Machine Resale Value

13. If, on the other hand, the solution of (9.16) and (9.17) did not satisfy our supposition, we would need to follow the procedure outlined earlier in the section. This would result ts= 0 or ts=T. If ts= 0, we would obtain T from (9.17), and conclude u*(t)=0, 0 tT. Alternatively, if ts=T, we would need to substitute x(T) into (9.14) to obtain T. In this case the optimal control would be u*(t)=U, 0 tT.

14. 9.1.3 An Extension The pure bang-bang result in the model developed above is a result of the linearity in the problem. The result can be enriched as in Sethi(1973b) by generalizing the resale value equation (9.3) as follows: where g is nondecreasing and concave in u. For this section, we will assume the sale date t to be fixed for simplicity and g to be strictly concave in u,i.e., gu 0 and guu < 0 for all t . Also, gt 0 , gut < 0 ,and g(0,t)=0; see Exercise 9.5 for an example of function g(u,t).

15. The standard Hamiltonian is where  is given in (9.8).To maximize the Hamiltonian, we differentiate it with respect to u and equate the result to zero. Thus, If we let u0(t) denote the solution of (9.21), then u0(t) maximizes the Hamiltonian (9.20) because of the concavity of g in u. Thus, for a fixed T, the optimal control is

16. To determine the direction of change in u*(t), we use (9.21) and value (t) from (9.8) to obtain Since  > , the denominator on the right-hand side of (9.23) is monotonically decreasing with time. Therefore, the the right-hand side of (9.23) is increasing with time. Taking the time derivative of (9.23), we have But and . it is therefore obvious that .

17. To sketch the optimal control u*(t) specified in (9.22), define , such that for t t1 and for tt2. Then, we can rewrite the sat function in (9.22) as In (9.24), it is possible to have t1 =0 and/or t2 =T . In Figure 9.2 we have sketched a case when t1 >0 and t2 <T. Note that while u0(t) in Figure 9.2 is decreasing over time, the way it will decrease will depend on the nature of the function g . Indeed, the shape of u0(t) , while always decreasing, can be quite general.

18. Figure 9.2: Sat Function Optimal Control

19. In particular, you will see in Exercise 9.5 that the shape of u0(t) is concave and, furthermore u0(t) >0, t 0, so that t2 = T in that case.

20. 9.2 Maintenance and Replacement for a Machine Subject to Failure T = the sale date of a machine to be determined, u(t) = the preventive maintenance rate at time t; 0 u(t) 1 R = the constant positive rate of revenue produced by a functioning machine independent of its age at any time, net of all costs except preventive maintenance,  = the constant discount rate, L = the constant positive junk value of the machine independent of its age at failure,

21. B(t) = the (exogenously specified) resale value of the machine at time t, if it is still functioning; h(t) = the natural failure rate (also termed the nature hazard rate in the reliability theory); F(t) = the cumulative probability that the machine has failed by time t , C(u,h) = the cost function depending on the preventive maintenance u when the natural failure rate is h. To make economic sense, an operable machine must be worth at least as much as an inoperable machine and its resale value should not exceed the present value of the potential revenue generated by the machine if it were to function forever.

22. Thus, Also for all t > 0, Finally, the cost of reducing the natural failure rate is assumed to be proportional to the natural failure rate. Specifically, we assume that C(u,h)=C(u)h denotes the cost of preventive maintenance u when the natural failure rate is h. In other words, when the natural failure rate is h and a controlled failure rate of h(1-u) is sought, the action of achieving this reduction will cost C(u)h dollars.

23. It is assumed that which gives the state equation Using (9.29), we can rewrite J as follows: The optimal control problem is to maximize J in (9.30) subject to (9.29) and (9.26)

24. 9.2.1 Optimal Policy The problem is similar to Model Type (f) in Table 3.3 subject to the free-end-point condition as in Row 1 of Table 3.1. Therefore, we follow the steps for solution by the maximum principle stated in Chapter 3. The standard Hamiltonian is and the adjoint variable satisfies Since T is unspecified, we apply the additional terminal condition (3.14) to obtain (See Exercise 9.6)

25. Interpretation of (9.33),the first two terms in (9.33) give the net cash flow, to which is added the junk value L multiplied by the probability [1-u*(T)]h(T*) that the machine fails. From this, we subtract the third term which is the sum of loss of the entire resale value B(T*), and the loss of the entire resale value when the machine fails, Thus, the left-hand side of (9.33) represents the marginal benefit of keeping the machine. Equation (9.33) determining the optimal sale date is the usual economic condition equating marginal benefit to marginal cost.

26. In the trivial case in which the natural failure rate h(t) is zero or when the machine fails with certainty by time t (i.e., F(t)=1), then u*(t)=0, Assume therefore h>0 and F <1 . Under these conditions, we can infer from (9.27) and (9.34) that

27. Using the terminal condition from (9.32), we can derive u*(T) satisfying (9.35): The next question is to determine how u*(t) changes over time. Kamien and Schwartz(1998) have shown that ; see Exercise 9.7. That means there exists such that

28. Here u0(t) is the solution of (9.35) (iii), and it is easy to show that . Of course, u*(T) is immediately known from (9.36). If u*(T)  (0,1) , it implies t2= T ; and if u*(T)=1, it implies t1= t2=T . For this model, the sufficiency of the maximum principle follows from Theorem 2.1; see Exercise 9.8.

29. 9.3 Chain of Machines We now extend the problem of maintenance and replacement to a chain of machines. By this we mean that given the time periods 0,1,2,…,T-1, we begin with a machine purchase at the beginning of period zero. Then, we find an optimal number of machines, say l, and optimal times 0< t1 < t2 <… t l–1< t l < T of their replacements such that the existing machine will be replaced by a new machine at time tj, j =1,2,… l. at the end of the horizon defined by the beginning of period T, the last machine purchased will be salvaged. Moreover, the optimal maintenance policy for each of the machines in the chain must be found.

30. Two approaches to this problem have been developed in the literature. The first attempts to solve for an infinite horizon (T= ) with a simplifying assumption of identical machine lives,i.e., for all j  1. Consider buying a machine at the beginning of period s and salvaging it at the beginning of period t > s. Let Jst denote the present value of all net earnings associated with the machine. To calculate Jstwe need the following notation for s  k t-1.

31. = the resale value of the machine at the beginning of period k , = the production quantity (in dollar value) during period k , = the necessary expense of the ordinary maintenance (in dollars) during period k , = the rate of preventive maintenance (in dollars) during period k , = the cost of purchasing machine at the beginning of period s ,  = the periodic discount rate.

32. It is required that We can calculate Jst in terms of the variables and functions defined above We must also have functions that will provide us with the ways in which states change due to the age of the machine and the amount of preventive maintenance. Also, assuming that at time s, the only machines available are those that are up-to-date with respect to the technology prevailing at s, we can subscript these functions by s to reflect the effect of the machine’s technology on its state at a later time k .

33. Let and be such concave functions so that we can write the following state equations: where  is the fractional depreciation immediately after purchase of the machine at time s . To convert the problem into the Mayer form, define

34. Using equations (9.43) and (9.44), we can write the optimal control problem as follows: subject to and the constraints (9.41), (9.42), and (9.39).

35. 9.3.1 Solution by the Discrete Maximum Principle We associate the adjoint variables respectively with the state equations (9.46), (9.47), (9.41), and (9.42). Therefore, the Hamiltonian becomes where the adjoint variables 1 ,2 , 3 ,and4 ,satisfy the following difference equations and terminal boundary conditions:

36. The solutions of these equations are: Note that are constraints for a fixed machine salvage time t . To apply the maximum principle, we substitute (9.53)-(9.56) into the Hamiltonian (9.48), collect terms containing the control variable uk , and rearrange and decompose H as

37. where H1 is that part of H which is independent of uk and Next we apply the maximum principle to obtain the necessary condition for the optimal schedule of preventive maintenance expenditures in dollars. The condition of optimality is that H should be a maximum along the optimal path. If uk were unconstrained, this condition, given the concavity of and , would be equivalent to setting the partial derivative of H with respect to u equal to zero, i.e.,

38. Equation (9.59) is an equation in uk with the exception of the particular case when and are linear in uk (which will be treated later in this section). In general, (9.59) may or may not have a unique solution. For our case we will assume and to be of the form such that they give a unique solution for uk . One such case occurs when and are quadratic in uk . In this case, (9.59) is linear in uk and can be solved explicitly for a unique solution for uk . Whenever a unique solution does exist, let this be

39. The optimal uk* is given as

40. 9.3.2 Special Case of Bang-Bang Control We now start the special case in which the problem, and therefore H, is linear in the control variable uk. In this case, H can be maximized simply by having the control at its maximum when the coefficient of uk in H is positive, and minimum when it is negative, i.e., the optimal control is of bang-bang type. In our problem, we obtain the special case if and assume the form and respectively, where and are given constants.

41. Then, the coefficient of ukin H, denoted by Ws(k,t ), is and the optimal control uk* is given by

42. 9.3.3 Incorporation into the Wagner-Whitin Framework for a Complete Solution Once uk* has been obtained as in (9.61) or (9.65), we can substitute it into (9.41) and (9.42) to obtain and , which in turn can be used in (9.40) to obtain the optimal value of the objective function denoted by .This can be done for each pair of machine purchased time s and sale time t > s .

43. Let gs denote the present value of the profit (discounted to period 0) of an optimal replacement and preventive maintenance policy for periods s, s+1, …, T -1. Then, With the boundary condition The value of g0 will give the required maximum.

44. 9.3.4 A Numerical Example Machines may be bought at times 0,1, and 2. The cost of a machine bought at time s is assumed to be The discount rate, the fractional instantaneous depreciation at purchase, and the maximum preventive maintenance per period are assumed to be respectively.

45. Let be the net return (net of necessary maintenance) of a machine purchased in period s and operated in period s . we assume In a period k subsequent to the period s of machine purchase, the returns , k > s, depends on the preventive maintenance performed on the machine in periods prior to period k. The incremental return function is given by , which we assume to be linear. Specially, where

46. This means that the return in period k on a machine purchased in period s goes down by an amount ds every period between s and k, including s, in which there is no preventive maintenance. This decrease can be offset by an amount proportional to the amount of preventive maintenance. Note that the function is assumed to be stationary over time in order to simplify the example. Let be the salvage value at time k of a machine purchased at s. We assume

47. The incremental salvage value function is given by where and That is, the decrease in salvage value is a constant percentage of the purchase price if there is no preventive maintenance. With preventive maintenance, The salvage value can be enhanced by a proportional amount.

48. Let be the optimal value of the objective function associated with a machine purchased at s and sold at t s+1. We will now solve for , s=0,1,2, and s<t  3, where t is an integer. Before we proceed, we will as in (9.64) denote by Ws(k,t), the coefficient of uk in the Hamiltonian H, i.e., The optimal control is given by (9.65). It is noted in passing that so that

49. This implies that In this example , which means that if there is a switching in the preventive maintenance trajectory of a machine, the switch must be from $100 to $0. Solution of Subproblems. We now solve the subproblems for various values of s and t (s < t) by using the discrete maximum principle.

50. From (9.65) we have Now, Similar calculations can be carried out for other subproblems. We will list these results.