Chapter 1 INTRODUCTION Consider the following questions: (i) How can I make comparisons between a nUlllber of populations, with confi­ dence that all of the cOlnparisons are correct? (ii) How can I select the best of a number of populations, with confidence in a correct selection? (iii) How can I select those populations which are best and are better than a control, with confidence of correct selectio~? An indirect luethod of answering such questions IS to conduct an analysis of variance and, if the hypothesis of hOlllogeneity is not supported, to conduct one of the many multiple comparisons tests. Often, however, an experin1enter is aware beforehand that there are differences between the populations and it is these very differences that she wishes to investigate. In this case, an analysis of variance is inappropriate. In this thesis, the above questions are investigated using a lllore direct solution, naillely ranking and selection, or multiple decision, theory. Three particular for­ mulations of this theory are discussed in this thesis: an indifference zone approach first introduced by Bechhofer (1954), a subset selection approach first considered by Gupta (1956) and a confidence bound approach initiated by Hsu (1981). Under 1 CHAPTER 1. INTRODUCTION 2 the indifference zone approach, a specified number of populations are selected with the assertion that these are indeed the best over a particular subset of parameter configurations. Under the subset selection approach, a (random-sized) subset of populations is selected with confidence that the true best are included over the entire parameter space. The confidence bound approach embraces both of these fornlulations by placing bounds on the differences between parameters of interest and using these bounds to make selection and ordering statements about the pop­ ulations. A confidence bound approach is adopted throughout this thesis, since this is seen to provide more infornlation than indifference zone- and subset selection-type rules. Because the selection decisions available under the other two formulations are contained in the statements which can be nlade under the confidence bound approach, the latter lends an appealing cohesiveness to the ranking and selection theory. In Chapters 4 and 5, the goals of comparing and selecting best populations are investigated. Lower confidence bounds are obtained in Chapter 4 on the difference between the worst selected and best non-selected parameters and upper and lower bounds are obtained on all parameters. In Chapter 5, improved bounds are in­ vestigated for the particular goal of selecting a subset containing best populations. Attention is turned in Chapter 6 to the goal of simultaneously cOlnparing popu­ lations with the best and with a control, again by placing confidence bounds on differences between parameters of interest. In each chapter, the practical application of the proposed rules is illustrated. The particular example pursued throughout the thesis is that of personnel selection, a field in which there is potential for the application of a wide variety of ranking and selection procedures. The personnel selection problem is discussed in detail in Chapter 3. Special results for the normal distribution are given for all goals in Chapters 4, 5 and 6 and appropriate tables are presented. The computation of these tables is also discussed at length. An important question that arises in Chapter 7 is that of the robustness of the procedures to this assulnption of normality. Prelirninary CHAPTER 1. INTRODUCTION 3 investigations into this problem are explored for the selection goal considered in Chapter 5. In the next chapter, a review of relevant literature is undertaken. This is mostly confined to work which is directly related to the comparison and selection problems considered in the later chapters. A precis of t.hese results is given in Appendix A. Chapter 2 LITERATURE REVIEW 2.1 Introduction This chapter discusses results in ranking and selection which are pertinent to the problems considered in subsequent chapters. Only single stage univariate results for parametric distributions obtained under a frequentist framework are considered. The norlllal distribution is discussed in detail. An introduction is given in this first section to three main approaches to ranking and selection, com1llonly known as the indifference zone (IZ), subset selection (SS) and confidence bound (CB) formulations. In Sections 2.2, 2.3 and 2.4, contributions 1nade under these three approaches for selection of best populations, comparison with a control and assessing robustness of selection rules, respectively, are reviewed. There is also some discussion of alternative designs and distributions, and related goals. A precis of the methods of a nUlllber of the authors referred to in this review 1S given in Table A.l of appendix A. Extensive reviews of ranking and selection literature are available in the reference texts of Gibbons, Olkin and Sobel (1977) and Gupta and Panchapakesan (1979) and, recently, excellent surveys of ranking and selection developlllents have been provided by Bechhofer (1985) and Gupta and Panchapakesan (1985). Other bibliographies have been compiled by Dudewicz and Koo (1985) and Gupta (1985), among others. 4 CHAPTER 2. LITERATURE REVIEW 5 2.1.1 Indifference Zone Approach Methods of making direct inferences about a number of treatments were proposed in the late 1940's and early 1950's by authors such as Wald (1947), Mosteller (1948), Mosteller and Tukey (1950), Duncan (1947, 1951,1952), Tukey (1949) and Scheffe (1953). These results are detailed in Miller (1981). The formal development of a theory of ranking and selection, however, is at ­ tributed to Bechhofer (1954). Consider the common problem of selecting the best normal mean ()i fronl k independent populations 1ri, i = 1, .. , k (with common known variance (J"2)~ based on the lnean Xi of n independent observations from each pop­ ulation. The Xi and ()i values are assumed t.o be ordered as follows: with 0 the entire parameter space. Then the goal becomes to select 1r p(k)' The "natural" selection rule (PI, say,) considered by Bechhofer is to select 1rR(k) with probability of correct selection (PCS) at least P*(I/k < P* < 1) whenever~. is in the "preference zone" with b a prespecified constant. (No requirement is made on the PCS if! is in the "indifference zone" 0 1 = 0 - Op.) The least favourable configuration (LFC) of means was shown by Bechhofer to be given by ()i = ()p(k) - b Vi #- p(k) and hence the infinlunl of thf' PCS was shown to be equal to inf p(){CSIPl} Op - inf P{Xp(k) > Xi Vi =I p(k)} Op / ~k-l( X + d)d~( x) (2.1 ) Pk(d) with d = yfnb / (J". Bechhofer suggested that his nlethod be used in the design stage of an experi­ lnent, to deternline the mininlum sample size n required for a given value of Pk ( d) CHAPTER 2. LITERATURE REVIEW 6 and b if there is no prior information about the true configuration of means. (Al­ ternatively, given nand b, Pk(d) may be calculated.) These results were obtained as a special case of the problem of completely ranking the k populations, which is discussed later in this review. Optimal properties of Bechhofer's (1954) have been considered by a large number of authors and have been reviewed by Gupta and Panchapakesan (1979). A criticism of the IZ approach is that b may be difficult to specify. Also, if the true configuration of means is far from least favourable, the procedure may be quite conservative. The loss function illlplicit in Bechofer's fornlulation and the fact that the PCS is only guaranteed over the preference zone have also been criticised; these points are discussed in a later section. In a recent paper, Bechhofer (1985) has discussed the origins of ranking and selection, in particular under the IZ fornlulation. In this paper, an overview was given of ilnportant results for selection of normal and Bernoulli populations. 2.1.2 Subset Selection Approach An alternative approach to the probleln of selecting the best normal population (with comnlon known variance) was introduced by Gupta (1956). Under this Subset Selection (SS) fonnulation, a random-sized non-empty subset of populations is to be selected, which contains 7r p(k) with at least a prespecified PCS over the entire paralneter space n. Gupta's procedure (P2, say,) is to select 7ri, Vi E G(d), with G(d) = {i: )[i > XR(k) - d}. The LFC occurs when all of the means are equal and hence the infimum of the PCS is given by igf P{ C S I P2} inf P{Xp(k) > Inax Xi - d} {1 14p(k) Pk(d) with Pd d) given by (2.1). CHAPTER 2. LITERATURE REVIEW 7 The prIce paid for not specifying [, ( and hence an indifference zone) is that possibly more than one population may be included in the selected subset, with no information as to which of these is the true best. This method is useful in the experilllental analysis stage and as a screening procedure to eliminate demonstrably inferior populations. As with the IZ approach, no prior information is required about the true configuration of paralnet.ers but again, if t.his is not. least favourable, the SS formulation may be conservative. The size of the selected subset is a randoIll variable and may be unacceptably large, especially if the configuration of means tends to be dose to least favourable. Properties of Gupta's (1956) rule have been considered by a number of authors, including Gupta and Panchapakesan (1979), Bjornstad (1985), Berger and Gupta (1980) and Gupt.a and Kim (1980). Gupta's procedure was also shown t.o be highly efficient for a particular Bayes' probleln considered by Chernoff and Yahav (1986) and was conjectured to be quite robust to a wide range of loss functions and adapt. ­ able t.o different priors. The evolution of the SS formulation has recently been traced by Gupta and Panchapakesan (1985). These authors have also given a sumlnary of important devel0Plllents under this approach. 2.1.3 Confidence Bound Approach An appealing method of cOlllbining both t.he 1Z and SS formulations was proposed by Hsu (1981). Unlike the 1Z and SS stateillents, which simply reject or select populat.ions, this third formulation constructs confidence bounds on relevant differ­ ences between population parailleters, which lllay then be used to Inake the desired selection decisions. For the problelll of selecting the largest normal 111.ean with COllllllon known vari­ ance, Hsu (1981) investigated lower and upper confidence bounds on of the fornl Oi - max OJ" Vi = 1, .. , k j::j:i (Xi - luax Xj~ + d)+ j::j:i CHAPTER 2. LITERATURE REVIEW 8 respectively. Hsu showed that over all 0, the joint confidence for the above bounds, for a particular value of d, is equal to the expression Pk ( d) given by (2.1). 2.1.4 Software A recent contribution to ranking and selection is the development of appropriate software and computer packages. To date, there are two general ranking and selec­ tion packages: RANKSEL, developed by Edwards (1985) and RS-MCB, developed by Gupta and Hsu (1984). The procedures available in RANKSEL are selection of the largest normal mean under both IZ and SS approaches and selection of all normal means larger than a control. This package is menu-driven and has a large collection of help files and error-recovery facilities. RS-MCB uses the CI approach to compare all normal means with the best (largest or smallest lllean), displaying the results both numerically and graphically. Appropriate IZ- and SS-type selection decisions based on these intervals are also in­ dicated. Both balanced and unbalanced one-way designs are allowed; for unbalanced designs, there is a "thrift" option which uses one (conservative) value of fJ rather than the default vector of values f RS-MCB, however, is not as user-friendly as RANKSEL and, because of the nature of the calculations, it is considerably slower. Both packages, however, are powerful with well-written procedures and are a most welcome addition to ranking and selection literature. 2.2 Selection of Best Populations 2.2.1 Indifference Zone Approach Location Parameters Bechhofer (1954) fornlulated his IZ procedure for the nl0re general goal of par­ titioning the k populations into r nonempt.y subset.s 51, ... , 5n such that 5i {7rP(Pi_l+1 )? .. ,7rp(pd} Vi = 1, .. ,r. The proposed procedure is, for i = 1, .. ,r, t.o CHAPTER 2. LITERATURE REVIE1¥ 9 include in the subset Si populations {7rR(Pi_l +lb ... , 7rR(pt}}. Results for the par­ ticular case of selecting the t largest means (without regard to order), so that T = 2,Pl = k - t,P2 = k are detailed in Table A.I. Bechhofer and Sobel (1954) and Bechhofer (1968a) extended these results to selecting the smallest scale parameter and tabulated, for the nornlal variances case, exact and approximate values of the PCS for a range of k, t, v and (T~(t+l)/(T~(t) values. Results for selecting and ordering the t smallest scale parameters were also given. For selected {P*, k, t} configurat.ions, Bechhofer (1954) tabulated the values of d satisfying Pk ( d) = P*. For t = 1, ot.her tables for this expression have been com­ puted by various authors, including Gupta (1963), lvIilton (1963) and Gupta, Nagel and Panchapakesan (1973). These t.ables have been reviewed by Gupt.a and Pan­ e hapakesan (1979) and Gupta, Panchapakesan and Sohn (1985) and are discussed in further detail in Chapt.er 4. Approximations to the sample size satisfying the probability requirement have been suggested by Dudewicz (1969), Raillberg (1972), Bechhofer, Kiefer and Sobel (1968) and Robbins and Sieglnund (1968). COlllparisons of these approximations have been Inade by Gupta and Panchapakesan (1979, pp. 21,22) and Dudewicz and Zaino (1971). Overall, it appears that Ramberg's approximation, based on the Slepian inequality, perfonns better over a wider range of k and P* values. If the value of n for specified values of P* and f, is unsatisfactory, Gibbons, Olkin and Sobel (1977, pp. 38-41) have discussed the implications of fixing nand f, and letting P* vary, or alternatively, fixing nand P* and letting b vary. Bechhofer's loss function for selection of the best population (7r p(k)) is given by Lcs = 1 - I(R(k~) = p(k:)) with I t.he indicat.or function. This has been criticised by a nUlllber of authors for placing the same penalty on choosing a population "close (but not. equal) to best" as choosing a population "far frolll best". Chernoff and ·Yahav (1986) severely criticised the formulations of Bechhofer (1954), Gupta (1956) and Desu and Sobel (1968) on this basis, arguing that. the PCS has little meaning without an appreciation of the cost of not select.ing the best populat.ion. Goel and Rubin (1977) and Reeves and CHAPTER 2. LITERATURE REVIElV 10 Sobel (1986) have also criticised this loss function. In terms of the complete ranking problem, the latter authors argued that one transposition of ranks is treated as severely as a completely inaccurate ordering. Their solution is considered in detail in a later section. Chen (1985) has also criticised the IZ formulation because, although the ana­ logue to the power of a test (controlling the PCS over Op) is satisfied, the analogue to the level of a test (controlling the PCS over all 0) is not considered. This ren­ ders the IZ approach inappropriate in many practical situations, as pointed out by Silllon (1977). Indeed, as Parnes and Srinivasan (1986) have noticed, satis­ faction of POp{R(k~) = p(k) I ()p(k) - ()p(k-l) 2:: 6} = P* (which is the probabil ­ ity requirement in Bechhofer ~s (1954) formulation) does not always ensure that Po {()R(k) 2:: ()p(k) - b} = P*. The second probability requirement is often achieved incidentally, as delllonstrated by Parnes and Srinivasan for Paulson's (1952) pro­ cedure and Bechhofer's (1954) normal means rule. Counterexamples of this were given, however, for multinomial probabilities and paired comparisons. Bishop and Pirie (1979) attempted to remedy this deficiency by propOSIng a class of selection procedures which allows no selection if a prior null hypothesis of homogeneity of parameters using a range t.est is not rejected. Chen (1981) extended this approach to simultaneous estimation and selection after a prelinlinary test. Chen (1985) proposed a similar method (see Table A.l) which entails no selection if fL is in a specified subset 0NZ of 0 - Op, selection of 7rR(k) if fL is in Op and neither selection nor rejection otherwise. This employs a llluch broader non-selection zone than the single point ( () p(l) = ... = () p(k)) considered by Bishop and Pirie (1979) and Chen (1981) and also differs froln these methods in that it is based on the largest and second largest order statistics rather than the largest and smallest order statistics. For the known variance case, the nlinimum required sample size required under Chen's procedure is always larger than that required under Bechhofer's (1954) rule, due to the introduction of a non-selection zone and hence a restriction on the level of the test. For the unknown variance case, Chen recommended either the specification of the ratio f, / (j in a single stage rule or a two-stage salllpling procedure. One criticism of Chen's method is that if the two best populations are CHAPTER 2. LITERATURE REVIEW 11 "close", both will be rejected, instead of a more appealing rule in which either or both of them would be selected. Fabian (1962) considered Bechhofer's IZ-type procedure in a slightly different framework of ~-correct ranking (see Table A.l). With the same pes, Fabian's approach, which asserts (for the usual lecation parameter case) that 8p(k) lies in the interval (BR(k) - ~,8R(k)) for a certain positive ~ value which depends on ):R(k) - XR(k-l), was shown to be a stronger statement than Bechhofer's (1954), which asserts that Bp(k) > BR(k) - d, with d > ~. Fabian's results were used by Hsu( 1981) in the fOrlnation of his confidence bounds, as discussed later in this re­ view. For the general ranking probleln, Fabian showed that the infimum of the PCS under his approach is always at least as large as the corresponding expression under Bechhofer's approach, with equality in the special case of selecting ?T p(k). Fabian's (1962) rule for selecting the t best populations was generalised by Feigin and Weisslnan (1981), using a monotone function 1/,( 8) > 8 such that the preference zone is defined as with r = {p(A> - t + 1), .. ,p(k)}. A corresponding definition of 1/'-correct selection (1/' - C 5) was also used. Feigin and Weissman proved that, for t = 1, their rule R is consistent; that. is, Fabian's (1962) result was shown to be a special case of Feigin and Weissnlan's result. This consistency property was also derived by Chiu (1974) for the particular case of nornlal populations and was proved by Bofinger (1986a) for general t values for location (see Table A.l) and scale paranleters. The idea of a 7./' - C 5 was also used by Carroll, Gupta and Huang (1975) for elinlinating the non-t-best populations (further detailed ill Table A.l and Chapter 5). Unequal variances and sample sizes A pro bleln arising under the IZ formula­ tion is of optilnal allocation of possibly unequal salnple sizes ni, i = 1, .. , k, subject to a fixed total .IV = ~7=1 ni, with variances cr; known or unknown and possibly CHAPTER 2. LITERATURE REVIEW 12 unequal. For the case of unequal known variances, Bechhofer (1954) suggested al­ locating sample sizes to the k populations in such a way that the variances of the sample means are equal, but noted that this was perhaps not optimal. The intuitive reasoning that an increase in any ni, i = 1, .. , k will result in an increase in the PCS was assumed and used by Ofosu (1973), but such an assumption was proved by counterexample to be invalid by Lam and Chiu (1976). These authors showed that, for k = 3 and for small equal differences fJ* = Op(k) - Oi, with equal values of a-; In1' an increase in np(k) results in a decrease in the PCS. Using a different (and arguably easier) method, Tong and Wetzell (1979) proved this result for general k > 2, with possibly unequal variances and for (possibly unequal) fyi, when t.he values n~/2bil(7"i,i = 1, .. , k are sufficiently small. It was also proved that, for general k, an increase in np(ib i < k, does in fact result in an increase in the PCS. This result has also been discussed by Dudewicz and Dalal (1975), Rinott (1978) and Bofinger(1979, 1985a). In the latter paper, Bofinger criticised Tong and Wetzell's (1979) general caution not to take too many observations from 7T"p(k} if the hi values are small and N is fixed, and also Dudewicz and Dalal's (1975) suggestion of an alternative (inefficient) procedure. Instead, Bofinger highlighted conditions which ensure that the PCS does not decrease in Tlp(k) in the LFC and also applied these results to other problems, such as Gupta's SS rule and the two-stage nlethods of Dudewicz and Dalal (1983) and Bofinger and Cane (1978). This demonstrates the importance of the allocation problenl, but highlights the complexit.y of an exact solution for k > 2, since any change in np(i) (for any i) in order to st.udy the effect on the PCS also changes the correlation structure of the underlying lllultivariate normal distribution. For large sample sizes, Tong and Wetzell (1985) provided a partial asymptotically optimal solution (see Table A.I) based on certain majorisation inequalities, but this is nonimplementable since it depends on t.he unknown paramet.ers bi (and possibly a-;), i = 1, .. , k. Tong and "''''etzell hence proposed an adaptive allocation procedure which estilnates {land Q2 and identifies 7T" p( k) sequentially. For the probleul of selecting the largest nonnal lnean with unknown and pos­ sibly unequal variances and possibly unequal salnple sizes, Dudewicz (1971) and CHAPTER 2. LITERATURE REVIEW 13 Dudewicz and Dalal (1975) have shown that no single-stage IZ-type procedure could satisfy the probability requirement. The latter authors proposed instead 2-stage procedures under both SS and IZ formulations. Optimum sampling from continuous distributions under the IZ formulation has also been considered by Somerville (1954) using the criterion of maximum expected loss, with particular results for selection of the largest normal mean (known vari­ ance). Gupta and Huang (1977) used a r-nlininlax (minimisation of the maximum expected risk over r, a class of prior distributions) criterion for more general location and scale parameters. For the nOrInal means case, with COlnmon known or unknown variance, Reeves and Sobel (1986) have discussed the problem of an optimal sample size n needed to obtain "good" complete rankings. Instead of the traditional 0 - 1 loss function, the authors considered three non-parametric loss functions based on ranks. None of these was shown to be uniformly superior and all three placed the same penalty on a transposition of two adjacent "close" populations as one of two adjacent but "distant" populations. For 81 < (}2 < .. < ()k, the authors proposed an alternative parametric loss function and investigated the LFC, the expected loss and various approximations to the corresponding value of n. This is a refreshing alternative to the traditional loss function, but it still appears to be conservative, since the LFC is often not justified. 2.2.2 Subset Selection Approach Location Parameters Gupta's (1956) SS rule for selecting the nornlal population (with common known variance) with the largest Inean is included in Table A.I. As already noted, the appropriat.e critical value is the sanle as that required for Bechhofer's (1954) pro­ cedure. Approxilnations to d, based on a lower bound for the PCS, have been proposed by Carroll (1974). Jones, Butler, Wright. and Swain (1987) also exanlined CHAPTER 2. LITERATURE REVIEW 14 an approximation to d, uSIng an extra term in the finite senes expansion of the probability expression considered by Gupta (1956). Gupta'5(1956) results for the comlll0n unknown variance case are also presented in Table A.I. The corresponding critical values, which are lOOp· percentage points of a 11lultivariate t distribution on v degrees of freedom, have been tabulated by such authors as Dunnett (1955), Gupta and Sobel (1957) and Krishnaiah and Armitage (1966). Gupta, Panchapakesan and Sohn (1985) have reviewed the coverages and derivation of these tables and further details of the tables are also given in Chapter 4. Gupta~ Panchapakesan and Sohn's (1985) very extensive tables were applied by the authors to other selection problelns, including cOlnparison with a control (Gupta and Sobel, 1957), prediction intervals to contain all k future Ineans and multiple comparisons of 111ean vectors in a general J\1anova lllodel (Krishnaiah, 1965). Gupta and Panchapakesan (1972) considered the SS problem for a more general class of stochastically increasing distributions {Fe,}, i=l .. k .Their procedure (see Table A.l) employs a class of functions hc,d(X) and reduces to Gupta's (1956) rule for the location paramter case with the usual choice h( x) = x + d, d 2: o. A criticism of Gupta's (1956) SS procedure is that if there is lllore than one population sharing the largest parameter value, only one of these need be selected to satisfy the probability requirement. An alternative approach taken by Chen and Dudewicz (1983), for stochastically increasing populations, selects all populations with Bi = Bp(k) in the subset with a minill1Ulll guaranteed confidence. Both location (see Table A.l) and scale paralneter cases were considered in detail. For the normal 111eanS case, their lllethod reduces to a Gupta-type selection rule, with the critical value d obtained as a percentage point of the range distribution (known variance) or studentised range distribution (CoIll111on unknown equal variance). Exponential distributions with both guaranteed knovnl and unknown spans were also discussed by the authors, with tables for the latter case. Chen and Dudewicz investigated how prior knowledge, such as the true nUlnber of best populations, could be used to select a stochastically Slllaller subset of populations. The authors also showed by nUll1erical exalllple that their rule selects a larger subset than Gupta's (1956) corresponding rule, but with confidence that all populations 7ri with Bi = Bp(k) are CHAPTER 2. LITERATURE REVIEW 15 included, which was argued to be more desirable in practice. Unequal variances and sample sizes A SS procedure for nornlal means with common known variance and unequal sample sizes was proposed by Gupta and Huang, W.-T. (1974) (see Table A.l). Gupta and Huang, D.Y. (1976) considered a slightly different solution to the same problem (see Table A.l) and, like Gupta and Huang (1974), also proposed rules for unequal sanlple sizes with common unknown variance, which are included in Table A.I. Both pairs of authors investigated upper bounds for the expected subset size and lower bounds for the infimunl of the PCS. An alternative solution to this problenl was considered by Sitek (1972) (see Table A.l), but her expression for the infimum of the PCS is incorrect, as pointed out by Chen, Dudewicz and Lee (1976). The latter aut.hors proposed a 111odification, included in Table A.l, of Sitek's rule, which reduces to Gupta's (1956) rule for equal sample sizes. The optimal choice of the constant a, which nlinilnises t.he expected sample size for fixed P*, however, was not. satisfactorily solved. By introducing a non-negative function aj = aj( nJ, .. , nk), Chen, Dudewicz and Lee generalised their procedure to include the results of Gupta and Huang (1976) and Gupta and Huang (1974) as special cases. The authors argued that. their rule was preferable to Gupta and Huang's (1974) and Gupta and Huang's (1976) rules, in terms of ease of calculation of the critical values and availability of appropriate tables. Also using aSS approach, Gupta and \Vong (1932.) proposed a single-stage procedure for selection of the largest. normal lnean with unknown and possibly unequal variances and possibly unequal sa111ple sizes. An upper bound was derived on the expected subset size. Selection of the t best A nUlnber of authors have considered the problem of selecting a subset containing all "good" populations wit.h a prespecified confidence, wit.h different definitions of "good" and different corresponding procedures. Pan­ chapakesan and Santner (1977) posed t.wo goals (see Table A.l) for st.ochastically increasing dist.ributions: a) t.o select. a nonenlpty subset. containing only good popu­ lations and b) to select. a subset of lllaxinlU111 size TIl (1 ~ TIl < k) including at least. CHAPTER 2. LITERATURE REVIEW 16 one good population. Expressions for the PCS and expected number of selected populations were derived and a large sample consistency property of the procedure for one of the cases was also established. Desu (1970) proposed a slight generalisation of the "good-bad" definition of pop­ ulations discussed by Lehmann (1961) (whose results are considered later in this review and in Table A.l), for a related problem of excluding non-best populations from a subset containing the best. Desu's procedure for the location parameter case is described in Table A.I. Gupta and Panchapakesan (1979, p.227) pointed out that for this particular case, Desu's critical value Co will be less than the spec­ ified constant <5; (see Table A.l) for a proper choice of n; this is achieved if the probability of correct elimination tends to one as n tends to infinity. Desu (1970) investigated the perforll1ance of his procedure by considering the expected number of inferior populations in the selected subset. Generalisations of this problem have been considered by Naik (1977) for t = 1 and by Carroll and Gupta (1977) for both location and scale parameter cases. The latter authors defined a non-t-best population 7ri to be one for which a specific function 1j.' of 8i and 8p(k-t+l) is greater than S0111e positive value ~. Huang and Panchapakesan (1978) approached the problem of completely rank­ ing the parameters 8i,i = 1, .. , k by selecting a (non-empty) subset of all the k! perll1utations of the set of integers {I, .. , k} to include the correct ranking (see Table A.l). The authors showed that for both location and scale parameter cases, their procedure is unbiased and they extended their results to stochastically increasing families and distributions star-shaped with respect to a known distribution. Scale Paralneters Gupta and Sobel (1962a) approached the problell1 of selecting the nonnal population w·ith the slnallest variance using a SS fonnulation. Known and unknown means were simultaneously considered, using an appropriate adjustment. to the degrees of freedoll1 of the chi-squared distribution. Tables of the critical values, which are percentage points of t.he smallest. of several correlated F-statistics, were presented in a conlpanion paper (Gupta and Sobel, 1962b). Gupta and Sobel's (1962a) results CHAPTER 2. LITERATURE REVIE1V 17 for general t values are discussed in detail in Chapter 5. The authors also considered two particular examples of the unequal sample size problem and investigated the resulting behaviour of the PCS. For the unequal sample size case, Gupta and Huang (1976) used Gupta and Sobel's (1962a) rule to obtain two different lower bounds on the PCS and to derive some large scale approximations. Gupta and Panchapakesan's (1972) general approach to subset selection of the best scale parameter is analagous to their rule for location parameters, discussed above, and reduces to the rule considered by Gupta (1956) with the obvious choice h( x) = ex ~ c ~ 1. For this choice, the supremum of the size of the selected subset was discussed in detail. A class of procedures designed to select the slnallest normal variance, based on salnple ranges, was considered by McDonald (1977) using a SS approach and appropriate tables were presented. McDonald compared his procedure with that of Gupta and Sobel's (1962a), using Monte Carlo nlethods to deternline the ratio of the expected subset size and the PCS under both a slippage and a geollletric configuration of the parallleters. Both procedures were shown to be equivalent for slnall 11 and k ( 11 ~ 15 and k ~ 10) but Gupta and Sobel's 11lethod was demonstrably better for larger It: and 11. Both rules were shown to have a maximum expected subset size of kP*. The general results of Gupta and Panchapakesan (1972) for convex mixtures may also be applied to subset selection rules with respect to ranges. 2.2.3 Combined Approaches By the Inid-1950's the theory of ranking and selection was established under two fonnulations: the 1Z approach and the SS approach. For some problems, a combi­ nation of the two approaches was attempted, for example in the case of restricted subset size procedures. By defining an appropriate preference zone rlp based on SOllle distance function of the unknown parameters fl( not necessarily the silnple distance fUllction used by Bechhofer (1954)), Inethods were proposed to satisfy re­ strictions on the nUlllber of populations selected and simultaneously guarantee that the selected subset contains the required populations with a prespecified confidence CHAPTER 2. LITERATURE REVIEW 18 for all fl in Op. Mahanlanulu (1967) considered the problem of determining the smallest common sample size n for selecting a subset of fixed size s containing the t best populations; that is, those with the largest B-values. Desu and Sobel (1968) approached the inverse problem of determining, for a given value of n, the smallest fixed subset size s which would contain the t best populations. Sobel (1969) discussed a method of selecting a subset of fixed size s containing at least one of the t best. Later, Santner (1973,1975) proposed solutions to the problem of selecting a subset of no more than s populations containing the best population. Santner's (1973) results were extended by Gupta and Santner (197!) for selection of normal means with common known variance. Also using a restricted subset size approach, Deverman (1969) proposed a rule for restricted subset selection of stochastically increasing distributions (see Table A.l). Devernlan and Gupta (1969) particularised these results for selection of nor­ mal means and variances. Selection of a subset containing the best population, with a restriction on the maximum expected subset size, has also been investigated by Huang and Panchapakesan (1976) and is included in Table A.1. Bounds were obtained by these authors on the required probabilities for the normal means (with comnl0n variance) and normal variance case in order to calculate a (conservative) sample size. Using a conlbination of SS and IZ approaches, Sobel (1969) proposed three procedures for selecting a subset of at least size s containing at. least one of the t best, with a nl0dification for each procedure if it is required that the subset size not exceed k - t + 1. His first procedure, denoted by R, is described in Table A.1. The second procedure, R', is equivalent to R for alnlost all paralneters k, t, P* and fJ but, unlike R, the critical values are independent of the value of b. Sobel's third procedure, Rm is appropriate for the particular case s = t and reduces to Gupta's (1956) procedure for fJ = 0, t = 1. The introduction of an IZ provides an improvement over Gupta's procedure for other values of b. Conlparisons between these rules and with Gupta's rule were made with respect to expected subset size. Yen (1969) and Sobel and Yen (1972) have shown that the asymptotic ( P* ~ 1 ) efficiency of R with respect to Rm CHAPTER 2. LITERATURE REVIEW 19 depends on certain conditions on the parameters. Sobel (1969) also showed that the asymptotic efficiency of a likelihood ratio procedure suggested by Bechhofer, Kiefer and Sobel (1968) with respect to R for the normal means problem, with t = 1, is zero. A more general problem of selecting s from k populations which contain at least c of the t best location (and scale) parameters has recently been considered by Giani (1986) using an IZ formulation. Generalising Fabian's (1962) approach (discussed later in this review), Giani showed that the usual IZ results can be extended to include confidence statements for the parameters of the selected populations, which are valid over the entire parameter space, without decreasing the PCS. Giani's (1986) procedure, which employs a random variable D i , i = 1, .. , k (instead of the usual constant 8), with Di < [, almost surely, is described in Table A.I. Giani's results have been proven by Fabian for t = c = s = 1 and by Feigin and Weiss­ man (1981) for continuous stochastically increasing distri bu tions. For independent normal distributions, Chiu's (1974) conclusions also imply Giani's results. For the special case t = c = s > 1, Hsu's (1984b) lower bounds for the difference between ()i and the tth largest parameter among {()j, j #- i} are identical to Giani's bounds, but Giani's theorenl does not allow any SS inference available with Hsu's. The par­ ticular cases a) c = s :s: t and b) c = t < s discussed by Mahamanulu (1967) were also considered by Giani. For case a), in which any s of the t best populations are chosen, Giani's procedure reduces to that of Sobel (1969) but with global confidence P* (not merely in the preference zone) that the upper bounds on () p(k-t+l) - ()i are simultaneously correct for all selected populations. Giani also showed that if the upper confidence bounds on ()p(k-t+l) - ()i vanish, so that m = t - \{ i : Di > O}\ is positive, at least m, of the populations with Di = 0 also are the best ones. More recently, Sobel (1987) has proposed a new fonllulation for selecting the t best from k nonllal populations (see Table A.1). Its novelty is in selecting two disjoint subsets Sc and S[, with S = Sc U 51, when ~\R(k-t) and X R (k-t+l) are very close, and selecting the usual Gupta-type subset otherwise. This penllits a subset of all good populations to be selected,and particular identification of any substantially better than the tth best. This paper builds on results of Chen and Sobel (1987). CHAPTER 2. LITERATURE REVIEW 20 2.2.4 Confidence Bound Approach Location Parameters Comparison with the best Hsu's (1981) confidence bounds, discussed previ­ ously and detailed in Table A.l, have been shown by the author to be sharper than those arising from Tukey's silllultaneous confidence intervals, which use per­ centage points of the studentised range distribution as critical values. Hsu also developed confidence bounds for all distances from the best (location parameters) in a non-parailletric framework and showed that these, too, are sharper than the non-parailletric analogue of Tukey's confidence intervals. Hsu (1984a) detailed, for the one-way balanced design, the relationships between his formulation and those of Bechhofer (1954), Gupta (1956), Fabian (1962) and Desu (1970). The author showed that Gupta's confidence set has a true coverage probability which increases rapidly from P* to one as the difference between B p(k) and B p(k-l) increases, whereas the true coverage probability of Desu's upper bounds for (Bp(k) - Bi , i = 1, .. , k) decreases under the same circumstances. Fabian's upper bound on Bp(k) - BR(k) was shown to be extremely conservative, while Hsu's simultaneous coverage probabilities were much closer to the nominal confidence requirement. In a later paper, Hsu (1985) demonstrated that his constrained simultaneous confidence bounds lllay be replaced by unconstrained confidence intervals (see Table A.l) with little increase in the associated critical values. This difference between the critical values was shown to decrease as k increases and is negligible for llloderate to large k. For k = 2, the constrained critical value is the one-tailed t value whereas the unconstrained value is the two-tailed t value. Modifications required for unbalanced designs, which use a vector of critical values, were noted to be t.he sanle as for the constrained intervals (Hsu, 1984b). A non-parametric analogue was also derived. In this paper, Hsu also detailed very carefully the types of inferences that could be lllade with different multiple conlparisons techniques, for all pairwise comparisons, lllultiple cornparisons with the best and Inultiple cOll1parisons with a control. Edwards and Hsu's (1983) results, which are based on Hsu's (1981) results, are discussed in a later section on cOlllparison with a control. Their generalisation of CHAPTER 2. LITERATURE REVIEW 21 Hsu's confidence bounds for the normal variance case is described in Table A.I. Simultaneous upper bounds on 0-; /0";(1) were obtained by setting d- = 0; for this case, d+ is tabulated by Armitage and Krishnaiah (1964) and Gupta (1965). A number of bounds on OR(k) - maxi,tR(k) 0i have been developed by various authors. Guttmann (1984) and Bofinger (1988a) investigated lower bounds of ei­ ther zero or -00 for the cases of k normal distributions with common known and unknown variances. For k = 3 or 4, Guttmann proved that for confidence in the usual range of hypothesis testing, the critical value is independent of the number of populations (unlike Bechhofer's (1954) results) and is equal to the value ta,v used in a 2-sided 2-sample test for a difference between means. For t.he problem of selec­ tion of the best normal population with common known variance (not constructing lower bounds), Bofinger (1988a) demonstrated that, for general values of k and for large P*, the one-sided critical t value for a two-sample test of difference between means satisfies the probability requirement (see Table A.l). A similar result, but with more stringent requirements on the value of P*, was shown to hold for the unknown variance case. Fabian (1962) investigated negative lower bounds on the difference between the selected best and the true best. GuttInann and MaYlnin (1984) and Stefansson, Kim and Hsu (1988) considered instead positive lower bounds on this difference. A compromise between negative and positive lower bounds was reached by Bofinger (1983), for both known and unknown variances. Guttmann and Maymin's nonlinear bounds were derived in a conditional setting for distributions with monotone like­ lihood ratio and known variance. There are difficulties, however, in extending this conditional setting to the case of unknown variances. Stefansson,Kim . and Hsu's (1982) approach is appropriate for known and unknown variances. They, too, used the critical value ta,v in their exact confidence set for multiple conlparisons with the best, based on non-equivariant tests. Bofinger (1983), on the other hand, used a critical value d > 0 satisfying Pk ( d) - (1 - P2 ( d)) = P* (and hence increasing with k). A precis of these results is given in Table A.I. For location paranleters of continuous distributions, Bofinger (1987) adopted a similar approach to Hsu (1985) and considered constrained (so that lower bounds CHAPTER 2. LITERATURE REVIEW 22 are positive) and unconstrained linear and non-linear confidence bounds for the particular problems of comparison with the best and comparison with the t best. Bofinger used Guttmann and Maymin's (1984) lower bounds for 8p (k) for the known variance case and applied them to the unknown variance case. This bound was compared with Hsu's (1984b), Bofinger's (1983), Guttmann and Maymin's (1984) and Kim, Stefansson and Hsu's (1986) bounds, both theoretically and numerically. Under a rule which selects the t largest observed location parameters, Bofinger considered unconstrained bounds on the difference between the worst selected and best non-selected parameters. These bounds were improved for special cases. Her results for the nornlal distribut.ion are given in Table A.I. Somerville (1987) has also considered bounds for 8p (k) - 8R (k) for k = 3, using a geonletric interpretation. His bounds were compared with those of Fabian (1962) and Sonlerville (1984) and regions of superiority of the various bounds were iden­ tified. Sinlultaneous confidence intervals for all pairwise differences between means were also considered, along the lines of Hsu (1984b) and Gupta and Hsu (1984). Selection of the t best A CB approach to the general problem of selecting t from k populations was investigated by Bofinger (1984) using a definition of a ~­ correct decision (~ - CD) as a special case of Fabian (1962). For the location parameter case, for example, Bofinger defined a ~ - CD to be made if the smallest selected parameter is not more than ~ below t.he largest non-selected paralneter. In a later paper, Bofinger (1986a) applied these results to a more general class of paralneters (see Table A.l), using an increasing function 'ljJ( 8) suggested by Barr and Rizvi (1966) and a corresponding definit.ion of a 'lj)-correct decision, silnilar to that proposed by Feigin and Weissman. A stronger confidence statement was investigated by Bofinger (1986b) using a lower bound on 8R (k-t+l) as a function of 8R(J\~-t) for a more general class of distributions sat.isfying certain conditions. In the latter paper, Bofinger obtained lower and upper bounds on all paramet.ers, the lower bounds being functions of the tth "best" paralneter and the upper bounds being funct.ions of the (k' - t )th "worst." paranleter, and denl0nstrated the use of these bounds to a) select good populat.ions, b) t.o eliminate bad populations and c) t.o do CHAPTER 2. LITERATURE REVIEW 23 both a) and b). Table A.l details Bofinger's results for the location parameter case, for which the author has shown that the required conditions always hold. For this case and the usual scale parameter case, her procedure reduces to Carroll, Gupta and Huang's (1975) procedure and may be generalised as shown by them. The derived bounds were shown to be slightly better for the location parameter case than those of Hsu (1984b) (see Table A.l). For the location parameter case, assuming that. {8p(k-t+lb .. , 8p(k)} have been chosen as the t best, Bofinger (1986b) const.ructed bounds on the differences between a) the worst selected and the best. not selected parameters, b) the best selected and the best not. selected parameters, and c) the average selected and average not selected parameters. These bounds on the populations (after selection) apply over the entire parameter space and were argued to be more useful than t.he traditional approach of estilnating the PCS after selection has taken place (see Table A.l). Appropriate existing t.ables for all of her bounds were detailed by Bofinger. Complete ranking A CB approach to the complete ranking problem for normal means has been provided by Bofinger (1985c), for situations in which the confidence intervals thell1selves are not. of primary interest, but. in which the Type III error (defined to be the conclusion that one population is better than another when ac­ tually it is worse) is still controlled. This nlethod, described as "honest ordering", is detailed in Table A.I. Bofinger's critical values were shown to be smaller than Tukey's (which are standardised range percentage points) and were tabulated for k = 2,3. Lower bounds on the probability of correct ordering were derived for gen­ eral k~ and t.he (conservat.ive) critical values satisfying these bounds were tabulated for k = 4, .. , 10. In another paper, Bofinger (1985d) argued that it. is sometilnes more pract.ical to "expand" a confidence interval for a parameter in either direction, perhaps to t.he nearest int.eger, and considered t.he resulting increase in confidence for such in­ tervals for t.he location paralneter case. Her results for the particular case of normal Ineans with comInon unknown variance are included in Table A.I. Using her "hon­ est ordering difference", Bofinger applied these bounds to the goal of all pairwise CHAPTER 2. LITERATURE REVIEW 24 comparisons of treatments and comparison with a control. For the latter problem, the new bounds were shown to be superior to the two-sided bounds proposed by Dunnett (1955). The author also noted that Hsu's (1984b) constrained intervals for the tth best are like expanded confidence intervals. Unequal variances and sample sizes Hsu (1984a) advocated that the CB approach brings a degree of uniformity to the unbalanced design problem. By using a vector of critical values to construct MCB intervals (see Table A.l), Hsu showed that the same methodology may be used for both equal and unequal sample sizes. This nlethod was demonstrated to be at least as efficient as Gupta and Huang's (1976) SS procedure for unequal sample sizes, the latter being shown by Berger and Gupta (1977) to be the only minimax rule among the comparative procedures of Chen, Dudewicz and Lee (1976), Gupta and Huang (1974) and Gupta and Wong (19ca.z.). The sharper inferences are attributed to taking the maximum of the upper bound over a subset C of {I, 2, .. , k} instead of the entire set. Edwards and Hsu (1983) also used a vector of critical values to extend their results to the unequal sample size case (see Table A.l). 2.2.5 Other Designs Two single-st.age procedures for select.ing the best level of each of the r factors under an r-way (r 2 2) classification without interaction, for normal populations (with COIllmon known variance), was considered by Bechhofer (1954) using an IZ approach. Both procedures are detailed in Table A.l for the two-way classification. The factorial procedure, which selects as best the level corresponding to the largest lllarginal sample lllean, based on n observations frolll each population, was shown to be Illore efficient. than the more tradit.ional one-at-a-tillle rule. This was confinned by Bawa (1972). U sing the IZ approach but with a different preference zone than that of Bech­ hofer's (1954), Bechhofer, Santner and Turnbull (1977) investigated the problem of selecting the largest interaction in a two-factor experilllent (for normal populations with COIllIllon known variance) with r levels of the first factor and c levels of the CHAPTER 2. LITERATURE REVIElV 25 second. Their procedures for two special cases are detailed in Table A.I. Bechhofer, Santner and Turnbull noticed that their problem is similar to that of comparing k populations with a standard (zero), since the interaction term is only of interest if it is sufficiently larger than zero. Rasch (1978) considered such selection prob­ lellls for balanced block designs. A generalisation of the problem of selecting the largest interaction in an r x c experiment was examined by Santner (1978), who used nonlinear programlning techniques to obtain the minimunl PCS using an IZ approach. Dudewicz (1977) and Bechhofer (1977) independently derived an IZ-type proce­ dure for the two-fact.or experinlent with int.eraction~ noting that their procedure may generalise t.o any nUlllber of factors. The rule, which select.s as best t.he largest saln­ pIe 11lean based on 11 independent observations frolll each populat.ion~ was shown by Dudewicz and Taneja (1982) to be valid for t.he complet.e fact.orial case wit.h known variance, when t.he goal is to select. the population with the largest mean, whet.her or not there is any interaction. They showed, however, that Bechhofer's (1954) procedure is asympt.otically lllore efficient. whenever there is no interaction in the lllodel. Bechhofer (1977) also considered a SS approach to this problelll. Taneja and Dudewicz (1984) proposed an alternative selection procedure (de­ tailed in Table A.l) which involves taking 11 independent observations from each of the k populations and testing the hypothesis that. all the interactions are zero. If t.he hypothesis is rejected at level Q, Dudewicz's (1977) and Bechhofer's (1977) procedures are applied; otherwise Bechhofer's (1954) 11lethod is used. U sing a combination of the IZ and SS approaches~ Huan1[ and Panchapakesan (1976) considered the probleln of selecting a subset containing t.he best. treatment under a complete block design with one observat.ion per cell, wit.h an added require­ ment. on t.he expected subset. size. Huan~ and Panchapakesan reduced the general problelll t.o a one-way design problenl (see Table A.1). Federer and McCulloch (1985) have compared Hsu's (1981) int.ervals for multi ­ ple cOlllparisons wit.h t.he best. (MCB) with three 11lultiple cOlllparison procedures, the LSD, St.udentised range (HSD) and Bonferroni (ESD) methods, for comparing CHAPTER 2. LITERATURE REVIEW 26 means from split plot and split block designs. They suggested the use of MCB in­ tervals if comparisons with the best are required and HSD intervals for comparisons among all means, although ESD intervals were admitted to be very versatile and possibly useful in more cOlnplex situations. Split-block and split-split plot designs were also discussed by the authors. A SS appproach to the problell1 of selecting the best treatment (known and unknown variance) under a randoll1ised cOll1plete block design with one observation per cell was studied by Gupta and Huang, D.·Y. (19~O). Their optimal rule (see Table A.l) tests the hypothesis that all selected treatments are not significantly different and are 1110re than ~ larger than the non-selected treatments, with ~ prespecified. Ranking and selection procedures were applied by Tong (1985) to a problem in genetic selection, that of selecting the r best candidates associated with scores -~p(k-7'+lb .. ,Xp(k) based on observed values (Xl +YJ, .. ,Xk+Yk) where Xi, i = 1, .. , k are nonnally distributed with variance 7 2 and Yi, i = 1, .. , k are iid N(O, (J2 / N) and are independent of _~ (see Table A.1). Under the IZ fonnulation, Bechhofer (1968a) considered ranking ll1Ultiply clas­ sified variances under a two-way design with Tl observations per cell, and proposed four goals which involve the selection of the best levels of both factors, the simul­ taneous selection of the best levels of both factors, and the selection of the largest interaction. The latter goal was shown to reduce to that of Bechhofer and Sobel (1954). 2.2.6 Related Goals Unbiased procedures based on contrasts of sample 111eans for nonnal populations with equal but unknown variance were proposed by Seal (1955, 1957). Seal's (1955) rule is detailed in Table A.I. The two papers differed in their ll1ethods of deriving the optilnal choice of the critical values, which highlights a criticisll1 Inade by Chernoff and 'Yahav (1986) of arbitrarily chosen performance criteria, such as Inaximum PCS, minill1Ull1 probability of excluding t.he worst. populat.ion, and/or minilllum expect.ed subset size on which these rules are construct.ed and cOll1pared. Deely and Gupta CHAPTER 2. LITERATURE REVIEW 27 (1968) have compared Seal's (1955) and (1957) rules with respect to expected subset sIze. Gupta and Hsu (1978) modified Seal's (1955) selection rule to prevent the pos­ sibility of P < 1/2, with the result that 1ri is selected iff Xi ~ XR(k-l) and/or Xi ~ ~;:;XR(j)/(k - 1) - davg , where davg is a constant satisfying the P* require­ ment. This class of "average-type" procedures was compared with Gupta's (1956) "lnaximum-type" rules. Unlike the results of Seal (1955), Deely and Gupta (1968), Devernlan (1969) and Deverman and Gupta (1969), which all base comparisons of performance of procedures on the operating characteristic over a subspace of 0, Gupta and Hsu's rule was shown to be optimal over the entire parameter space with respect to some prior. Rizvi (1963, 1971) considered a slightly different problelll of selecting the t pop­ ulations with the largest values of Ai = IBil/a from k normal populations with common known variance under both SS and IZ formulations (see Table A.l). (The parameter A is useful, for example, in the comparison of electronic devices.) Under the IZ approach, for the case in which at = aia2, with a 2 and ai known, Rizvi's procedure is applicable if the sample sizes are chosen to 111ake the sample variances equal but, as with Bechhofer's (1954) procedure, this is not necessarily optimal. Re­ sults obtained for Bechhofer's rule in this case may be extended to Rizvi's method. Guttman (1961) and Guttman and Milton (1969) discussed a problem of select­ ing the smallest. normal nlean (with known and unknown variance) in terms of the coverage Ai = ~(a-e,) of a fixed interval A = (-oo,a) (a known) using the param- a, eters bi = (Bi - a)/ai. The goal is t.o select the population corresponding to bp(l). For the case of equal variances, their procedures reduce to Gupta's (1956) methods for selecting the slnallest normal mean. If the goal is to select the smallest variance with common unknown means, their procedure corresponds to that of Gupta and Sobel (1962). 2.2.7 Other Distributions Gupta and Panchapakesan's (1972) general SS theory was applied t.o convex mix­ tures, with part.icular attention to weights which reduce to Poisson probabilities CHAPTER 2. LITERATURE REVIEW 28 in a special case, and to negative binomial weights. The former may be used, for example, for non-central chi-squared and non-central F variables and the latter for the distribution of the multiple correlation coefficient R2. Gupta and Pancha­ pakesan (1969) and Gupta and Studden (1970) have also considered these cases with the usual function h( x) = ex, e ~ 1. Another choice of h( x) which has been used by Gupta and Huang, D.Y. (1975) in the selection of Poisson populations is h( x) = ex + d, e ~ 1, d ~ o. The choice of h( x) for a given class of functions remains, however, substantially subjective. A minimax method of selecting the gamma population with the largest (or smallest) scale parameter (known shape parameter) has been investigated by Ofosu (1972) under an IZ formulation. Particular results were given for Weibull scale paranleters and normal variances. Gupta (1963) approached this problem under a SS formulation, using a rule with properties similar to those discussed for the general scale parameter case. In the case of unequal sample sizes, a lower bound for the infimuln of the pes over the entire parameter space was derived by Gupta and Huang (1976). Also using an IZ fonnulation, Barr and Rizvi (1966) and Raghavachari and Starr (1970) proposed methods of selecting the t exponential populations with the largest parameters Ii (representing the guaranteed life of the population 7T"i), assuming the other parameter (} known. Barr and Rizvi (1966) developed methods for selecting the unifonn population with the largest range and generalised this to a certain class of nonregular distributions. The same problem was also considered by McDonald (1977) using a SS formulation, based on the range of the sample. For selection of location parameters, Hodges and Lehmann (1963) discussed a class of estimators, based on the rank test statistic, which were shown to be approx­ inlately normally distributed for large n. One-sample Hodges-Lehmann estimators were used by Gupta and Huang (1974) under a SS formulation for selecting the best t location parameters. Also using a SS approach, Gupta and Singh (1980) consid­ ered sample medians, which are a particular class of Hodges-Lehmann estinlators, for the selection of best normal and double-exponential populations. Arguments of sinlplicity, symmetry, unbiasedness and uniformly minimum variance unbiased were CHAPTER 2. LITERATURE REVIEW 29 given in favour of the sample median for non-normal distributions. Their procedure was shown to perform favourably compared with Gupta's (1956) means procedure for heavily contaminated distributions. Other authors who have used the sample median in selection procedures for non-normal distributions are Gupta and Leong (1979) for double exponential populations, and Lorenzen and McDonald (19 %1) and Gupta and Panchapakesan (1974) for logistic populations. The use in ranking and selection problems of Tukey's generalised lambda distri ­ bution GLD, suggested by Tukey (1960) and generalised by Ramberg and Schmeisser (1972,1974) to include both symmetric and asymmetric distributions, has been dis­ cussed by Gupta and Sohn (1985). The authors showed that the performance of selection procedures for a large number of parametric distributions, including the uniform, normal, logistic, laplace and t distributions, can be investigated by chang­ ing the values of the parameters of the GLD and that good approxinlate critical values for these distributions may be obtained using the GLD. Gupta and Sohn (1985) proposed selection rules under a SS formulation for selection of the largest location parameter from k synunetric G LDs, based on sample nledians (see Table A.l). The asymptotic relative efficiency of the rule cOlllpared with that of Gupta (1956) was shown to increase for heavier-tailed distributions. Ordering populations from a family of distributions characterised by a binary or­ der relation with respect to a known distribution was first considered by Barlow and Gupta (1969). A number of other authors have considered similar partial-ordering relations and have been reviewed by Gupta and Panchapakesan (1979, pp.317-331). Gupta and Panchapakesan have also discussed ranking and selection procedures for other distributions, such as Huang's (1974) rules for Pareto distributions. 2.3 Comparison with a Control 2.3.1 IZ Approach Investigation into the problem of conlparing populations with a control was ini­ tiated by Paulson (1952), using an IZ-type fOf1nulation. His goal, to select the CHAPTER 2. LITERATURE REVIEW 30 best population provided that it is better than a control, was considered for both normal means and binomial proportions of successes. The proposed rule for the normal means case for known variance is detailed in Table A.I. Paulson also de­ rived approximations to the critical value and to the minimum sample size. For the unknown variance case, multi-stage methods were proposed by Paulson (1962) using the same LFC and the usual pooled estimator 52 of the variance. Bechhofer and Turnbull (1978) generalised Paulson's (1952) results for the nor­ Inallneans case (with known and unknown variances) for a known specified standard (see Table A.l). Their approach reduces to Paulson's procedure for a particular (practical) choice of the three paralneters bo = 0, [,1 = bz and for this case the au­ thors presented appropriate tables. They also showed that for b1 = [,2 = fJ*, the combinations of sample size and critical value satisfying the probability require­ ments also guarantee with the sanle confidence that at least one of the t (1 ::; t < k) best populations will be selected for certain configurations of fl. (Call this guaran­ tee M, say.) For the unknown variance case, Bechhofer and Turnbull proposed a two-stage procedure. For nl0re general (absolutely continuous) distributions, Turnbull (1976) pro­ posed a sinlilar IZ-type rule for the same pro blenl (see Table A.l). Turnbull showed that for the location and scale paranleter cases, his procedure satisfies Bechhofer and Turnbull's guarantee M under certain configurations of fl. A generalisation of Paulson's (1952,1962) results for selecting all treatments bet­ ter than the control with confidence P; and identifying the best treatment, if more than one is selected, with confidence PI*, was investigated by Dunnett (1985). His method, unlike that of Bechhofer and Turnbull (1978), allows for an unknown con­ trol. For the case of nonnal means with known variance (see Table A.I) appropriate tables were developed for use in detennining the optinlal allocation of observations, but no dainls were Inade as to the optilnalit.y of this nlethod for unequal sample sizes. Dunnett (1985) acknowledged that the LFC used in his procedure may be overly conservative in sonle cases and derived a generalisation of the expression for the PCS of the best population under the assumption that the LFC is of the form t = 0, i=l, .. ,k-m for some m, 1 ::; m ::; k. A conlputer &.,( .. ) .... . = &( .. -~ l=k-m+l, •• ,k 'P ) J CHAPTER 2. LITERATURE REVIEW 31 program was supplied by the author to evaluate this expreSSIon (and hence the more conservative expression in Table A.l). Dunnett also considered this problem for binomial populations. Again using an IZ approach, Bristol and Desu (1985) considered a slightly dif­ ferent problem of selecting the largest guarantee time "y only if it larger than a known standard and otherwise selecting the standard, with "y an unknown pa­ rameter from the two-parameter exponential distribution with common standard deviation B. Bristol and Desu's single stage procedure for known B is detailed in Table A.l, with critical values for this rule tabulated by the authors. Bechhofer and Turnbull's (1978) guarantee M was shown to hold for their method. For the case of unknown B, a two-stage procedure similar to that proposed by Bechhofer and Turnbull (1978) and Desu, Narula and Villareal (1977) was discussed by Bristol and Desu. They suggested considering only positive values of d~ and then replacing d~ by d~, hence reducing the number of constants and changing the non-selection zone. In practice, however, one may wish to consider a negative value of d~ for the purpose of selecting populations "not too nluch worse" than the standard. There is also the problem of the choice of d~, which can often be set to zero but is still a subjective decision. (These problems are considered in Chapter 6 and are avoided by using a CB formulation.) U sing an IZ-type approach similar to that of Bristol and Desu (1985), Bristol, Mithongtae and Chen (1987) considered selection of the best exponential population (in tenns of guaranteed life spans) provided it is bet.ter than an unknown cont.rol (see Table A.l). Tables of nand d satisfying the probability requirements were also provided. A two-stage rule was specified for unknown scale parameters. Paulson (1952) and Bechhofer and Turnbull (1978) assumed independence be­ tween all treatnlent.s, including the control. Leu (198~) adapt.ed t.heir goals t.o the case in which all of t.he k~ experilnent.al treattnents have equal variances and covari­ anCes, but. the control populat.ion Inay have a different variance, and the covariance bet.ween a t.reattnent and the cont.rol may differ from the covariance between t.wo t.reatments. For the known control case, Leu's procedure reduces to Bechhofer and Turnbull's (1978) if independence bet.ween treattnents is assulned. His results for CHAPTER 2. LITERATURE REVIEW 32 both known and unknown controls are given in Table A.I. For the case in which the control population has the same variance as the treatments and 0: = j3 (see Table A.l), Leu's rule was shown to be easier to apply than Paulson's (1952) (in which a = f3 = 0). A two-stage procedure, which reduces to Bechhofer and Turn­ bull's (1978) rule under independence, was proposed by Leu (1988) for the case of unknown variance and unknown control. Lehmann (1961) proposed a different approach to the treatment-control problem by defining both "good'~ (Bi ~ Bo + .6.) and "bad" (Bi ~ Bo) populations with respect to the known or unknown control, with .6. a specified constant (see Table A.l). Leh1nann derived a 1nininlax procedure for certain fa1llilies of distributions, in which the expected nUlnber (or proportion) of bad populations in the selected subset is lllinimised subject to the require111ent that the infinlum of the pes be at. least. P*. The infimum was taken over a subset. of the paranleter space in which at least one of the populations is good. A number of definitions of "correct selection" was considered by Lehmann: selection of an expected number (or proportion) of good populations, selection of at least one good population, s~lection of the best population provided that it is good, and selection of all good populations. For the normal means case with common unknown variance, Lehmann showed that a minimax solution was not obtained but proposed a solution that is approximately minimax under the assumption that all parameters B are positive. Tong (1969) investigated a similar goal of si1llultaneously selecting all those populations which are "sufficiently better" and rejecting all those which are "suf­ ficiently worse" than an unknown control. His procedure for normal populations with C01111110n known variance is described in Table A.I. Randles and Hollander (1971) showed that Tong's assumption that the ith comparison depends only on Xi and -Y-o is necessary for the minimax property derived by Tong. For the unknown variance case, Tong constructed two-stage and sequential procedures. Selection of those and only those populations better than a fixed known or unknown standard was studied by Schafer (1977), using a different selection criterion fr01n tha.t proposed by Leh111ann (1961) and a pivotal function with a chi-squared distribut.ion. His results are detailed for the case of normal 11leans and known CHAPTER 2. LITERATURE REVIEW 33 standard in Table A.I. The unknown standard case was treated similarly, but Schafer was unable to find an analytic expression for the value of t at which the minimum PCS occurs for either case. Other useful pivotal functions were also considered. For the problem of comparing normal populations (with known variance) with two controls, Seeger (1972) followed Tong's (1969) approach and defined a correct decision based on five disjoint and exhaustive subsets of the parameter space. His results are detailed in Table A.I. Cane (1979) showed, however, that Seeger's expression for the mininlum PCS is incorrect. 2.3.2 SS Approach Conlparison with a control was also considered by Dunnett (1955), who used a SS approach to choose a (possibly enlpty) subset of all normal populations (coillmon unknown variance) with nleans larger than the Illean of the unknown control, with confidence at least P* over the entire paranleter space. Dunnett derived upper, lower (see Table A.l) and two-sided joint confidence bounds for ()i - ()o,i = 1, .. ,k, based on the Illeans of ni observations froln 7ri, i = 0,1, .. , k. Critical values for the case ni = n, 'i = 1, .. , k, so that the COlllInOn correlation coefficient is given by p = n/(no + n), were tabulated by Dunnett (1955) and have also been tabulated by Dunnett and Sobel (1954), Krishnaiah and Armitage (1966), Gupta, Panchapakesan and Sohn (1985) and others, for various cOlnbinations of k and p. Approxinlate critical values for Dunnett's two-sided confidence bounds were presented by Dunnett (1955), with exact values tabulated in a later paper (Dunnett, 1964). An alternative SS solution to the treahnent-control probleln was investigated by Gupta and Sobel (1958),whose results for the nOnnalIlleanS case, with both known and unknown control and known and unknown variances, are presented in Table A.I. Gupta and Sobel also considered ganlllla populations and binomial proportions of success. Chen (1980) also discussed selection of a subset of nonnal 11leanS better than a known or unknown control in the event of known or unknown possibly unequal variances (see Table A.l). CHAPTER 2. LITERATURE REVIEW 34 Berger (1981) considered a SS approach, closely related to that taken by Chen (1980), to selection of all normal means larger than a control. Berger discussed a wider range of covariance structures than Chen, with identical methods for coin­ ciding structures. All procedures have the form Xi 2:: Xo - cSE(Xo - Xd, with c an appropriate constant and SE(Xo - Xd the standard deviation of the difference between Xo and Xi or an appropriate estimate. Berger and Gupta (1980) have investigated the advantages of using the standard deviation of this difference in a different selection probleln. By expressing the critical value as a function of the correlation structure, Berger reduced the dimension of the expressions for the PCS and hence enabled existing tables to be used for all of the covariance structures considered. The author asserted that this makes Chen's (1980) results easier to use and also allows a wider range of models to be considered. Berger's (1981) results for various conlbinations of known and unknown correlation and known and unknown variance are included in Table A.l. Sequentially rejective rules have been discussed by Brostrom (1981 ) for multiple comparisons with a control and multiple comparisons with the best for the normal means case. Brostrom's results are also discussed in Chapter 6. Unequal variances and sample sizes The optimal allocat.ion problelTI for com­ parison with a control was posed but. not really solved by Dunnett (1955) and was later investigated by Bhattacharya (1956, 1958), Bechhofer (1969) and Bechhofer and Nocturne (1972). Bhattacharya's nlethods differed in the loss functions consid­ ered and the corresponding consistency of the rules. Results obtained by Bechhofer (1969) are detailed in Table A.l. Bechhofer and Noct.urne (1972) constructed op­ tin1.al allocation results similar to Bechhofer's (1969) for the two-sided comparisons proposed by Dunnett (1955). For the case in which the known variance is possibly unequal to the known vari­ ance of the experinlental treatinents, Sobel and Tong (1971) proposed a slightly different fornlulation t.o that. of Bhattacharya (1956,1958). Using Tong's (1969) method of partitioning populations into randolll subset.s, these authors derived globally optinlal comparisons satisfying two rules. Their first rule nlinimises the CHAPTER 2. LITERATURE REVIE1V 35 expected number of misclassified populations and their second rule maximises the PCS. An asymptotically optimal sequential sampling rule was suggested for the unknown variance case. Bechhofer and Turnbull (1978) generalised Bechhofer's (1969) results and de­ rived globally optimal confidence statements for the case in which the known vari­ ances of the treatments are possibly unequal (see Table A.l). Favourable compar­ isons of their rule with three alternative allocation rules were also made, but it was noted that the savings depend critically on the variation in sample sizes and on the value of P*. 2.3.3 CB Approach Edwards and Hsu (1983) demonstrat.ed that. Hsu's (1981) constrained intervals for multiple comparisons with the best. lllay be adapted from intervals for multiple comparisons with a control, if t.he latter can be computed regarding any population as the control (see Table A.l). Unlike Dunnet.t's (1955) rule for multiple compar­ isons with a control, Edwards and Hsu made no assumptions about independence between treattnents. The new intervals were compared with Tukey's intervals, the former being shown by silllulation lllethods to be superior in all but a few unusual configurations, and by nUlllericallllethods to be aSYlnptotically superior since they are always shorter in length and shorten further when the best treatment is obvious. Their aSYlnptotic efficiency against a wide class of conjectured competitors was also established. The adaptive quality in Edwards and Hsu's (1983) int.ervals, which adjusts for bias by the size the int.erval, was discussed by the authors at length, with the Inidpoint of the confidence interval advocated as a less positively biased estinlator of Bp(k) than the "naive" estiluator .XR(k) . For k = 2, this estimator re­ duces to the hybrid estiulator studied by Blulllenthal and Cohen (1968), which was shown to perform well for certain choices of the crit.ical value d. Edwards and Hsu (1983) also gave non-parallletric intervals based on ranks, which are a generalisation of Hsu's (1981) non-parauletric results. Using a CB formulation,5tefansson,Kim and Hsu (1988) considered a rule for selecting all populations (assullled t.o have stochast.ically increasing distributions) CHAPTER 2. LITERATURE REVIElV 36 better than a control, with confidence P*, using a non-negative distance function {)(()i,()j). Their results are displayed in Table A.I. CB approaches for comparison with a control in the event of unknown and unequal variances have been investigated by such authors as Dudewicz and Dalal (1975), Dudewicz, Ramberg and Chen (1975), Wilcox (1979,1984,1985), Bofinger and Lewis (1987) and Taneja and Dudewicz (1987). All of these authors proposed multistage methods. 2.3.4 Other Considerations For a general class of distributions, Roberts (1964) considered a procedure, which is aSYlllPtotically optimal under a fixed subset size condition, for selecting the best only if it is better than a control (see Table A.l). Gupta and Leu (1986) have proposed isotonic procedures for selecting populations better than a standard for two-paranleter exponential distributions. Minimax multiple t-tests were developed by Gupta and Meiscke (1985) for comparison of k nornlal populations with a control. Gupta and Hsiao (1981) have used a number of different formulations for selecting populations close to a control. Bechhofer and Turnbull (1977) suggested two sampling plans for selection of the fraction qp(k) (of non-defective items produced by an industrial process) only if it larger than a specified standard q~. Their method is similar to that previously discussed for normal means (Bechhofer and Turnbull (1978)). A review of authors who have considered the problem of drawing inferences on a vector of treatment-control contrasts was present.ed by Bechhofer and Tamhane (1981), who thenlselves investigated balanced treatment-control incomplete block designs. Using a different approach~ Hedayat. and Majumdar (1985) derived fami­ lies of A-optimal block designs treatment-control comparisons (which minimise the sum of the variances of the best linear unbiased estimate of the treatment-control contrasts ). The problem of conlparing multiply classified variances from normal populations with a (normal) control for multifactor experinlents has been discussed by Bechhofer (1968b), using a lllultiplicative lllodel introduced in a companion paper (Bechhofer, CHAPTER 2. LITERATURE REVIEW 37 1968a) which was discussed earlier in this review. Two-sided comparisons were also considered by the author with tables for selected combinations of the parameters. 2.4 Robustness of Ranking and Selection Rules Some particular robustness results obtained by a number of authors for particular procedures have already been mentioned in this review. The poor performance of nonparanletric techniques relative to their parametric counterparts (discussed, for example, by Dudewicz, 1966), as well as the comparative wealth of results obtained for well-known distributions, makes the question of robustness an important one. The robustness of luultiple comparison techniques has been investigated by such authors as Carmer and Swanson (1973), Tamhane (1979) and Dunnett (1980a, 1980b, 1982). Carmer and Swanson showed that the LSD method was preferable to the methods of Scheffe, Tukey and SNK under a fixed probability requirement and was also easier to use. Tamhane (1979) and Dunnett (1980b) used slightly different conditions of variance heterogeneity to compare a number of multiple comparison procedures by Monte Carlo methods. Dunnett (1980a) also used Monte Carlo ex­ periments to compare multiple comparisons rules under unequal sample sizes. Dunnett (1982) conlpared a means procedure and a k-sample rank sum test with several robust estimators of location for pairwise multiple comparisons, with respect to stability of experimentwise error rates under a variety of non-nornlal distribu­ tions (robustness of validity) and average confidence interval lengths (robustness of efficiency). As expected, the rank-sulu Dlethod outperformed the means procedure for non-norInal distributions but. was itself out.perforIued by robust. luethods for extrelne-tailed dist.ributions. Using Monte Carlo methods, Zaher and Heiny (1984) also cOll1pared selection procedures based on lueans (IV1), medians (D) and rank­ SUlllS (S), for selecting the best normal population under heterogeneity of variance. The order of superiority was shown t.o be M, D and S for variance increasing with the Inean, while for variance decreasing with the mean, S was generally superior to M and D was quite inefficient. Robust. ranking and selection procedures have been considered by Gupta and CHAPTER 2. LITERATURE REVIEW 38 Huang, D.Y. (1980) from the point of view of most economical multiple decison rules. Robust and nonparametric procedures based on joint ranks have been shown by Rizvi and Woodworth (1970) to be unsatisfactory in that the PCS over the entire parameter space cannot be controlled. Hsu (1982) constructed, instead, subset selection procedures based on pairwise rather than joint rankings of the samples, for which the PCS is controlled over O. Dudewicz and Mishra (1985) were the first authors to directly investigate the ro­ bustness of a ranking and selection procedure. These authors referred to Dudewicz's (1966) favourable efficiency cOlnparisons between paralnetric and non-parametric formulations and hence considered the robustness of Bechhofer's (1954) selection rule to non-nonnality. They used a finite series expansion of the PCS to calcu­ lat.e exact values of the PCS for the unifornl distribution (short-tailed) and Monte Carlo approximations to the PCS for the Student's t distribution on three de­ grees of freedom (long-tailed), for k = 2,3,6, n = 1, .. ,10 and preference zones ()p(k) - ()p(k-l) 2: b* = -1.0,1.0,3.0. By comparing these values with the exact PCS values under the normal distribution, Dudewicz and Mishra proclaiuled Bechhofer's rule to be extremely robust to both long- and short-tailed distributions over a broad range of parameter values and hence recommended its use for a wide class of dis­ tributions. It would be interesting to generalise these results for a broader range of distributions, including asymmetric and discrete distributions, and for a wider selection of parameter combinations. Chapter 3 AN APPLICATION OF RANKING AND SELECTION 3.1 Introduction One field to which ranking and selection theory has not been largely applied to dat.e is that. of personnel selection. A typical employment. problenl is t.o select "best" candidates for a particular position or to compare all candidates, based on the opinions of a number of judges. Often, some sort of "confidence" of correct selection or conlparison would be desirable in such selections and comparisons. The many current personnel selection Inethods based on qualitative infornlation cannot provide such selection guarantees. It seenlS, then, that the required goals are those currently pursued in ranking and selection theory and, if information about candidates can be expressed in some way as quantitative "scores", useful selection procedures may be developed which satisfy the specified confidence requirements. This application has been considered by 11engersen and Bofinger (1987), whose approach is outlined in this chapter. The purpose of this is twofold: firstly, it indicates a novel direction for the application of ranking and selection theory and secondly, this application is used in subsequent chapters to illustrate the proposed goals and procedures. The problem of combining infornlation about candidates has been considered 39 CHAPTER 3. AN APPLICATION 40 by a number of authors. Genest and Zidek (198b) have reviewed a wide range of such methods, including the use of odds ratios, belief functions and Bayesian for­ mulations. These authors dealt only in passing with group decision making and did not admit preference-based opinions. Non parametric selection procedures, based on a simple quantification rule involving the sum of vector ranks, have been con­ sidered by Dudewicz and Lee (1978) and Lee and Dudewicz (1980,1981,1982) for the k-treatment n-block problem in general and the candidate selection problem in particular. These authors required that each judge give an overall relative rank for each candidate. with the option of not ranking if a particular pairwise ordering can­ not be nlade. Bechhofer and Sobel (1954-) also considered the candidate selection probleIll and proposed a rule which selects as best the candidate who is most fre­ quently ranked as best by the judges. Lee and Dudewicz (1985) have deIllonstrated that the latter procedure is less efficient aSYlllPtotically than Friedman-type rank vector methods. Both theory and practice tend to support the view that a quantitative rule is effective in cOInbining infonnation about different attributes of interest possessed by a candidate and opinions fronl different judges. It is unclear, however, that an "overall" rank or score given by each judge for each candidate is adequate. The nlost skilful and ilnpartial judge Illay find it difficult to ensure (and be seen to ensure) that all relevant attributes are adequately considered in an overall ranking. It is also difficult. to ensure that each judge has the saIne overall view of what is required for the particular job and, consequently, the desired person. Mengersen and Bofinger (1987) have proposed a personnel selection methodology which eInploys a very simple scoring rule to cOInbine inforn'lation about candidates from a number of judges and then applies traditional ranking and selection pro­ cedures based on the resulting scores. A computer package, PERSEL, has been developed which assists in the scoring process and perfonns the selection proce­ dures. The scoring rule, described below, is not asserted to be optilnal in any sense, but its simplicity and flexibility is appealing to a wide class of users. The proposed nlethodology is also appealing because it necessitates a very thorough definition of the job and of the attributes required of the successful candidate. This in itself CHAPTER 3. AN APPLICATION 41 results in more efficient selection of personnel. 3.2 The PERSEL Methodology Consider the case in which n judges must decide between k candidates, based on a attributes of interest. It is assumed that the candidates Cu may indeed be ordered by their true (unknown) "scores" (}u, u = 1, .. , k, which are some function of the a attribut.es and which must be estimated in some way. The methodology proposed by Mengersen and Bofinger for this personnel selec­ tion problem is outlined below. 1. A set of well-defined attributes required in the successful candidate is de­ veloped. This set may contain both "essential" and "desirable" attributes. More precisely, an essential attribute is taken to be an attribute to which a minimum level of perfonnance is attached. If a candidate does not attain the specified minimum level for this attribute, she is excluded from further consideration. (For example, an essential attribute may be "typing", for which a minimum level of thirty words per minute is required.) Desirable attributes do not have such levels attached to them and are not used to eliminate candidates prior to scoring, but in the scoring process a higher score is preferable for such attributes. It is necessary that each of these attributes is well defined so that judges may attach meaningful scores to them. 2. Each of the desirable attributes A w , 11) = 1, .. , a, say, is weighted according to its relative importance. Mengersen and Bofinger suggest an implicit weighting method, in which a "lnaxinlum score" Fw is at.tached to each attribute A w , with a larger value of Fw indicating a more important attribute. Each judge will then be required to award, to each candidate, a score ranging fronl zero to Fw for the attribute Au,. The maxilnum scores Fw lllUSt be specified prior to scoring. If p individuals (I;;, z = 1, .. ,p, say) are involved in specifying these weights and agree­ ment cannot be reached by discussion, a possible option is for each person 1z to individually propose a set (FzI, Fz2 , •• , Fza) and to take as the final maximum score Fw = p-l ~=l Fz'Un W = 1, .. , a. (In practice, it may be nl0re convenient to use CHAPTER 3. AN APPLICATION 42 the whole number nearest Fw.) In many cases, it may be the judges themselves who develop these maximum scores (so that p = n) prior to any actual scoring of candidates. 3. Each judge Jv , v = 1, .. , n, awards a score Y~vw, from zero to Fw , to each can­ didate Cu, u = 1, .. , k for each attribute A w , w = 1, .. , a. Notice that no discussion between judges should take place while candidates are being awarded scores. The final score given by the vth judge for the uth candidate is simply a Yuv = LYuvw ,u=l, .. ,k;v=l, .. ,n. w=l Because of the implicit weighting method employed (described in Step 2), the scores }Tuv appropriately account for the relative importance of each of the a attributes of interest. If a is not too small; if the Inaxill1um scores for each attribute are of reasonable relative size and if the range of scores (0 to Fw) for each attribute Au" w = 1, .. , a, is well covered by the judges, these scores Yuv~ 11 = 1, .. , k, v = 1, .. , n are assumed to be normally distributed, with Illean Bu and variance a~. It is also assumed that the final scores Yuv are independent. Mengersen and Bofinger stress that the judges consider each candidate on her own merits, with­ out comparison with other candidates. (There is no compulsion to give unique scores for candidates.) This, combined with the emphasis on no interaction be­ tween judges during scoring, assist in reducing dependence in the resulting scores, although independence may still be a strong assumption in this situation. This is further discussed in Chapter 7. 4. The overall score for candidate Cu, 11, = 1, .. , k, is the average of all judges' final scores for the candidate and is given by n Bu = Xu = n- 1 LYuv v=l (This aSSUllles that each judge's opinion holds equal weight.) The (unknown) variances a~, u = 1, ""' k, are assumed to be comInon and may be estimated as the residualIllean square of the Yuv values in an analysis of variance, after accounting for possible "judge" and "candidate" effects, with corresponding degrees of freedoll1 v. CHAPTER 3. AN APPLICATION 43 The scoring method described above may be applied to a wide range of person­ nel selection problems. It ensures that each attribute is appropriately accounted for in the final selection and that each judge has an equivalent input into the se­ lection process. The aggregation method may be modified if necessary to include a weighting function to reflect differences in the importance of particular judges' scores (overall or with respect to particular attributes). The specification of attributes is also flexible. Combinations of attributes may be specified and scored as separate attributes, in order to "reward" candidates who possess these desirable combinations. External tests, such as typing or psychological assessnlents, may be easily incorporated. Although thorough discussion between judges regarding attributes is actively encouraged prior to any actual scoring, it is stressed that each judge must award scores to candidates independently of the other judges and the other candidates. As well as contributing to independence in the final scores, this aspect. of the methodol­ ogy is particularly appealing if the judges and/or candidates are separated by time or distance. It is noticed that the specification of a large number of "essential" attributes nlay result in a large nunlber of candidates being initially excluded. Mengersen and Bofinger suggest that, instead, a small number of essential attributes (if any) be specified and that a set of "minimum scores" Mw be attached to the more important of the renlaining "desirable" attributes. For these attributes Aw , the values n }/uw = n-1 LYuvw v=l are calculated for each candidate Cu' If any Y uw 15 less t.han the corresponding value ]\([w, the candidate is then excluded from further consideration. Agreement on these Inininlunl scores nlust be made before any scoring takes place. ( Notice that these minimum scores are different fronl the lnininllun levels required for essential attributes. The latter are preset and are designed to exclude obviously unsatisfac­ tory candidates before any scoring takes place. The nlininlunl scores for desirable attributes are also preset, but candidates are only excluded if the judges' cOlnbined scores for these attributes are low.) CHAPTER 3. AN APPLICATION 44 As stated previously, the scoring method described above is not asserted to be "best" in any sense, but it is simple and flexible. It may be appropriate, once quantitative lnethods are established in the personnel selection field, to introduce other scoring rules, such as some of those reviewed in Genest and Zidek (1986.), or to consider the problem in a multivariate framework. The introduction of a quantitative scoring rule, followed by objective ranking and selection procedures, based on the scores, which give some statistical confidence of correct selection or cOlnparison, benefits both employers and enlployees. It leads to (and can be seen to lead to) nl0re equitable comparison froln the candidates' viewpoint and can be useful in giving candidates reasons for non-selection. It also results in n10re effective selection for the employers, sin~e the ideal person is rig­ orously specified beforehand and the viewpoints of all judges are combined in an objective way. In cases in which candidates are sought for similar positions over a number of tilne periods (such as for entry into college or particular grades in a large organisation), the lnethod may be used iteratively to improve selection. By evalu­ ating the perfonnance of present successful applicants, the relevant attributes, the weights attached to thenl or the nUlnber of judges eluployed in the scoring process luay be 1110dified. 3.3 Ranking and selection procedures Based Oll the candidates' overall scores Xu, 1l. = 1, ." k, goals that may be useful in personnel selection include the following: (i) Conlparison and selection of the t best candidates, 1 ::; t < k; (ii) COIl1parison and selection of the best candidate, providing she is better than a control; (iii) COlnplete ranking of all k candidates. Any procedure proposed for one of the above goals must be accoll1panied by a prespecified confidence of correct selection or comparison. The first goal is discussed CHAPTER 3. AN APPLICATION 45 in Chapters 4 and 5, and the second goal in Chapter 6. A procedure proposed by Bofinger (1985) may be adopted for the third goal and was used in the software package PERSEL, described below. This approach was discussed in Chapter 2 and some slight improvelnents are discussed in Chapter 4. 3.4 PERSEL - A software package for personnel selection Although the scoring Inethodology and selection and ranking procedures described above lIlay be applied using existing tables, it is useful to present to potential users of these 111ethods a COlllputer software package which guides users through the scoring process and which executes the various selection and cOlnparison procedures. Such a package, PERSEL, has been developed by Mengersen and Bofinger (1987) and is uniquely tailored to the personnel selection proble111 (although the subroutines incorporated in the package are capable of adaptation to Inore general purposes). A SU111mary of the package is given by Mengersen (1987) and a more detailed de­ scription of the practical use of the package Inay be found in the PERSEL manual. The general features of PERSEL are described below and an overview of the structure of the package is presented. The cOlnputational methods for obtaining the critical values required in the selection and conlparison routines are described in Chapters 4, 5 and 6 of this thesis. 3.4.1 Overview of PERSEL As lllentioned above~ PERSEL is designed to be used for personnel selection, so all exanlples aud output are in ternlS of selection of short lists of candidates, ranking of candidates, nUluber of judges and so ou. The package is divided into two lllain sections. The first deals with the scoring Inethodology to be used by judges iu the evaluation of candidates. The second section uses these scores to carry out the required selection and ranking procedures. CHAPTER 3. AN APPLICATION 46 Section 1: The scoring methodology The scoring rule used in PERSEL is that described above. The first section of PERSEL is designed to do the following: 1. HELP: Give an overview of the scoring methodology if required. 2. SCORING SHEETS: Sheets are printed out, which may be used by those involved in determining the (prespecified) Inaximum and minimum scores at ­ tached to attributes Aw, w = 1, .. , a and by the judges involved in the actual scoring of candidates. These sheets include - an "attribute score sheet", to be used by each individual involved in determining the values Fw and Mw. Each individuallz , z = 1, .. ,p records her personalluaximum and minimum scores, Fzw and M zw , respectively, for each attribute Aw, w = 1, .. , a. - a "final attribute sheet", which PERSEL computes from the "attribute score sheets" and which records average nlaxinlum and minimum scores Fw and ll,fw for each attribute. These are the values to be used in subse­ quent scoring of candidates. - a "judge score sheet" to be used by each judge, on which are recorded the scores Yuvw for the wth attribute by the vth judge for the uth candidate, u = 1, .. ,k,v = 1, .. ,n,w = 1, .. ,a. - a "summary score sheet", which PERSEL computes from the "judge score sheets" and which records final judge scores Yvw = 2::=1 Yuvw for each judge Jv for each candidat.e Cu. - a "minimum score sheet", which PERSEL computes from the judge score sheets and which records the values }~w = 2::=1 l~vw, for each attribute Aw for each candidate eu ' The average of these values is also displayed and may be compared with the prespecified minimum scores Mw for appropriate attributes. Candidates who obtain a score Yuw smaller than .l\.fw for any attribute Aw should not be considered further. CHAPTER 3. AN APPLICATION 47 3. ENTERING SCORES: Candidate names and scores may be entered in one of three ways: - Final judge scores y~v = 2:~=1 Yuvw given by the vth judge for the uth candidate, u = 1, .. , k, v = 1, .. , n, 'W = 1, .. , a, may be entered in an external data file. - Final candidate scores Xu = n-1 2::=1 X uv , being the average of the final judge scores for the uth candidate, may also be entered in an external data file, with the estimated variance S2 and the associated degrees of freedo111 v. - Values of Xu, 11. = 1, .. , k, S2 and v may be entered interactively and is stored by PERSEL in an external file. The required structure of the external data files is described in the HELP facility of PERSEL and also in the manual. Notice that if the score sheets described above have been used, the data file required for Option 1 will already exist. If Option 1 is adopted, PERSEL calculates the values of Xu, u = 1, .. , k, S2 and v. The value of S 2 is taken to be the residual mean square in an analysis of variance after accounting for "candidate" and "judge" effects. These values are are written to a file similar to that described for Option 2, and are used in the subsequent selection and ranking procedures of PERSEL. Section 2: Selection and ranking procedures The following options are displayed in the Inain lllenu of PERSEL: PLEASE TYPE: o To exit PERSEL. 1 For general information on PERSEL. 2 To generate scoring sheets. 3 To calculate the size of a selection committee. CHAPTER 3. AN APPLICATION 48 4 To select a fixed size short list. 5 To select a short list containing the t best candidates. 6 To rank all candidates. 7 To compare candidates with the true best candidate. The following aetion is taken after one of the above options is ehosen. o Close all files and exit PERSEL. 1 Display general help routine. Display main menu. 2 Generat.e seoring sheets as deseribed in Section 1 above. Return to maIn menu. 3 Display option of HELP or CONTINUE. If CONTINUE is chosen, request the following: nUlnber of candidates (k), number of candidates to be included in short list (t), desired guarantee P* that the best candidate is included in the short list, and "indifference value" 8/(1' (Bechhofer's usual indifference value expressed in t.erms of t.he standard deviation). Out.put is the value of n such that. p~~) (d) = P*, that is, the minimuln number of judges (n) required such that. the t candidat.es with largest. observed scores may be selected for the short list, with probability P* that at. least one of the selected candidates has a true score within 8/(1' of the true best seore. An option is then given of repeating with different P* and 8/(1' values, or returning to main nlenu. 4 Display option of HELP or CONTINUE. If CONTINUE is chosen, request t.he following: number of candidat.es (J..'), nUlnber of eandidates to be included in the short list (t), nUlnber of judges (n), "indifference value" (8/ (1'), method of ent.ering data and data input file. Output includes the t candidates with t.he largest observed scores to be included on the short list and the confidence P* that at least one of the selected candidates has a true score within 8/(1' CHAPTER 3. AN APPLICATION 49 of the true best score. The expression for P* has been shown by Bofinger (pers.comm.) to be given by The following options are then displayed: (i) repeat with different 8/ (T value; (ii) rank all selected candidates (passes to option 6); (iii) COlllpare selected candidates with the best (passes to option 7); (iv) return to lllain menu. 5 Display option of HELP or CONTINUE. If CONTINUE is chosen, request the following: nUlllber of candidates (h~), nUlllber of "best" candidates to be selected (t) ~ nUlnber of judges (11) ~ desired guarantee P* that the t best candidates are indeed included on the short list, Inethod of entering data and data file naille. (Help is also available on data input.) Output includes those candidates to be included on the short list. This is a direct application of the procedure P2 described in Chapter 5, for selecting a subset containing all t good norlllal populations with COllUllon unknown variance (location parameter case). Options as described in (4) above are then displayed. 6 Display option of HELP or CONTINUE. If passing directly frolll the main menu and CONTINUE is chosen, the following are requested: number of candidates (k), number of judges (n), desired guarantee P* that no candidate is declared to be "better" than another if the latter is in fact "better" than the former, nlethod of entering data and data file nallle. If the request is Inade from option 4 or 5, only P* is requested and only those included in the selected subset are ranked. Output includes a display of candidates ordered by their observed scores~ with "lines" connecting candidates who cannot be differentiated at this confidence level. (A candidate who is not connected by a line to a candidate with a lower observed score may be asserted to have a larger true score than the latter with silllultaneous confidence P*.) The ranking procedure eillployed is that developed b~r Bofinger (1985) and was discussed in Chapter 2. Options then include repeating with different P* CHAPTER 3. AN APPLICATION 50 value, comparing these candidates with the best (Option 7) or returning to maIn menu. 7 Display option of HELP or CONTINUE. If passing directly from the main menu and CONTINUE is chosen, the following are requested: number of candidates (k), number of judges (n), desired guarantee P* of correct com­ parisons, method of entering data and data input. file. If this option is called from options 4 or 5, only P* is requested and cOlllparisons are only made for t.hose included in the selected subset. Output includes, for each candidate, upper and lower bounds on the difference between her true score and that of the (unknown) best candidate, with simultaneous confidence P*. The bounds are those derived by Hsu (1984) and were detailed in Chapter 2. Options in­ clude repeating with a different. P* value, ranking all candidates or returning to main menu. The following section describes a particular example of personnel selection, which will be elnployed in the following chapters t.o illustrate the particular goals and procedures. 3.5 Example At a (hypothesised) university, it is an annual tradition for first year science stu­ dents to award prizes to two lect.urers who, in their opinion, have contributed most to t,heir education in specific ways. In past. years, the decision regarding the suc­ cessful lect.urers was made at a general 11leet.ing of all interested st.udents. Aft.er lnuch debat.e by the most vocal students on a wide range of relevant and irrelevant issues, a count of hands indicating favourable "overall opinions" was 11lade. Many students, however, complained that they did not have equal opportunity to air their views before the vote was taken and that often irrelevant issues overshadowed more illlportant attributes possessed by the lecturers. CHAPTER 3. AN APPLICATION 51 For this reason, the students decided to approach the matter of selecting the best lecturers in a more scientific manner (since that indeed was their discipline). They decided to adopt the methodology described above. The "desirable" attributes agreed upon are presented in Table 3.1, with corre­ sponding maximum and minimum scores. (No "essential" attributes were specified.) The eight lecturers to be assessed were: David, Carole, Debbie, Wayne, Wendy, Chris, Russell, Janice. Twenty-one students who attended classes taken by each of the eight lecturers were involved in the scoring process, using the twelve attributes listed in Table 3.l. A score Y~LVW was given by each judge J v to each candidat.e Cu for each attribute AU' ,u = 1, .. , 8, v = 1, .. , 21, 'til = 1, .. , 12. The total scores }Tuv = L~=l l~v1L' are given in Table 3.2. The average candidate scores Xu = (21 )-1 L~~l Yuv, U = 1, .. ,8 were calculated to be: David 145.7 Carole Debbie Wayne 134.7 125.2 118.7 Wendy 111.5 Chris Russell Janice 103.7 89.5 70.2 The estimated variance, taken to be the residual mean square in an analysis of variance after accounting for student and lecturer (judge and candidate) effects, was calculated to be 8 2 = 107.04 on 140 degrees of freedolll. After calculating average attribute scores Y·uw = (21)-1~~~1}/~vw''til = 1,2, it was revealed that David and Chris failed to attain one of the necessary minimum scores prespecified for Attributes 1 and 2. (David obtained an average attribute score of 4.8 for Attribute 2 and Chris obtained an average attribute score of 3.9 for Attribute 1.) This was surprising in the case of David, since he had obtained the largest average candidate score Xu, but showed that although David excelled in most areas, he failed to perfonn satisfactorily in one of the attributes perceived to be most irnportant. These t.wo lecturers were thus eliminated from further consideration. In the next three chapters, it is assu1ned that the scores Yuv are independent and nonnally distributed, with common (unknown) variance (J"2. The validity of these assulllptions has been discussed in general ternlS previously in this chapter and will be investigated for this particular example in Chapter 7. CHAPTER 3. AN APPLICATION 52 Table 3.1: Attributes and Scores for Assessing Lecturers Attribute Max. Score Min. Score Au; Fw 11lw 1. Individual interest in st.udents 40 5 2. A bilit.y t.o mot.ivat.e st.udent.s 30 5 3. Availability for help out.side class tinles 20 4. \Vell organised lectures 20 5. Present.ation of lect.ures 20 6. Marking: int.erest. & adequacy of comments 20 7. Relevance of tutes/pracs to lectures 10 8. Well organised tut.orials, practicals 10 9. Adequate preparation for tutes/pracs 10 10. Adequate preparation for examinations 10 11. Involvenlent. in general student. events 5 12. Involvelnent in student social events 5 CHAPTER 3. AN APPLICATION 53 Table 3.2: Scores Yuv for all Lecturers Judge Candidates No. David Carole Debbie Wayne Wendy Chris Russell Janice 1 146 111 116 115 118 99 93 71 2 149 127 123 97 121 98 79 63 3 141 152 117 137 110 100 79 64 4 147 143 127 124 109 96 85 73 5 149 132 120 106 117 91 101 63 6 135 128 128 122 127 100 96 72 ,... 134 130 124 134 100 106 82 81 I 8 141 142 125 117 113 99 107 81 9 175 135 119 126 110 115 106 66 10 125 138 126 129 109 92 63 83 111 136 144 103 100 93 121 80 69 12 161 116 118 124 111 111 90 75 13 142 142 127 126 121 104 74 61 14 147 142 118 130 119 100 100 63 15 162 123 138 106 104 111 96 59 I 16 I 141 146 133 110 108 115 85 85 17 I 148 140 146 112 116 118 96 72 18 I 131 145 133 118 86 89 97 73 19 I I 150 121 138 128 111 91 84 64 I 20 I 142 137 118 116 133 111 99 79 21 I 158 134 132 116 106 111 88 58 I Chapter 4 CONFIDENCE BOUNDS FOR THE T BEST POPULATIONS 4.1 Introduction The use of multiple cOlnparisons t.ests and silnultaneous confidence intervals, for corllparing a nUll1ber of populations, was initiated by such authors as Dunnett. (19ssL Tukey (194q) and Scheffe (1953), whose approaches have been discussed in the lit.erature review in Chapter 2. More recent.ly, Bofinger (1985) has used a confidence int.erval fonnulation to develop a more efficient procedure for pairwise ordering of populations~ this rule was also det.ailed in Chapter 2. All of these ll1ethods~ however, are not l1l0st efficient if the goal is to select "good" populations, that is, populations with large (or s111a11) pararneters of interest. For this goal, only SOl1le of the pairwise cOl1lparisons are of interest. As discussed in Chapter 2, a lllore appropriate Inethodology for this problern is the confidence interval approach developed by Hsu (1981, 1984a. 1984b). As discussed in Chapter 2, Hsu's approach was used by Bofinger (1986a) to investigate bounds for particular pararlleters of interest. For exalnple, Bofinger considered lower bounds for all paralneters as a function of the tth best paralneter, and upper bounds for all parallleters as a function of the (k - t )th worst paranleter. Following Hsu's (1985) results, Bofinger (1986b) has also considered unconst.rained 54 CHAPTER 4. CONFIDENCE BOUNDS 55 bounds on parameters of interest. Mengersen and Bofinger (1987) also considered a confidence bound approach to the selection of the t best from k populations, for both location and scale parame­ ters. By extending Bofinger's (1986a) results, they achieved improved (constrained) bounds for the particular selection goal considered. These results are discussed in detail in this chapter. The selection goal investigated by Mengersen and Bofinger (1987) may be stated as follows: Goal G1: Select 7ri,i E It, with It = {p( k~ - t + 1), p( k - t + 2), ... , p( k )} and the rules are based on observed values of the random variables Xi, i = 1, ... , k from populations 7ri with continuous distribution functions F( x - 8i ), such that and If 8p (k-t) is not strictly less than 8p (k-t+l)? it is usual to "tag" the population which is to be identified as best. This is not an important issue under the confidence bound approach, since the conclusions are in tenns of confidence intervals, irrespec­ tive of any equality anl0ng the 8 values. Ties between the X values may occur in a practical situation and Inay be broken at randOln. Because of the assumption of continuous distributions, however, this need not be considered in the development of the following theory. This chapter is divided into two parts: Part I investigates the location parameter case and Part II considers the scale paralneter case. For both parts, two nlain ainlS are considered: (a) Under the "natural" selection rule: Rule Rl: Select 7ri, i E Gt, with G t = {R( k - t + 1), ... , R( k ) } , CHAPTER 4. CONFIDENCE BOUNDS 56 construct a lower confidence bound for the difference between the minimum selected parameter and the maximum non-selected parameter, and (b) construct upper and lower bounds for all (}i,i = 1, .. ,k, as functions of the (k - t)th smallest parameter (}p(k-t) and the tth largest parameter (}p(k-t+l) respectively. These bounds may then be used to select. "good" populations (1I"i, i E Id and eliminate "bad" populations (1I"i' i (j It). The particular cases of selecting normal means with common unknown variance (Part I) and selecting variances from normal populations (Part II) are discussed in det.ail and appropriat.e tables are included. Details of these tables, including derivation, int.erpolation and approximations, are also presented. The results of this chapter are extended to other selection and ranking procedures and a numerical application is discussed. 4.2 Location Parameter Case 4.2.1 Comparison of the worst selected and best non-selected populations Consider the selection goal Gl and the corresponding procedure Rl. Letting we wish t.o const.ruct, a lower bound for (}L - (}M, the difference between the minimum selected parameter and the maximU111 non-selected paranleter. Such a bound cannot be regarded as a "confidence" bound in the usual sense, since Land l\,f are random variables. Instead, we can interpret it in a "prediction" sense, as suggested by Bofinger (1986b). That. is, if the observed value of the lower bound is greater t.han zero, we can predict with a given confidence that the populat.ions 1I"i' i E G i , will perfonn better than those 1I"i, i (j G t , in terms of the para111eter (), so that a "correct selection" has been nlade. For t = 1, Fabian (1962) construct.ed a lower bound on (}D = (}R(k) - nlax (}i i¥ R( k) CHAPTER 4. CONFIDENCE BOUNDS 57 of FL = min(O, XR(k) - XR(k-l) - d) and showed that For general t, Mengersen and Bofinger (1987) extended Fabian's results and proposed a lower bound on B L - B M of BL = min(O, XR(k-t+l) - "'YR(k-t) - d) . The "confidence" stat.ement. that can be made about. B L is given by Theorem 4.1. Theorem 4.1 by The minimum probability that the difference BL - BM is greater than BL is given That is, inf P{ BL - BM > min(O, XR(k-t+l) - XR(k-t) - d)} = Pk,t( d) . fiEO Proof The expression Pk,t( d) was noticed in Chapter 2 to be the probability correspond­ ing t.o the event. E 1 , say, that. all of t.he (st.andardised) variables Yi = ... Yi - Bi , i E It, are great.er than all of the values Yj - d = Xj - Bj - d, j tt It; that is, Mengersen and Bofinger showed that El irnplies t.he event of interest: and then noted that t.he least. favourable configuration of fi for the event E2 occurs for Bi =- B j - d Vi E '"'it, j tt '"'it , CHAPTER 4. CONFIDENCE BOUNDS so that the infimum of P{E2 } is given by P{E1 }. By writing E1 as Mengersen and Bofinger showed that and that {'t -I- G t } n E1 ~ {L tf- 1t and ME,t} n E1 =? { -~ L - 8 L < -~ M - 8M + d} =? {XR (k-t+1) - 8L < XR(k-t) - 8M + d} . Hence E} ~ E 2 • Moreover, the infinlum of Pk,t(d) is attained when we take which gives the required result. Normal distributions with common unknown variance 58 Consider the particular case in which Xi nN( 8i , 0-2n-1 ), i = 1, .. , k, with 0-2 unknown and estimated by 52, such that 1/52/ (T2 has a chi-squared distribution on v degrees of freedoill. The lower bound for 8 L - 8M becomes with "confidence'~ coefficient given by where <1>(.) is the standard normal distribution functioil and G v ( .) is the distribution function for 5/0-. If d and F(x) are replaced in Theorelll 4.1 by d5n- 1 / 2 and <1>(xn1 / 2 /0-) respec­ tively, the infi111U111 of the probability, conditional on 5 = 5, is easily shown to be CHAPTER 4. CONFIDENCE BOUNDS 59 the inner integral of (4.1). By integrating over the distribution of S/a-, we can see that the required probability is indeed p~~)( d), which proves Theorem 4.2. Theorem 4.2 with p~~)( d) given by (4.1). , Values of d satisfying are given in Table B.1 (located in Appendix B) for P*=0.75,0.90,0.95; k~=1, .. ,15; t= 1, .. [k/2]; v =5( 5 )20,24, 30,40,60,120,cx>. The computation of this table and a review of other relevant tables are given later in this section. Upper and lower bounds on all paraUleters Hsu's (1984b) simultaneous confidence bounds for OJ, j = 1, .. ,k, of [Op(k-tlj) + nun(O, )[j - .YR(k-tU) - d), Op(k-tlj) + nlax( 0, )[J' - XR(k-tlj) + d)] , for 0 p(k-tlJ) the (k - t )th slnallest Oi Vi =I j and "'YR(k-tlJ) the (k' - t )th smallest Xi Vi i- j, ( 4.2) with confidence coefficient Pk,t( d), have been discussed in Chapter 2. These intervals were derived using the pivotal event E 1 • Mengersen and Bofinger (1987) have 11lodified these bounds to deal with the tth largest and (k: - t )th snlallest paranleters, by replacing e p(h-tlj) by () p(k-t+l) in the lower bound and e p(k-t) in t,he upper bound. They achieve this by noticing that () p(k-tlJ) = { () p( k - t) V j E It ()p(h-t+l) Vj rf- It and CHAPTER 4. CONFIDENCE BOUNDS For the lower bounds, then, Vj E Tt, Op(k-t+l) is correct and Vj rt It, ()p(k-tlj) = Op(k-t+l) Similarly, for the upper bounds, Vj rt Tt, Op(k-t) is correct and Vj E It, 8p(k-t/j) = 8p(k-t) Hence, the bounds for 8j ,j = 1, .. , k, may be written as with confidence I j = [Bp(k-t+l) + nlin(O~ Xj - XR(k-tlj) - d), Bp(k-t) + max(O, -'Yj - "YR(k-tlj) + d)] (}inf P{8i E I j Vj = 1, .. , k} = Pk.dd) . _EO 60 ( 4.3) As demonstrated by Hsu (1984), these intervals may be used in selection proce­ dures for the goal: (a) Select "good" populations and (b) Eliminate "bad" populations. The appropriate rule is to select as "good" all populations 7ri for which (since XR(k-tlj) = XR(k-t) V j E G t ) and elilninate as "bad" t.hose for which (since XR(J-,-tlj) = "YR(k-t+l) Vj E G t ). Notice that. this rule is equivalent to selecting all populations 7ri with lower bounds on (}i (given by (4.3)) greater than 8p(k-t+l) and elinlinating all populations 7ri with upper bounds on Oi less than (}p(k-t). As discussed in Chapter 2, this confidence bound approach is 1110re infonnative than either the subset selection or indifference zone fonnulations, since information on all 81·~ i = 1, .. , k, is provided. Hence, populations that are "ahnost good" (with a. lower bound not too luuch below (}p(lt-t+l») and those "nearly bad" (with an upper CHAPTER 4. CONFIDENCE BOUNDS 61 bound not much above Op(k-t») may be identified and may be of interest to the experimenter. If the goal consists only of (a) or (b), however, this procedure can be improved by considering the single selection goal, as detailed in Chapter 5. Notice also that, since these int.ervals use the same pivotal event El as when finding the bounds on OL - OM, we can, with t.he same joint confidence Pk,t( d), put. a lower bound on OL - OM of BL and construct intervals for OJ of Ij,j = 1, .. , k. Normal distributions with common unknown variance Consider the particular case of cOlllparing normal means, discussed above. Using these results we call, \vith confidence p~~\ d) given by (4.1), construct intervals for OJ , j = 1, .. ~ k ~ of [Op(k-t+l) + min(O, Xj - XR(k-t/j) - dSn- 1/2), Op(k-t) + max(O, Xj - XR(k-t/j) + dSn- 1/2)] Hence, with confidence p~~)( d), we can simultaneously: (a) Select 1Ti with "¥i - XR(k-t) 2:: dSn -1/2 as "good"; (b) Elinnnate 1Ti with X 1" - .YR(k-t+1) ~ -dSn-1/2 as "bad", and (c) State that OL - OM > min(0,XR (k-t+1) - XR(k-t) - dSn- 1/2 ). As mentioned above, values of d satisfying p~~)( d) =0.75, 0.90, 0.95 for various (1'/, k~, t) combinations are presented in Table B.1 (see Appendix B). Discussion of table of values for p~~)( d) = P* Existing Tables The expression p~~)( d) given by (4.1) has been discussed and tabulated by a llUlllber of authors, including Bechhofer (1954) and Gupta (1963). For finite degrees of freedolll and general values of t and p, the corresponding prob­ ability expression is given by ( ) 10 00 j+oo Up1/2 - dS / a Pk'~ (d,p) = (1 - q,( ( )1/2 ))td(q,(u))k-tdGv(S/a) o -00 1 - p (4.4 ) CHAPTER 4. CONFIDENCE BOUNDS 62 with p the common correlation coefficient. This reduces to p~~)( d) given by (4.1) for p = 0.5. For t = 1, (4.4) has been tabulated by Milton (1963), Gupta (1965) and Gupta, Nagel and Panchapakesan (1973) for known variance (v = (0) and by Dunnett and Sobel (1954), Dunnett (1955), Gupta and Sobel (1957), Krishnaiah and Armitage (1966) and Gupta, Panchapakesan and Sohn (1985) for general values of v. The range of parameters covered and accuracy of values tabulated by the above authors are sUlllnlarised in Table 4.1. The computational techniques used to produce these tables differed with each author. Dunnett (1955) bracketted the required probability by d - O.l,d + 0.1 and then used inverse interpolation to find an appropriate d value. The Cornish­ Fisher expansion with four adjustInent tenns was used by Gupta and Sobel (1957). For d = 0.1(0.2)6.1, Krishnaiah and Anuitage (1966) used 40- point Gauss-Hermite quadrature and 48-point Gauss quadrature for the inner and outer integrals, respec­ tively, of the expression P~~)(d), and then used cubic interpolation to find an appro­ priate percentage point. Gupta, Panchapakesan and Sohn (1985) applied Hartley's (1944) method of approxilllating P:~)\ d, p) by a finite differential-difference equa­ tion, in which the first term corresponds to Pk.I(d) (infinite degrees of freedom) and the four succssive correction tenns are in powers of V-I. The secant method was then used to find the required percentage point. Depending on the value of p, the integral expression in each of the tenns was COlllputed using either 60-point Gauss-Hennit.e quadrat.ure or a Gaussian quadrature formula over int.ervals of length 0.5, with the range of integration depending on the value of p. COlnput.er evaluation of PJ.~~\ d) (or d) is available using RANKSEL (Edwards, 1985), for general t and 1/ = 00, and RS-MCB (Gupta and Hsu, 1984), for t = 1 and general v. More recently, Jones, Butler, \'\;-right. and Swain (1987) have produced a progranl for calculating percentage points of Pk,l (d) and produced tables for k up to 2000. The authors used a Newt.on-Cotes three-point fonnula with finite range based on the required error bounds. The 11lodified regula falsi luethod was then applied to obtain the required percentage point. No. 1 2 I 3 4 5 6 7 8 9 10 CHAPTER 4. CONFIDENCE BOUNDS Table 4.1: Existing tables for p~~)( d, p) Author k (a) (b) Bechhofer (1954) dV2 4 2( 1 )10, 11 12 13,14 Gupta (1963) d 3 2(1)51 Gupta, Nagel and d 4 2(1)11 Panchapakesan (2)51 (1973) James, Butler, d 6 2(1)401 Wright and Swain (10)2001 (1987) Milton (1963) d 8 3(1)10 (5)25 Dunnett and d 3 2 Sobel (1954) pA- 5 2 Dunnett (1955) dV2 2 2(1)10 Gupta and d 2 2(2}10 Sobel (1957) (1}16(2)20 ( 5)40,50 Krishnaiah and d 2 2(1)11 Armitage (1966) Gupta d 5 2(1)10(2)20 Panchapakesan and Sohn (1985) (a) Value tabulated by author (b) Decimal places in tables t p. /d (c) 1(1 )[k/2] .05(.05).80 2(1)5 (.02}.90(.01) 3(1)5 .99,.995, 5 .999,.9995 1 .75,.90,.95, .975,.99 1 .75,.90,.95, .975,.99 1 .50,.80,.70, .95,.975,.99, .995,.999 1 as for 3 1 .50,.75,.90, .95,.99 1 .O( .25 )2.5 (.5)10.0 1 .95,.99 1 .75,.90,.95 .975,.99,.9975, .999,.9995 1 .95,.99 1 .75,.90,.95,.99 .90,.95 (c)Parameter points (p. or d as applicable) at which (a) is tabulated. v 00 00 00 00 00 1(1)30(3) 60(15 )120,150 300,600,00 as above 5( 5 )25 15( 1 )20,24 30,36,40,48, 60(20)120, 360,00 5( 1 }35 15(1}20,24 30,36,48,60, 120,00 15,17,20,24,36 60,120,00 63 p .5 .5 .10,.125,.20, .25,.30,1/3, .375,.40,.50, .60,.625,2/3 .70,.75,.80, .875,.90 .5 as for 3 ±.5 ±.5 .5 .5 .0(.1 ).9 .1{.1).6 .7(.1).9 CHAPTER 4. CONFIDENCE BOUNDS 64 Computation of values of p~~)( d) In order to calculate the percentage points satisfying p~~)( d) = P*, with p~~)( d) given by (4.1), Mengersen and Bofinger (1987) considered an approxitnation using Bechhofer's (1954) tables. This method, detailed below, reduces the double int.egral in the probability expression to a single integral and then approximates this int.egral by summation. The resulting expression is Inuch faster to compute and gives values to within 0.01 of the exact percentage point. Mengersen and Bofinger expressed (4.1) as P(//)(d) k,t 10 00 Pk,t (dS / (J )dG//( S / (J) 1 - 10' Gv(u/d)dPk,,(u) with Pk,d 11.) t.he inner integral of (4.1). ( 4.5) As noted earlier, percentage points of Pk,t('u) have been extensively tabulated by Bechhofer (1954). Mengersen and Bofinger thus approximated P~~)(d) by with summation over the probability int.ervals considered by Bechhofer and using his tabulated values of 11.. (b..Pk,t( u) denotes forward differences between the values of Pk,t( 11).) The distribut.ion function GA·) was evaluated using the Numerical Algorithms Group (NAG) Fortran library routine G01BCF. This routine ilnplements an algo­ rithul based on t.hat of Hill and Pike (1967) which uses the recursive procedure (X/2)///2('-x/2 P(x,v + 2) = P(x.v) + / ) , r( v 2 + 1 wit.h P(x,2) = (-x/2 ; P(x, 1) = 2(1 - <1>( VX)) and P(x,v) = 1 - G//(x) . CHAPTER 4. CONFIDENCE BOUNDS 65 For large degrees of freedom (v 2:: 120), the routine G01BCF uses the following normal approxinlation to P(x, v). If X has a chi-squared distribution on v degrees of freedom, then (9V)1/2[(~Y )1/3 + ~ -1] 2 v 9v has approximately a standard normal distribution. The method of false position discussed by Lorenzen and McDonald (1981) was used to iterate to a value of d, so that 1 P* - P 1< b = 2.0E - 5 . U sing initial values of dl = 12.0 and d2 = 0.50, an intermediate value d3 was calculated, so that d3 = d1 - ~(d1)(d2 - d1)/(Ll(d2) - Ll(d1)) with Ll( d) = P~~)( d) - P*. Depending on the sign of Ll( d3), d3 replaced either d1 or d2 and the process was repeated until Ll( d3) < h. The resulting value of d3 was then rounded to two decimal places (dF, say) and Ll( dF) was computed. If ~(dF) 2:: 0, then dF became the final value. Otherwise, Ll( dF + 0.005) was computed. If this value was positive, dF renlained the final value; otherwise dF was increased by 0.01 and !l.( dF) was again computed. The resulting values of d satisfying P* = 0.75,0.90,0.95 for various k, t and v are presented in Table B.l. (Since P~~)(d) = p~~!_t(d), only t values up to [k/2] are tabulated. ) For {k, t} cOlnbinations not considered by Bechhofer, the percentage points sat­ isfying the probability levels used by Bechhofer were cOlnputed using the package RANKSEL (Edwards, 1985), which inlplelnents a 16-panel Gauss- Hermite quadra­ ture routine. Accuracy considerations In order to investigate the accuracy of the values presented in Table B.1, u values satisfying Pk,t( u) equal to the mid-values of the probability intervals used by Bechhofer (1954) were calculated using RANKSEL. The new d values, dN EW, say, were conlputed over this finer SUlnnlation for various CHAPTER 4. CONFIDENCE BOUNDS 66 {P*, v, k, t} combinations and were compared with the corresponding values dTAB , say, from Table B.lo No observed difference exceeded 0.0035 and the larger differ­ ences occurred, as expected, for small v and large P*. In most cases, dT AB was larger than dNEW. (The exceptions were for P* = .95, t = 1 and large v.) The NAG routines employed in producing the values in Table B.1 were single precision routines. An alternative method of calculating the values was also used, in order to provide a second check on the tabulated values. This program (referred to hereafter as Pktnud) modifies Jones, Butler, Wright and Swain's (1987) program for calculating d values corresponding to Pk ,l (d) to deal with general values of t and then inlplements the method proposed by Gupta, Panchapakesan and Sohn (1985) to approxilllate the outer integral of (4.1). The new percentage points are expected to be accurate to at least four decimal places, but there is a marked increase in CPU time required to compute the values. Selected percentage points were compared with those in Table B.1, with agreement in all cases to within one in the second decimal place. Dunnett's (1955) values d-/2 in his Tables 1a and 1b, with his p value corre­ sponding to k - 1, also do not differ by more than 0.01 from the corresponding values of d in Table B.lo Comparison was also made with Gupta, Panchapakesan and Sohn's (1985) values of d-/2, taking t = 1, p = .5 and k = K + 1 (K being used by Gupta, Panchapakesan and Sohn). In the small number of cases in which the pairs of (rounded) values differed in the second decilllal place, the values in Table B.1 were the more conservative. These values were altered to agree with Gupta, Panchapakesan and Sohn's more accurate values in these cases. As a further test for any "outliers" in Table B.1, for each {P*, t} combination, the d values were regarded as a response surface over k and v, so that d = f1 + /31 k + /321/ + (/311 A~2 + /312 kv + /322V2) + + (/3111 k3 + /3112k 2v + ... + /32221/3) . (4.6) Under this nlodel~ the nlultiple regression routine BAR3T, incorporated in the computer package BAR3 (Burr, 1975), was used to obtain expected values of d and CHAPTER 4. CONFIDENCE BOUNDS 67 corresponding residual mean squares (RMS). Over all of these response surfaces, twenty-eight values of d corresponding to slightly larger RMS values were identified. Many of these values occurred for very small or large {k, t, v} combinations and most occurred for k = 2, t = 1. All values so identified were confirmed to be accurate to the two decimal places, by comparison with Gupta, Panchapakesan and Sohn's (1985) tables (for t = 1) and by computation using the routine Pktnud described above for other values of t. Interpolation For values of v not included in Table B.1, the usual harmonic interpolation is suggested. This appears to be quite satisfactory, as illustrated by Figure 4.1, in which values of d are plotted against 22 values of V-I for selected values of k and t. The values of d for v not in Table B.1 were calculated using the routine Pktnud. For values of p~~) (d) between 0.75 and 0.95, linear interpolation using d2 and In( 1 - p~~) (d)) may be used. In Figure 4.2, values of d2 are plotted against seven values of In(l - P~~\d)) for various {k,t,v} combinations, with values of d for the additional p~~)( d) computed using the routine Pktnud. It appears that this interpolation is reasonable for v > 5. Further applications The availability of tables for p~~)( d) for general values of t and v Inay enable some Inethods derived for the infinite degrees of freedom (known variance) case, which utilise values of Pk,t( d), to be extended to the case of general degrees of freedoln. Similarly, SOllle Inethods derived for t = 1 and general v may be ext.ended to the case of general t values. In other cases, solutions to ranking and selection problellls have involved the expression p~~) (d) but, partly because of the unavailability of tables, upper bounds on the derived probabilities have been derived instead, in order to use existing tables (of p~~\ d) or p~,C;\ d)). Such results lllay be ilnproved by the direct use of the expression p~~\ d). One such exanlple of this is discussed below. As discussed in Chapter 2, Bofinger (1985) has proposed a lnethod of ordering k nonnal populations based on their Ineans, using an "honest ordering" difference CHAPTER 4. CONFIDENCE BOUNDS P = 0.75, K = 4 2.15 8 2.OsL:J 1.15 1.75 1.15 '--__ '--_--1'--_--1'---_--1 0.111 D.IfI D.1D 1/, - I:: 1 P = 0.75, K = 15 -- - I:: 2 4.05 - - I:: 3 -- I:: ~ 185 -- 1::$ - I:: 6 105 D.l~ D.2O :l _______ 1~~ .. 1_ - I D.III D.IfI l10 D.1~ D.2O 1/t P = 0.95, K = 4 4.1 ~ uL:J 31 .u 3.1 , , , //,/' , , , , , " , , , , , , - I:: 1 P = 0.95, K = 15 --- I:: 2 1.6 -- 1::3 , , , , , , Figure 4.1: Plot. of d vs 1/1/ for various t , , , , 68 " CHAPTER 4. CONFIDENCE BOUNDS 69 K=2,T=1 DIgrees of f I1edom 2D o.,ees of Freedom - ,=5 - ,=5 --- ,:20 --- y = 20 - - w= «J -- ,=60 16 -- ,=40 -- ,=60 - - ,= 120 - - y = 120 ~ ~~~~~~~~~~~~ -3.0 -2.8 -2.6 -2.4 -71 -7.0 -1.1 -1.6 -1.4 -11 -71 -7.' -2.4 -21 -2.0 -1.8 -1.1 -1.4 -11 M(1-P) 1n{1-P} K = 8, T = 1 K = 15, T = 3 zo DIgrtes of FIIIdom C Degrees of r rtedom - , = 5 - ,= 5 --- , = 20 --- y = 20 - - , = «l - - ,: 40 -- , : 60 y = 60 - - , = 120 - - y = 120 ~12 -1)25 21 4~~~~~~--~~~~ -3.0 -2.8 -2.6 -2.4 -71 -7.0 -1.1 -U -1.4 -11 In(1-P} Figure 4.2: Plot of d2 vs In( 1 - p~~) (d)) for various v , CHAPTER 4. CONFIDENCE BOUNDS 70 (HOD). Joint confidence intervals for (}i-(}j Vi,j=I, .. ,k of the form were constructed with confidence with Yi independent standard nOrInal variables. Bofinger tabulated values of or)( 0) satisfying for k =2(1 )10, 0 =0.05,0.01 and values of v from 10 to 00. Because of the conlplexity of R}:)(or)(o)), the tabulated values are actually upper bounds on o~l/)(o) for k ~ 4 (with the exception of k = 4, v = 00, for which the percentage point is exact). For k = 4, Bofinger derived the result. ( 4.7) with Instead of directly computing p~~) (d) for t > 1, Bofinger used a lower bound ( 4.8) with If percentage points satisfying (4.7) are used instead of the tabulated conserva­ tive values (sa.tisfying (4.8)), Bofinger's intervals may be ilnproved. The conservativeness of Bofinger's tabulated values was investigated by conduct­ ing a Monte Carlo experiment for k =4(2)10, v =10,30,120,00 and 0=0.01,0.05 to CHAPTER 4. CONFIDENCE BOUNDS 71 approxinlate the probability of Type III error (asserting that a population is better than another when in fact it is worse) given by P{ at least one Xi - Xj > dS I Bi = Bj} . The least favourable configuration of means Bp(i) = Bp(i+t)-, i = 1, .. , k - 1 was used, with each combination of {Rr) (d), v, k} considered as a separate exper­ inlent. The satne set of randoln variables was used for each experiment since, as indicated by Dunnett (1980), this inlproves precision by introducing a positive error correlation between the sinlulated probabilities and hence decreasing the standard error expected in the differences between the probabilities. For each experinlent, 100,000 simulations were conducted, giving a standard error of less than 7E-4 for a = O.S and 3E-4 for a = 0.01. The NAG Fortran library routines GOSDHF and GOSDHF were used to simulate an estinlate S2 of (1"2, and k independent nOrInal variables Yi, i = 1, .. , k, respectively. The results of the experiment are presented in Table 4.2. It appears that the bounds are not too conservative for all k and v considered, but perfornl better for small k and for large v. For k = 4, the less conservative percentage points satisfying (4.7) nlay be lnore desirable for small {Rr)(d), v} combinations. Unfortunately, using p~~\ d) directly in Bofinger's expressions for k > 4 does not. as readily iInprove the bound, although the results of the Monte Carlo experiment indicate that it is for larger values of k that such improvements lnay be desirable. 4.3 Scale Parameter Case 4.3.1 Extension of Location Parameter Results Consider the problem of identifying the t populations with the snlallest scale param­ eters. (Obvious nlodifications lnay be l1lade if large scale parallleters are required.) In this case, the randolll variables li,i = 1, '" k, have cont.inuous distribution func­ tions F(y/~'d, 'l/Ji > 0 Vi = 1, .. , I . .'. CHAPTER 4. CONFIDENCE BOUNDS Table 4.2: Simulation results: Values of a for a = 0.05,0.01 a is observed probability of Type III error. Q = .05 Q = .01 k/v 10 30 120 00 10 30 120 00 4 .043 .044 .047 .049 .009 .010 .010 .010 6 .039 .040 .043 .043 .008 .009 .009 .010 8 .035 .039 .040 .041 .007 .008 .008 .009 10 .032 .037 .041 .041 .007 .008 .008 .009 Let the 'l/'i and li be ordered as follows: 72 The "good" populations are now those with the smallest values of 'ljJ, so the goal is to select those 7ri, i E It', where It' = {p(1),p(2), .. ,p(t)} (4.9) and the corresponding procedure is to select those 7ri, i E Gt', where G t' = {R( 1 ), R( 2), .. , R( t )} . (4.10) The results obtained in the previous section may be directly extended to this case, after nlaking the transfonnations e = - log 1)'; ~y- = - log 1- and d = - log c . The two problel11s detailed for the location paranleter case are discussed below. a) The first problem is to conlpare t.he largest selected value of·ljJ (1/'L) with the slnallest non-selected value of 1/~ ('lj'M). For this we consider the ratio 7/)LI~'M CHAPTER 4, CONFIDENCE BOUNDS 73 and obtain a lower bound of so that. where (4.11) is the infimum of t.he probability of correctly selecting the t smallest scale parameters using Bechhofer's Indifference Zone approach (the indifference zone being given by with c prespecified) and using the selection rule given by (4.10). Hence we lllay choose {7rR(l), .. , 7rR(t)} as the populations corresponding to the t snlallest scale parameters and, if YR(t)/}TR(t+l) < c, we may say with confidence Qtl ( c) that a correct selection has been made. b) The second problem is to obtain lower and upper bounds for all V'j j = 1, .. , k, with the lower bounds functions of'l/'p(t) and the upper bounds functions of ~p(t+l)' U sing the results of the previous section, we obtain an interval for '1/'j, V j = 1, ." k of [Vlp(t+l) lllin(l, cYj/YR(tlj)), 'if'p(t) max(l, c-1}j/lR(tij»)] with confidence coefficient Q~~l(c) given by (4.11) and lrR(tij) t.he tth snlallest of Hence, with confidence Qtl (c), we can select as "good" those populations 7ri for which the upper bound on V'i is ~p(t) and elilninate as "bad" those 7ri for which the lower bound on 1/\ is 'If'p(t+l)' Also, all bounds in a) and b) nlay be obtained silllultaneously with no decrease in the confidence. CHAPTER 4. CONFIDENCE BOUNDS 74 Normal distribution case For the particular case in which Yi is the sample variance based on n observations froin a normally distributed population 7ri, i = 1, .. , k, the distribution function F( y / 7f'i) is chisquare on v = n - 1 degrees of freedom. The expression Qk~l ( c) for this case has been tabulated by Carroll, Gupta and Huang (1975) as a function of k, t, rand d2 , with their value of r equivalent to v and their d2 corresponding to c. This table is quite linlited in its parameter range (with a selection of cOinbinations of k· :::::: 10. t :::::: 5 and v between 2 and 8). Also, for the procedures considered in this chapter it is nl0re convenient to tabulate the percentage points as functions of k, t, v and Q~~}( c). Such tables for t = 1 have been provided by Bechhofer and Sobel (1954), who considered selection of scale parameters under an indifference zone approach, and by Gupta and Sobel (1962), who used a subset selection approach to this probleul. Table B.2 presents percentage points for the case of general values of t, for various conlbinations of {Qtl(c),v,k,t}. (This table is located in Appendix B.) The values in Table B.2 were cOlnputed using the 64-point Gauss-Laguerre quadrat.ure routine D01BAF from the NAG Fortran library. Using the method of false position (described in the previous section) for iteration, a value of c satisfying I Qk~l(c)-p* l k: /2, c appears to be closer to the percentage point of Qi~l-t( c) than that of Q~~l( c). These observations, and the general liberality of the approxinlation, indicat.e that t.his approxiIllation Illay not be generally useful for t > 1. It may be possible to improve the approximation by introducing some constant (perhaps a function of k, i, v and/or P*), but this has not been considered here. 4.4 Example U sing the exaillple developed in Chapter 3, we have 6 lecturers with final scores conlbined over 21 judges, given by: Carole Debbie Wayne \Vendy Russell Janice 134.7 125.2 118.7 111.5 89.5 70.2 wit.h s2 = 107.04 on 104 degrees of freedoll1. CHAPTER 4. CONFIDENCE BOUNDS 1\1 • . o K = 2, T = 1 OJ K = 10, T = 3 Q.5 ~-- --0.4 --- ------------ -- D.2 0.1 D.O 0.1.t 0.111 D.!I P o.gnes 01 FrIICIom V = 5 V = 20 V = 40 v = 60 V = 120 Degrees 01 Fnedom V=5 V = 20 V = 40 V = 60 V = 120 -- ................. , ...... '-, ............. I 0.11 O.fl t 0). With 75% confidence, then, the students may award Carole and Debbie the prizes for "best" lecturers. If 95%, confidence is required, however, the appropriate value of d from Table B.1 is 3.46 and the corresponding statelllents that lllay be made are: • declare only Carole to be "good"; • declare only Wendy, Russell and Janice to be "bad"; • declare that the s111allest true score of Carole and Debbie is no more than 1.32 "points" below the largest true score of the others (since OL - OM > -1.32). In this case~ the students nl.ay be prepared to award the prize to only one lecturer, Carole, with 95% confidence. Alternatively, if Debbie is worthy of consideration on some other grounds (that were not. considered in the scoring process), it. lllay be decided that. a difference of no lllore than 1.32 bet.ween her true score and the largest true score of Wayne, Wendy, Russell and Janice is tolerable, and hence the two prizes would be awarded to Carole and Debbie. Notice that. in this case, the confidence that both Carole and Debbie are "good" is not 95%, but. the confidence CHAPTER 4. CONFIDENCE BOUNDS 79 bounds have provided the judges with information on which to make their final selection decisions.