A simple method of determining precisely the degree to which the mental test of 28 school children reflects or is related to their scholarship records and the teacher’s estimate, as shown in the table on Page 96, is to plot the relationship graphically, which has been done in the accompanying diagrams.
In each diagram a heavy diagonal line shows approximately where the plotted points would fall if the relationship were perfect between the numbers of errors in the educational measurements and the other measure of ability. It is clear that the relationship shown in each diagram is far from perfect, but it is not clear from the diagrams which rating of the teacher is most nearly approximated by the educational measurement scores. To discover this relative degree of relationship, a mathematical calculation must be made. For the purposes of testing the correspondence between the scores in the various Mentimeter tests and the production records or supervisor’s ratings of the group of persons tested, it is sufficient to calculate what is best called “a coefficient of coördination.”[7]
7. Calculated by a somewhat more complex formula, approximately the same measure of relationship might be found, called by the more familiar name “coefficient of correlation.”
The first step in the calculation of a coefficient of coördination is the transformation of the original scores into figures indicating order of merit. In the case of the sixth-grade class here referred to, the teacher’s ratings of intelligence need not be changed, for they are exactly the kind of ratings necessary: 1 indicating the brightest and 28 the dullest pupil, so far as the teacher was able to judge her pupils at the end of a year’s work. Since the educational measurements scores reported are the number of errors made by each child, the rank of the child making the smallest numbers of errors will be 1, while the rank of the pupil making the largest number of errors will be 28. On the other hand, the scholarship marks are the summaries of the teacher’s percentage marks for a half year, hence the best pupil is the one making the highest percentage. In scholarship, then, the highest percentage should get the rank of 1 and the lowest percentage a rank of 28.
The first three columns of the following table give ranks in the place of the original figures which indicated numbers of errors in measurements and percentage in scholarship. Where two or more individuals are entitled to the same rank, the figure used is the middle value of the ranks. Thus in the case of the educational measurements scores, two girls made 16.5 errors. There are but two pupils making better showings, and therefore Ruth and Helen would normally rank third and fourth, but since we have no evidence as to which should rank third and which fourth, each is given a rank of 3.5. Similarly it will be observed that Alexander, LaMonte, and Leo each obtained a percentage of 93 in scholarship, therefore the three boys named share equally the fourth, fifth, and sixth rank, each being given 5 as a rank; and the next highest pupil, Amelia with a percentage of 92, is given 7 as a rank.
| RANKING OF SIXTH-GRADE PUPILS | DIFFERENCES IN RANKINGS | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Name of Pupil | A Educational Measurements |
B Teacher’s Ranking |
C Scholarship Marks |
A to B | A to C | B to C | |||
| d | d2 | d | d2 | d | d2 | ||||
| Adelaide | 12 | 19 | 18.5 | 7 | 49 | 6.5 | 42.25 | 0.5 | 0.2 |
| Ruth | 3.5 | 15 | 9 | 11.5 | 132.25 | 5.5 | 30.25 | 6 | 36 |
| Alexander | 9 | 7 | 5 | –2 | 4 | –4 | 16 | 2 | 4 |
| LaMonte | 14 | 6 | 5 | –8 | 64 | –9 | 81 | 1 | 1 |
| Earl | 28 | 18 | 24 | –10 | 100 | –4 | 16 | –6 | 36 |
| Joseph | 6 | 20 | 18.5 | 14 | 196 | 12.5 | 156.25 | 1.5 | 2.2 |
| Amedeo | 27 | 14 | 18.5 | –13 | 169 | 8.5 | 72.25 | 4.5 | 20.2 |
| Leo | 16 | 3 | 5 | –13 | 169 | –11 | 121 | –2 | 4 |
| William | 17 | 9 | 21 | –8 | 64 | 4 | 16 | –12 | 144 |
| Isabel | 8 | 21 | 25 | 13 | 169 | 17 | 289 | –4 | 16 |
| Ida | 13 | 4 | 3 | –9 | 81 | –10 | 100 | 1 | 1 |
| Hazel | 1 | 10 | 9 | 9 | 81 | 8 | 64 | 1 | 1 |
| Frederick | 23 | 26 | 16 | 3 | 9 | –7 | 49 | 10 | 100 |
| Charles | 20 | 13 | 18.5 | –7 | 49 | –1.5 | 2.25 | 5.5 | 30.2 |
| Edward | 11 | 1 | 2 | –10 | 100 | –9 | 81 | –1 | 1 |
| Benjamin | 22 | 24 | 26 | 2 | 4 | 4 | 16 | –2 | 4 |
| Bruce | 19 | 22 | 14 | 3 | 9 | –5 | 25 | 8 | 64 |
| Alden | 18 | 12 | 14 | –6 | 36 | –4 | 16 | –2 | 4 |
| George | 21 | 17 | 14 | –4 | 16 | 7 | 49 | 3 | 9 |
| Alice | 10 | 11 | 12 | 1 | 1 | 2 | 4 | –1 | 1 |
| Almira | 2 | 5 | 1 | 3 | 9 | –1 | 1 | 4 | 16 |
| Helen | 3.5 | 2 | 9 | –1.5 | 2.25 | 5.5 | 30.25 | –7 | 49 |
| Elizabeth | 24 | 23 | 27 | –1 | 1 | 3 | 9 | –4 | 16 |
| Amelia | 7 | 8 | 7 | 1 | 1 | 0 | 0 | 1 | 1 |
| Edwin | 5 | 16 | 11 | 11 | 121 | 6 | 36 | 5 | 25 |
| Robert | 25 | 28 | 28 | 3 | 9 | 3 | 9 | 0 | 0 |
| Edna | 15 | 27 | 23 | 12 | 144 | 8 | 64 | 4 | 16 |
| Samuel | 26 | 25 | 22 | –1 | 1 | –4 | 16 | 3 | 9 |
| Σd2 = 1790.5 | 1411.5 | 611.0 | |||||||
The coefficient of coördination, being an index number to show the closeness with which two rankings correspond, is dependent upon the differences between the rankings of the various individuals in the two measures being compared. The formula used is ρ = (6Σd2)/n(n2 − 1), where ρ stands for the coefficient of coordination, d stands for the difference between an individual’s rank in the two measures, and n stands for the number of individuals ranked in the two traits. The capital sigma, Σ, stands for the sum of whatever follows it, in this case the squares of the differences between the two rankings.
We may now employ the formula to find the coefficient of coördination between rank in educational measurements and rank in the teacher’s judgment as to intelligence. The difference between the ranks in column A and column B of the above table is given in the fourth column. Adelaide had a 12 in column A and a 19 in column B, so the difference (7) appears in the fourth column and its square (49) in the fifth column. Similarly the difference between Ruth’s 3.5 and her 15 is 11.5, the square of which is 132.25. Finding the squares of all the differences between rank in A and rank in B, and adding these squares together at the bottom of the table gives 1790.5, which may now be substituted in the formula for Σd2. n, the number of pupils is in this case 28, and therefore n(n2 − 1) is 28 (28 squared less 1) = 28 (784 − 1) = 28 × 783 = 21924. The substitution in the formula then goes as follows;
| ρ = 1 − | 6Σd2 | = 1 − | 6 × 1790.5 | = 1 − | 10743. | = 1 − .490 = .510 |
| n(n2 − 1) | 28 × 783 | 21924. |
The coefficient of coordination between rank in the educational measurements and rank in the teacher’s estimate of intelligence for the sixth grade class is .51, which suggests the question of how to interpret a coefficient after it is found.
A coefficient of 1.00 would mean perfect coördination and would only be found when there were no differences whatever between the two rankings considered. Such a perfect relationship will probably never be found, except by some freak of chance, for even when a group of persons is retested with the same test there is almost certain to be some change in their relative standings. A coefficient of 0.00 would indicate no relation whatever between the two rankings, while a coefficient of –1.00 would mean perfect correlation of a negative sort, the person getting highest in one measure getting lowest in the other, the person scoring next to the highest in one scoring next to the lowest in the other, and so on. Perfect negative correlation is as infrequent as perfect positive correlation.
The coefficient found between the teacher’s estimates of intelligence and the results of educational measurements, .51, indicates a really useful degree of coördination. Unless a Mentimeter test shows a coefficient of coordination of .25 or more with the production records (or other reliable measure of true ability), it may be considered as having little value in helping to select and differentiate men for that particular line of work. If the coefficient is above .5, the test is quite useful, and the nearer the coefficient approaches 1.00 the more confidence one may place in the test as a means of selecting and classifying men in that particular field.
The sixth column of the table on page 329 gives the difference between the test results rankings and the scholarship marks rankings, and the seventh column gives the squares of these differences, the sum of these squares being given at the bottom of the seventh column as 1411.5. By substituting in the formula,
| ρ = 1 − | 6Σd2 | = 1 − | 6 × 1411.5 | = 1 − | 8469. | = 1 − .386 = .614, |
| n(n2 − 1) | 28 × 783 | 21924. |
it appears that the tests more closely correspond with the average of the scholarship marks given by the teacher than with the teacher’s estimate of intelligence. This is partly to be explained by the fact that the tests given were measurements of ability in school subjects rather than tests of intelligence, and still more by the fact that the teacher gave scholarship marks on the basis of relatively objective examinations while her estimates of intelligence are always wholly subjective.
The eighth and ninth columns on page 8 give the differences between the ranks in the teacher’s estimates of intelligence and the ranks in the scholarship marks given during a half year. The coefficient of coördination worked out from these differences is
| .833 | ( | ρ = 1 − | 6 × 611 | = 1 − | 3666 | = 1 − .167 = .833 | ) |
| 28 × 783 | 21924 |
which would seem to indicate that the teacher drew very heavily on her knowledge of the relative scholarship of her pupils in making her estimates of their intellectual capacities.
The three coefficients worked out above for 28 pupils in a sixth grade are typical of the mathematical relationships the reader will wish to work out between known degrees of ability in a certain type of work and the results of the Mentimeter tests. The coefficients of coördination for the sixth-grade pupils studied above are, between
| Educational Measurements and Estimated Intelligence | = .51 |
| Educational Measurements and Scholarship Averages | = .61 |
| Estimated Intelligence and Scholarship Averages | = .83 |
No method of forecasting degree of success in one line of work from quality of performance in another task (or in a test) will give a perfect coefficient of coordination of 1.00, but the nearer the coefficient approaches 1.00 the more reliance one may put in the test which furnishes such a ranking of the individuals.