Archival study

 

You may want to assign thefollowing article:

Burns, B. D.(2004). The effects of speed on skilled chess performance. PsychologicalScience, 15, 442-447.

 

The author uses archival data(records of chess matches) to try to estimate the extent to which differencesin chess skill is due to quickly recognizing situations as opposed to being dueto contemplating a wide range of possible moves. The article is short and theauthor uses fairly simple statistics (Pearson rs and one-way ANOVA). To help your students understand the article,give them Table 1.

 

 

 

 

 

 

Table 1

Helping Students Understand the Article

Section

 Tips, Comments, and Problem Areas

Abstract

… restricting search processes: There are two ideas here. First,  because thinking through possible moves is part of chess skill, a better player will be better at thinking through moves than a poorer player. Therefore, if you speed up the chess game so that the better player cannot use this skill, you take away some of the good player’s advantage and so the good player may be more likely to lose. Second, if all highly skilled players are almost equally good at thinking through possible moves and what differentiates these highly skilled players from each other are their skills at instantly recognizing situations (patterns of chess pieces and what those patterns mean), then, among highly skilled players, speeding up the chess game should not hurt the favored player.

Introduction: Beginning

Single fixation: one look

Serial search: looking through memory for something, one item at a time.

Reach asymptote… : peak

Algorithm: logical step-by-step process

Uniform: the same

Computational models: computer simulations

Introduction: Previous studies

Calibration: making sure that the measure is accurate

Introduction: A new blitz study

Variance in players’ skill: players differ (vary) in their skill at playing regular chess. One question in this study is how much of the differences between chess players’ skill can we predict just from knowing their performance in blitz chess. 

Curtailed: reduced.

Restricted: limited

Monotonic relation: consistent; straight line relationship

Deviate: differ

Equalization: make more similar (equal); erase differences between players of different abilities

 

Last paragraph

      We can be reasonably confident that an Australian with a rating of 1200 is better than an Australian with a rating of 1150. However, we would be less confident that an Australian with rating of 1200 is better than an American with a rating of 1150. 

 

Introduction: Analysis

First paragraph

     Suppose that

     Person A’s rating was 2000;

     Person A won 5 games in a tournament;

     there were 100 players in that tournament;

     49 of the 100 had rankings below 2000; and

     74 of the 100 won fewer than five games.

In that case, for the purpose of analysis, Person A would get a ranking score of 50 (50th percentile) and a player’s score of 75 (75th percentile).

Magnitude: size

 

Formula 1

Suppose someone played 10 games against 10 opponents and was expected (based on the person’s rankings relative to the person’s opponents) to win 6 games, draw 2, and lose 2. The person’s expected score would be 7 (one point for each of the 6 wins equals six, then add two halves for the two ties). If the person actually won 8 games and lost 2, the person’s “actual score against X” would be 8 (one point for each of the 8 wins). The person’s deviation against X would be positive .10 ((8-7)/10). The positive deviation score indicates the person did better in the blitz tournament than that person’s rankings would predict. If, on the other hand, the person did worse than that person’s rankings would predict, the person would get a negative deviation score. For example, if the person did not win or draw any of the 10 games, the person’s score would be negative .7 ((0-7)/10).

 

Formula 2

The formula for equalization factor (EF) is quite clever. The only way a player gets a high positive score if that player (a) does better against higher ranked players than would be expected and (b) does worse against lower ranked players than would be expected. For example, if a player’s deviation against stronger opponents is +.2 and that player’s deviation against weaker opponents is -.3, that player’s EF score would be +.5 (because +.2 - -.3 = +.5). In other words, to get a positive equalization score,  a player must achieve more upsets (have more ties or wins) against players ranked above that player than would be expected and  also, in turn, be upset (have more ties or losses) against players who are ranked below that player than would be expected. In short, the more positive EF, the more likely the higher ranked player will lose or tie. In other words, a positive EF score would indicate that favored players are more likely to be upset in blitz chess than in regular chess.

       If a player merely plays better than expected against everyone,  the player’s EF would be near zero. For example, if the player’s deviation against stronger opponents was +.5 and the player’s deviation against weaker opponents was also +.5, the player’s EF would be 0 (because .5 - .5). Similarly, if a player was having a bad tournament and did consistently worse than the player’s ranking would predict against both stronger and weaker opponents, that player’s EF score might be 0. For example, if the player’s deviation score against stronger opponents was -.7 and the player’s deviation score against weaker opponents was also -.7, the players  EF score would be 0 (because -.7 - -.7 is 0). Also note that if someone did as well against both weaker and stronger opponents as their rankings predicted, their EF score would also be 0 (because 0 – 0 = 0).

      How would a player get a negative EF score? A negative EF score would occur if a player was both (a) more likely than expected to win against lower-rated opponents and (b) less likely than expected to win or tie against higher-rated opponents. In short, the more negative the EF score, the more likely the higher ranked player is to win. In other words, a negative EF score would indicate that favored players are less likely to be upset in blitz chess than in regular chess.

     If rankings do a poorer job of predicting chess outcomes for lower skill players than for higher skill players, EF scores will be higher for lower skill players than for higher skill players. In the extreme case, if rankings were useless in predicting outcomes for lower skill players (e.g., players with ratings of 1200), the EF for players with ratings of 1200 would be +2 whereas, if the ratings were perfect predictors of outcomes for higher skill players (e.g., players with ratings of 2200 and above), the EF for players with ratings of 2200 and above would be –2.  The mathematics of the EF scores are summarized in the table below.

 

Record against stronger players

Record against weaker players

EF

Same as expected from rankings

Ex: Deviation  = 0

Same as expected from rankings

Ex: Deviation  = 0

Zero

 

 

Ex: EF   =  0 - 0

Worse than expected from rankings

Ex: Deviation  = -1

Worse than expected from rankings

Ex: Deviation  = -1

Zero

 

Ex: EF   =   -1 - -1=0

Better than expected from rankings

Ex: Deviation  = +1

Better than expected from rankings

Ex: Deviation  = +1

Zero

 

Ex: EF   =   +1 - +1 = 0

Better than expected from rankings

Ex: Deviation  = +1

Worse than expected from rankings

Ex: Deviation  = -1

Positive

 

Ex: EF   =   +1  - -1 = 2

Worse than expected from rankings

Ex: Deviation  = -1

Better than expected from rankings

Ex: Deviation  = +1

Negative

 

Ex: EF   =   -1  -+1 = -2

 

 

 

 

Results:

Dutch blitz sample

First paragraph

Note that the correlation between ratings and scores was very high. To get a better understanding of the last sentence, see pages 164-167 of Research design explained.

 

Second paragraph

Note that the higher a group’s rating, the less positive its EF.

Note that, as we discuss on pages168-173, people can use a variety of tests on correlational data and that different tests are often equivalent (although some may be less powerful than others). In this case, the author tests the significance of a correlation (as described on pages 168-169 of Research design explained) and then does an ANOVA (as described on pages 172-173 of Research design explained)—and both tests lead to the same conclusion.

Results:

American blitz sample

First paragraph

See comments for first two paragraphs under Results: Dutch blitz sample.

Results:

Australian blitz sample

See comments for first two paragraphs under Results: Dutch blitz sample.

Results:

Australian nonblitz sample

The author does a clever study showing that the results are not due to  some problem with the EF formula. The authors finds that EF does not correlate with ratings in the nonblitz sample but that EF does correlate with ratings in the blitz sample.

Discussion

 First paragraph

      Second sentence of first paragraph

                81% …. : is .9 squared (for more, see page 164 of Research design   explained)

     Next to last sentence of first paragraph

       reach a plateau: not be able to increase (in this case, experts with chess ratings above 2100 may all have about the same search abilities)

Second paragraph

      The author makes an argument for the external validity of the results.

Fourth paragraph

Apperception: relating what you see or sense to past experience; how something is interpreted—and the interpretation may be associations from memory are triggered by the event

Opening theory: strategies for moving pieces at the beginning of a game; chess experts can refer to a wide variety of sequences of moves at the start of a game by name (e.g., Bishop’s opening) and have memorized standard defenses against a wide variety of sets of beginning moves.  Note that a player who did not have such knowledge would be in trouble, especially in a blitz match.

 

 

 

 


Back to Chapter 6 Main Menu

Back to Featured Article Menu

Back to RDE Main Menu