Interrogate Interrogate each of your articles using the four big validities. For each article, work through the four big validities in turn, indicating whether the article does a good or bad job on each front. As you write, keep in mind that you are demonstrating your mastery of this material! Show me that you know how to ask questions about each of the four validities, show me that you know what the answers to these questions mean. Finally, show me that you understand what it means to prioritize validities as you interrogate a study. Details: Construct validity: Evaluate the measures and manipulations Restate each variable and how it was operationalized. (œThey operationalized ___ by doing _____). After you describe each measure or manipulation, indicate any reliability and validity information provided in the article (refer to figure 5.7 in your textbook). Do they give Cronbach’s alphas; if so, what are they? Are they good or bad? Do they give interrater reliabilities; if so, what are they? Do they give any evidence for predictive or concurrent validity, perhaps to referring to how the measure has been used in past research? Using this information, give your assessment”is each measure reliable and valid? If such information is not included, then evaluate that”what would you like to see? For manipulated variables, are the manipulations construct valid, in your opinion? (see Chapter 10). If it’s manipulated variable, do they do any manipulation checks to show you that their manipulation did what they intended it to do? Does the manipulation have face validity? Statistical validity: Evaluate how strongly the results support their argument. For each major result, evaluate its effect size to the best of your ability. State the effect size if it’s given in terms you understand (e.g., r or d). If it’s a beta in a regression table, how big is the beta for the main variable of interest, compared to the others on the table? Are the major effects statistically significant? What does this mean? As you evaluate statistical validity, discuss how strongly and how well the results pattern supported the authors’ hypotheses. See if you can evaluate how strong the result is in some œreal world terms, like an increase in IQ points or a weight loss of some number of pounds, or a reaction time in seconds. Internal validity: Can the study support a causal claim? Does it intend to? First, state what specific causal claim the researchers wish to make (if it’s an experiment) or might wish to make (if it’s a correlational study). If it’s an experiment, apply the causal rules, including the internal validity threats in Table 11.1. Do any apply to your article? If not, what is your overall evaluation? Can they make a causal statement? If it’s a multiple regression design, apply the causal rules, and for internal validity, review what variables were controlled for in the design. Can you think of any other third variables? What is your overall evaluation? Can they make a causal statement? Before criticizing for third variables, remember that a confound is a problem with systematic variance (a true confound) and not unsystematic variance (an obscuring factor). So if you think that some people in the study might have been in a bad mood, that will not be a confound unless people in only one team were in a bad mood, and the others were not. External validity: To whom or to what other contexts can the results be generalized? External validity addresses two issues : to whom (if anyone) can the results be generalized , and to what other situations or settings? Remember that it’s how a sample is drawn, not how many people are in it, that determines external validity. Consult Figure 7.5 for a reminder. Also, remember that in many studies, external validity is not the first priority. Your evaluation should acknowledge this, and explain why.