Title |
Statistical Conclusion Validity: Some Common Threats and Simple Remedies
|
---|---|
Published in |
Frontiers in Psychology, January 2012
|
DOI | 10.3389/fpsyg.2012.00325 |
Pubmed ID | |
Authors |
Miguel A. García-Pérez |
Abstract |
The ultimate goal of research is to produce dependable knowledge or to provide the evidence that may guide practical decisions. Statistical conclusion validity (SCV) holds when the conclusions of a research study are founded on an adequate analysis of the data, generally meaning that adequate statistical methods are used whose small-sample behavior is accurate, besides being logically capable of providing an answer to the research question. Compared to the three other traditional aspects of research validity (external validity, internal validity, and construct validity), interest in SCV has recently grown on evidence that inadequate data analyses are sometimes carried out which yield conclusions that a proper analysis of the data would not have supported. This paper discusses evidence of three common threats to SCV that arise from widespread recommendations or practices in data analysis, namely, the use of repeated testing and optional stopping without control of Type-I error rates, the recommendation to check the assumptions of statistical tests, and the use of regression whenever a bivariate relation or the equivalence between two variables is studied. For each of these threats, examples are presented and alternative practices that safeguard SCV are discussed. Educational and editorial changes that may improve the SCV of published research are also discussed. |
X Demographics
As of 1 July 2024, you may notice a temporary increase in the numbers of X profiles with Unknown location. Click here to learn more.
Geographical breakdown
Country | Count | As % |
---|---|---|
New Zealand | 1 | 14% |
United States | 1 | 14% |
Chile | 1 | 14% |
Switzerland | 1 | 14% |
Canada | 1 | 14% |
Unknown | 2 | 29% |
Demographic breakdown
Type | Count | As % |
---|---|---|
Scientists | 4 | 57% |
Members of the public | 2 | 29% |
Practitioners (doctors, other healthcare professionals) | 1 | 14% |
Mendeley readers
Geographical breakdown
Country | Count | As % |
---|---|---|
Germany | 2 | 1% |
Canada | 2 | 1% |
United States | 2 | 1% |
Netherlands | 1 | <1% |
United Kingdom | 1 | <1% |
Italy | 1 | <1% |
Spain | 1 | <1% |
Brazil | 1 | <1% |
Unknown | 184 | 94% |
Demographic breakdown
Readers by professional status | Count | As % |
---|---|---|
Student > Ph. D. Student | 35 | 18% |
Student > Master | 25 | 13% |
Student > Doctoral Student | 23 | 12% |
Student > Bachelor | 17 | 9% |
Researcher | 12 | 6% |
Other | 33 | 17% |
Unknown | 50 | 26% |
Readers by discipline | Count | As % |
---|---|---|
Psychology | 41 | 21% |
Social Sciences | 18 | 9% |
Medicine and Dentistry | 13 | 7% |
Business, Management and Accounting | 12 | 6% |
Computer Science | 9 | 5% |
Other | 43 | 22% |
Unknown | 59 | 30% |