Ecological fallacy
This article explains what an ecological fallacy is, and why it is important to know about this in the context of ILD research.
An ecological fallacy—also known as the generalization slip—stems from a mismatch between the kind of data your have and how you analyze these on the one hand, and how you interpret the results with respect to the research question you are trying to answer on the other hand. It is an error that occurs for instance when someone tries to make inferences about what happens within an individual based on results found at the level of the population.
In this article, you will find: 1) the ecological fallacy when you are looking at proportions; 2) the ecological fallacy when you are looking at correlations; 3) other examples from the psychological literature where ecological fallacies appear.
1 Ecological fallacy when dealing with proportions
An ecological fallacy occurs when the proportion of individuals in a population with a certain characteristic is interpreted as the proportion of times an individual exhibits that characteristic. For instance, if you find that 20% of adults are smokers, it would be an ecological fallacy to assume that a person from this population smokes 20% of the time.
While for more stable features (e.g., biological sex, ethnicity, educational degree), it may seem rather obvious that such generalizations from the level of the population to the level of the individual is a mistake, it is equally problematic when you focus on features that vary within persons over time. For instance, if 40% of students report feeling stressed on a given day, this does not mean that each student spends 40% of their day feeling stressed.
Susan studies the prevalence of panic attacks among teenagers. To this end, she asks a sample of teenagers whether or not they had a panic attack in the past two weeks; about 1% of the participants indicates to have had such an experience.
Susan wonders whether this tells her anything meaningful about how frequently an individual experiences panic attacks: Can the result she obtained be translated to the individual level? When thinking about this, Susan realizes that she should not claim that the probability of having a panic attack for any given teenager during any two week period is 0.01. There may be large individual differences, where some individuals experience frequent attacks, while the majority never experiences a panic attack.
Such generalizations from the population to the individual are possible if specific conditions, known as ergodicity, hold. However, these conditions are very stringent, and ergodicity is unlikely to be found in psychology. This means that, in general, you should avoid interpreting proportions in the population as informative about proportions within a person.
2 Ecological fallacy when dealing with correlations
In the context of ILD research, concerns about the ecological fallacy are frequently raised in the context of correlations. An often used illustration of the problem is the typing example (Boker & Martin, 2018; Hamaker, 2012).
Assume you have a sample of people, and you let them type for a minute; afterwards, you determine for each participant how many words they typed, and the percentage of typos they made. When you consider the relation between these variables, you see there is a negative correlation as shown in Figure 1: Individuals who typed faster made fewer mistakes, whereas individuals who typed slower made more mistakes.
The ecological fallacy occurs if you try to generalize this group level result to the level of the individual, assuming a particular person makes fewer mistakes when typing faster and more mistakes when typing slower.
In fact, we can be fairly certain that at the level of the individual, we will see a positive relation: When the person types faster, this person makes more mistakes, and when the person types slower, the person makes fewer mistakes. This is illustrated in Figure 2, which shows the repeated measures of a single person in red, and their average in blue.
The reason for the difference in results at the group level and the individual level, is that the group level result is based on the [inter-individual variation]; the latter consists of a combination of [intra-individual variation], and variation due to stable differences between individuals. This is the foundation of the [within-between problem].
Chris studies the relation between parental support and self-esteem of young adults by obtaining a large sample of first-year students and measuring how much support they received from their parent(s) during the past month and their self-esteem. She finds that the cross-sectional correlation is close to zero, and concludes that, if we are interested in developing an intervention for students with low self-esteem, we do not need to consider the support provided by parents.
Chen and Agda point out that Chris’ conclusion is premature: While the cross-sectional correlation computed by Chris may be close to zero, the within-person correlation may be positive, meaning that students may experience more self-esteem at moments when they receive support from their parents.
The only case where such generalizations from the population to the individual and vice versa can be safely made, is when ergodicity holds. As the conditions for ergodicity are extremely difficult to be met, we should in general assume that ergodicity is absent in psychology; this implies that generalizations across levels can only be made when we have established that the result of interest is identical at different levels.
3 Examples of ecological fallacy in psychology
Ecological fallacies are a concern not just when considering descriptive statistics like proportions and correlations, but also in the interpretation of results from more complex analyses. Below, two examples from the psychological literature are presented.
3.1 Five factor model of personality
The five factor model of personality is a well-known model that can be used to compare people on five distinct personality dimensions. The five factor structure has been obtained in cross-sectional research, when factor analyzing inter-individual variation. Important proponents of the five factor model of personality have argued that: “Personality processes, by definition, involve some change in thoughts, feelings and actions of an individual; all these intra-individual changes seem to be mirrored by interindividual differences in characteristic ways of thinking, feeling and acting.” (p.199, McCrae & John, 1992).
This is an example of the ecological fallacy: The researchers assume that the results obtained by analyzing inter-individual variation must reflect the patterns that are present in intra-individual variation. If this were true, it would imply that by studying inter-individual differences, we can learn about processes that operate intra-individually. But if this is not (exactly) true, it is a generalization slip.
To see whether the assumption expressed by McCrae & John (1992) is correct, Ellen uses daily diary data that were obtained using a five factor questionnaire. She analyzes the data for each person separately, using an exploratory factor analysis.
The results show that most people are characterized by fewer than five factors; moreover, the nature of the factor varies across individuals. Hence, Ellen concludes that the five factor model of personality cannot be generalized to the individual level.
This example is based on Hamaker et al. (2005).
3.2 Switching between plural and singular
Another-somewhat more subtle-way in which ecological fallacies may occur is when researchers obtain results at the group level, but describe these using singular terms (e.g., referring to an individual). Valsiner (1986) used excerpts from various studies showing how easily authors switch from talking about the results in plural form (e.g., adolescents, fathers, and mothers), to using singular terminology (i.e., adolescent, father, and mother).
Jaan considers excerpts from various studies, and sees that many authors easily switch from talking about the results in plural form (e.g., adolescents, fathers, and mothers), to using singular form (i.e., adolescent, father, and mother). For instance, in one of the papers he considers, the researchers first indicated that adolescents who argued a lot with their mothers, tend to spend more time with their fathers; this is all in plural form, which matches the level at which the results were obtained.
But when the researchers try to provide an explanation for these findings, they switched to singular, saying that this pattern may result from the adolescents (still plural) finding the relationship with their mother (singular) stressful, and seeking out the company of the father (singular) as a kind of emotional reserve.
This example is based on the work of Valsiner (1986).
Michael obtained cross-sectional data from a large sample of couples and finds that couples who report more frequent arguments also tend to report higher levels of emotional closeness. This is a relation that pertains to the group level.
However, when writing up the results, Michael states that when a spouse argues with their partner, it helps them feel closer, and he hypothesizes that having spousal arguments allows for emotional expression and resolution.
In switching from plural to singular (i.e., “a spouse”), Michael is interpreting the group level result as pertaining to the individual level.
4 Think more about
It seems that the risk of falling prey to the ecological fallacy is especially prominent when cross-sectional research is done resulting in the estimation of population characteristics, while the researcher is actually interested in a research question about processes that operate within individuals. The ecological fallacy can also involve interpreting results from a single individual as being informative about an entire population, but this version of the error appears relatively rare in psychology and related fields.
While the differences between results obtained with cross-sectional research and results obtained with research based on N=1 time series data can be attributed to a difference in the types of variance that the results are based on, there are additional factors that are likely to play a role. Cross-sectional research tends to be based on using (much) longer time frames for the measurements than what is common in N=1 and N>1 ILD research (e.g., measures are obtained with reference to the past month versus the past day). Furthermore, when you want to compare results across cross-sectional and N=1 or N>1 ILD research, you should also pay attention to whether the results are based on contemporaneous or lagged relations.
All these choices may affect the results, and results should thus be interpreted in light of such choices. In general, it is important to be aware of the level to which your research question pertains, and to make sure that this corresponds to the level to which your analysis results pertain.
5 Takeaway
The ecological fallacy occurs when you generalize results from the group level to the individual or vice versa; such cross-level generalizations can only be safely made when ergodicity holds. When the process that is studied is non-ergodic, such cross-level generalization are a generalization slip.
Ecological fallacies may occur somewhat implicit when researchers are describing their results and they switch from plural to singular terminology; the latter suggests that the results pertain to a process that is situated within a person. Moreover,when group-level results are interpreted in terms of underlying mechanisms, ecological fallacies are likely to occur; such speculations are quite common in discussion sections of papers, but often involve thinking about the results as pertaining to a process that takes place within a person over time.
To prevent ecological fallacies in your work, it is important to be aware of the level to which your analysis results pertain, and to carefully consider the language that you are using to describe and interpret your results. While you hypothesize about the reason for certain patterns in your data, it may be informative to think of alternative explanations. For instance, you can think about how the [within/between problem] may play a role in the results you obtained, and when speculating about causality, you may think of possible confounders that should be accounted for.
6 Further reading
We have collected various topics for you to read more about below.
- Ergodicity in psychology
- Intra-individual versus inter-individual correlation
- [Intra-individual versus inter-individual research]
- [The within-between problem]
Acknowledgments
This work was supported by the European Research Council (ERC) Consolidator Grant awarded to E. L. Hamaker (ERC-2019-COG-865468).
References
Citation
@article{hamaker2025,
author = {Hamaker, Ellen L. and Hoekstra, Ria H. A.},
title = {Ecological Fallacy},
journal = {MATILDA},
number = {2025-05-23},
date = {2025-05-23},
url = {https://matilda.fss.uu.nl/articles/ecological-fallacy.html},
langid = {en}
}