fact finder
A survey about young people’s image of men caused a lot of excitement over the weekend. Accordingly, one in three thinks it is okay to use violence against women. But there is a lot of criticism of the methodology of the survey.
Misogynous role models, rejection of homosexuals and acceptance of domestic violence: For many young men in Germany, this is apparently part of their image of masculinity – at least that’s what they think Results of the survey “The Conflict of Masculinity” by the non-governmental organization Plan International vicinity. Numerous media reported about it, including the daily News and tagesschau.de. But does the online survey even allow for these conclusions?
Probably not, says Katharina Schüller, board member of the German Statistical Society. “I find it amazing that so many media reported uncritically about this survey without asking first,” she says. “I’m actually hoping for a higher level of competence in handling data in the editorial offices.” According to Schüller, there are a few points of criticism that reduce the validity of the survey.
Quotas “based on official statistics”
For the survey, 1,000 men and 1,000 women between the ages of 18 and 35 were asked about their views on ten aspects of masculinity using an online questionnaire. According to market researcher Eckhard Preis, who designed the survey for Plan, women and men each received a different questionnaire. Homosexual men received an adapted questionnaire in which the word partner was replaced by partner. According to Preis, there were a total of 44 statements that the respondents were asked to rate on a four-stage scale (strongly disagree, tend to disagree, tend to agree, totally agree).
This was then summarized in the second step for the survey in approval or rejection, without the individual differentiations. “For us, the only thing that counted was the approval, not the degree of approval,” says Preis.
Plan’s report also states that “on the basis of official statistics, quota specifications were made” in order to “ensure the representativeness of the sample”. “We formed three age groups and we combined that with low education and higher education,” says Preis. “So basically we have six groups.” There was also a regional quota to interview participants from different areas. For this purpose, Germany was divided into four regions.
“Simply wrong from a statistical point of view”
Graduate statistician Schüller complains above all that there is a lack of information on how the sample for the survey was recruited. “The quota specifications are too short-sighted,” says Schüller. In the case of online surveys, a fundamental distortion in the data is very likely because of the panels. A panel designates the group of people that is used for surveys.
In such online panels there are often more people who need to communicate or are curious and also have a certain affinity for technology. The authors of the survey themselves point this out. “These are not necessarily things that you correct, for example by weighting school education, age or region,” says Schüller. “That’s just wrong from a statistical point of view.”
Recruiting participants promotes homogeneity
According to Schüller, it should have been checked beforehand what bias (German: distortion) there is in the entirety of the panel in order to make estimates of how much this distortion affects the data. In addition, those surveyed by the market research institute are not only paid for taking part, but also for recruiting new participants. “That promotes homogeneity rather than heterogeneity in this panel,” says Schüller.
According to market researcher Preis, the challenge is always to prevent all possible distortions, because: “Overall, certain people are more likely to take part in surveys than others.” However, this applies to any type of survey, whether online or, for example, by telephone. “In market research, you depend on people who are somehow willing to provide information.”
Schüller also criticized the fact that the answer options were summarized in the result, which is methodologically not entirely clean. There are other tools in social research to deal with tendencies, for example, that someone never selects full agree or disagree on a scale. Instead of summarizing the answer options, it makes more sense to identify and eliminate people with such answer tendencies.
Majority did not name any role models
According to Price, the question of a role model was open-ended. The Plan report states that the men surveyed most frequently named soccer player Cristiano Ronaldo, entrepreneur Elon Musk and ex-martial artist and influencer Andrew Tate as prominent role models. “The latter is best known for its misogynistic and violent TikTok videos,” the report reads.
However, Price points out that most responded that they have no role model. Only 23 percent named a role model at all, most of them their own father. Due to the high variance in mentions, the mentions of Tate or Musk are in the low double-digit range – with 947 valid completed questionnaires from men.
Schüller would like more transparency
According to the authors, a total of 104 people filled out the questionnaires incorrectly. Here, too, Schüller would like more transparency as to what could have been the reason. “I wonder whether the questionnaire was validated beforehand.” In addition, the number of participants who did not even answer the questionnaire, the so-called non-responders, is also interesting. This rate, in turn, could indicate a selective sample, for example because the group of responders tended to give different answers than the non-responders would have given.
“The so-called non-response bias is a critical issue that should not be neglected,” says Schüller. “And a clean study would document all of that.” Price cannot say how many non-responders there were. “That’s a matter for the panel operator, who is responsible for recruiting people for us. And of course he invites more people than take part in the end. But I can’t say what the relationship is like.”
In order to test the questionnaire in advance, it is also processed internally by impartial people, says Preis. “In this way we track down possible problems of understanding and can further optimize the questionnaire texts.” Price also says that there are differences to scientific publications. “What we do is aimed at the public and not at a specialist audience. We also have shorter time schedules and cannot fall back on any state research funds.”
By following industry standards and the guidelines of the industry associations, as well as measures such as quoting the sample to ensure representativeness, he and his colleagues would also be able to make reliable statements, says Preis.
“Men tend to adopt conservative role models”
From the point of view of Nicole Bögelein from the Institute for Criminology at the University of Cologne, many of the findings of the Plan survey are not surprising. “What you see in studies is that even today, biological sex is a key factor in how gender roles are viewed.” Men tended to behave much more in conformity with their role and tried to serve the traditional image of men. “Men seem to lean more toward a conservative role model and women toward an egalitarian one.”
In this context, researchers speak of hegemonic masculinity, says Bögelein. “That’s the cultural ideal that radiates male dominance, a complementary hierarchical division of labor and ultimately also a dominant heterosexuality. So a clear orientation towards the fact that it should be very traditional, just like it’s always been.”
Intimate partner violence a male problem
The image of hegemonic masculinity may make acts of violence more attractive to men than to women. “In the bright field statistics, we see very clearly that women are much more frequently affected by intimate partner violence than men,” says Bögelein. “And we see that when women are victims of violence, it is mostly by their partner or ex-partner.” According to the police crime statistics there were around 140,000 cases of intimate partner violence in Germany in 2021. 80 percent of the victims were female, almost 79 percent of the perpetrators male.
According to Bögelein, factors such as education or origin do not play a decisive role in the male perpetrators. “There is violence in partnerships in all sections of the population.”
It is also not surprising that many young men are hostile towards homosexuals, says Bögelein. In a Study by the Federal Anti-Discrimination Agency 44.3 percent of the male respondents stated that they find it rather uncomfortable to very uncomfortable when two men show affection in public. In the Plan survey, it was 48 percent. Overall, according to the anti-discrimination agency’s study, men were twice as likely to agree with anti-queer statements as women.
family ministry want gender roles break open
According to a spokeswoman for the Federal Ministry for Family Affairs, some of the results of the Plan survey also contradict the findings from other surveys. In the Study “Men’s Perspectives” For example, only 23 percent of those questioned were in favor of a partially traditional partnership and only seven percent in favor of a consistently traditional partnership – at Plan it was about half.
However, conventional models of masculinity would still be in the majority. Therefore, the breaking of gender roles, protection against violence and the strengthening of caring men and fathers are important political goals.