For years, headlines have pondered whether Michael Phelps is the world’s greatest Olympian. News accounts of the American swimmer’s prowess often make the claim in passing. Since the 2012 London Olympics, Phelps has held the most medals of any athlete in the modern Olympics. In Rio he added another six medals to his total, five of them gold.

He has 23 Olympic gold medals. But do medals make the man?

Phelps’ medal tally

On a simple medal count, Michael Phelps is ahead of every other person who’s competed in a modern Olympics. He didn’t win a single medal in 2000, when he was 15, but he more than made up for that in every subsequent Games. He has won 23 gold medals, three silver and two bronze. If we consider just gold medals, Phelps’ closest competition is the Soviet Union’s Larissa Latynina, a gymnast who during her Olympic career (1954 to 1964) secured nine gold medals and 18 medals overall (she held the record as the most awarded modern Olympian till Phelps smashed her tally in 2012).

That means when it comes to gold medals, Phelps has well over twice as many as his nearest competitor. Only 38 countries in the world have won more gold medals over their history of competing in the Olympics than Phelps has all on his own.

So give him the crown?

On a simple medal tally, Phelps reigns supreme. But is he the greatest? We turned to an Olympic historian, who says that is a rather complicated question.

“There’s no question about his greatness,” says Dr Ian Jobling of the University of Queensland’s Centre of Olympic Studies. “But the greatest? Well, as soon as you say that you come into so many other variables.” He doesn’t begrudge commentators who make the claim about Phelps, given its inherent newsworthiness, but he can quickly reel off a number of factors that make things rather complex.

The benefits of professionalism, and of long, continuous training

The modern Olympics changed markedly in the late 1970s. Before then, only “amateurs” — those who didn’t earn an income from their sport — were allowed to compete.

This had big consequences for the careers of many athletes. Take Frank Beaurepaire, an Australian long-distance swimmer who won three silver and three bronze medals between the 1908 and 1920 Olympics, and set 15 world records.

At 14, he won races at the Victorian and Australian swimming championships, earning himself a place on the Olympic team in the 1908 London Olympics. He won a silver in the 400m freestyle and a bronze in the 1500m freestyle. By 1910, he’d broken six world records. By 1911 though, he’d taken a job as a swimming instructor with the Victorian Department of Education. This meant he was ineligible from the 1912 Olympics, as he was deemed a swimming “professional”. He served in the ADF in World War I and wouldn’t return to the Olympics until 1920, when he won a further two silver and two bronze.

Another factor of this amateur rule is that those who competed in earlier Olympics weren’t able, barring unusual independent wealth, to train full time. Independently wealthy athletes had the serious advantage of not having to pay the bills.

One cannot fairly compare the dominance the world’s best modern athletes with sporting prowess achieved in the stop-start careers of the athletes of yesteryear, Jobling says.

Phelps has also benefited from a long, uninterrupted run — a path he’s shared with others in recent decades. But most athletes in history have not had it so smooth. During the height of the Cold War, there were boycotts in 1980 and 1984. The Olympics didn’t happen during both world wars.

The sheer number of swimming events

But Phelps’ medal tally isn’t just phenomenal compared to Olympic history — it’s phenomenal compared to athletes today. But comparing Phelps against athletes of his own generation is also a fraught process.

If you look at a list of the 20 athletes who’ve amassed the greatest number of Olympic medals, 14 were either swimmers or gymnasts.

There’s a good reason for this — there are simply more Olympic events in which a swimmer or gymnast can win a medal, and the skills used to win one event are usually transferable to other events. A champion swimmer will excel at most of the strokes, and there’s a separate medal for each one. In swimming, there are 16 races at which a male swimmer has a shot at a medal. In gymnastics, there are eight events for men and six for women.

Compare that to say, archery. There are four medals given out for archery every year — one to individual women’s and men’s competitions, and another women’s and men’s medals for the team competitions. So the world’s best archer can only win, at the most, two medals an Olympics (and one is reliant on belonging to a country with other good archers to dominate the team event). There are only two medals up for grabs in volleyball — a women’s and a men’s.

The sport allocated the greatest number of medals in the Olympics is athletics (23 for men and the same number for women), but unlike with swimming, it’s unlikely a sprinter would excel at hammer throw.

Phelps swam in six events at Rio, earning five gold medals and one silver. For comparison, Usain Bolt, the world’s fastest man, is only competing in three events these Olympics — the 100m sprint, 200m sprint and 100m relay.

There’s another factor to this worth considering, and that’s that only in some individual sports are athletes’ results the sole result of their own effort. For many team sports, or even knock-out sports like judo, this isn’t the case. A champion basketballer is only as good as his opponent’s defence allows him or her to be. Dominance in a speed race is a factor only of individual effort and skill. A sprinter cannot be slowed down by the actions of the other competitors. Dominance in many other sports is impacted both by a team’s athletic prowess and the faults of the opposition. Both types of competitions give out medals, but the universal metric obscures the fundamentally different nature of competition in many Olympic sports. Medals are all the same — but what it takes to win them isn’t.

The verdict: it’s impossible to say

Michael Phelps is a phenomenal athlete. But a number of factors mean he’s been able to stack up the medals with less difficulty than many of his predecessors. It’s a foolhardy exercise to compare athletes across different sports at the Olympics, and even more foolhardy to compare across different eras.