Beyond completion rates: what an ant mill teaches L&D

Part of Measuring behaviour change.

A circle that looked like work

In 1921 the naturalist William Beebe published an account of something he had watched at his research station in British Guiana [1]. A column of army ants, six lines wide in places and most of them carrying eggs or larvae, had looped back on itself. Each ant followed the one in front. The ends of the column met, and the column became a circle.

Beebe measured it. By his account the circle was about 1,200 feet round, which is roughly 365 metres. By Beebe's reckoning, a single ant took about two and a half hours to complete a lap, and the ants he watched had gone round perhaps fifteen times. They kept moving through the day and into the night. Eventually a few stragglers wandered off the circle, others followed their scent, and the column broke away [1].

Nobody in that circle was lazy. Every ant was doing exactly what it was built to do: follow the trail. The trail just didn't go anywhere.

The L&D version of the loop

Most of us have seen a programme run like that. Completions go up. Satisfaction scores go up. The dashboard is green. Next year's budget is renewed on the strength of it. And behaviour on the job is much the same as before.

That isn't a criticism of the people running it. Completion and satisfaction are easy to collect, they arrive quickly, and the systems we use are built to report them. They are the trail. The problem is what they can't show: whether anyone does anything differently afterwards.

Researchers have a name for that gap. In a 1988 review, Baldwin and Ford defined transfer of training as the degree to which people effectively apply what they learned in training to the job, and keep applying it [2]. They described transfer as a long-standing problem then, and it hasn't gone away. In a later study, Saks and Belcourt report that training professionals from 150 organisations estimated that 62%, 44% and 34% of employees apply training material on the job immediately, six months and one year after training [3]. Those are professionals' estimates, not measurements, which is part of the point: it is rarely measured directly.

What a completion rate can and can't tell you

A completion rate answers one question well: did people finish the course? That matters for compliance records and for planning. It does not answer:

• Did people use the skill in the following weeks?
• Did their manager see a difference?
• Did it last beyond the first month?

A satisfaction score has a similar limit. It tells you how the session felt. It can't tell you whether the session changed anything, and asking people to rate their own learning brings its own biases, such as wanting to look good or to appear consistent [4].

None of this makes completion data useless. It makes it a record of activity. The mistake is reading it as a result.

Breaking out of the circle

Beebe's ants didn't stop because someone explained the problem to them. A few broke away, and the rest followed a new scent. For an L&D team, the new scent is a different question asked at a different time.

Instead of "did they complete it?", ask "what are they doing differently, and who has seen it?" Instead of asking on day one, ask again after 30, 60 and 90 days. Pick two or three behaviours that matter for the programme, and decide in advance who can observe them: the learner, their manager, a peer, or a record of the work itself.

That is harder than counting completions. It is also a much better way to find out whether the circle is going anywhere.

What this means for your programmes

• Keep completion data, but label it as activity in every report.
• For one programme this quarter, name the two or three behaviours it is meant to change.
• Decide who could observe each one, and when you'll ask.
• Report what you find, including "we don't know yet".

This is the ground Julian covers in his talk The Ant Mill: why hard-working L&D teams march in circles.

General information on training design, not professional advice for your organisation.