Autopilot on, brain off? What we can learn from aviation about automation

Ever since the launch of AI assistants such as ChatGPT and Claude, numerous companies at various levels have been considering automating more of their operations – from emails to decisions and even entire processes. The aim is to make workflows more efficient – and, as an added bonus, save on human resources. A win-win? Not so fast, because EU law requires that a human must remain ‘in the loop’ when it comes to significant decisions. But why is that, actually? And is it really enough simply to throw any person into a process anywhere? It’s not quite that simple. If you want to automate effectively, you need more than just a human equivalent of a scarecrow. To understand what’s required, it’s worth looking up. After all, in aviation, automation has been an integral part of everyday practice for decades. As a best-practice example, the sector demonstrates what really matters: whether it’s clear communication or the continuous development of training programmes. In this blog post, we’ll look at what we can learn from aviation about the harmonious interaction between humans and machines for other sectors that are only just beginning to grapple with automation.

Flying is the safest form of civil transport there is. This becomes apparent right from the certification stage: the acceptable probability of a ‘catastrophic failure’ – that is, an acute risk of a crash – is 10⁻⁹ (EASA, 2003, CS-25.1309); this equates to one such failure in every billion flight hours. The actual accident rate in 2025 stood at 1.32 per million flights, corresponding to 49 accidents with a total of 394 fatalities worldwide (IATA, 2026).¹ These figures are no less saddening for that, but the 2,832 road deaths in the same year in Germany alone certainly put them into perspective (ADAC, 2026). And yet, around 3 per cent of people suffer from a fear of flying (Grimholt et al., 2019).² Do we not trust the pilots to get us safely from A to B? Or do we not trust the aeroplane to do what it is supposed to do? Doesn’t the AI autopilot do all that anyway these days? Not quite.

Automation in the air

The role of technology in assisting humans has been considered in aviation for a long time (Bainbridge, 1983). Landing, for example, has been automatable since the 1950s. By way of comparison: back then, computers were still larger than people and could not even remotely do what even the simplest, tiniest computers are capable of today. A great deal has changed technologically since then, yet flying still involves a surprising amount of manual work, as you can see if you take a look inside the cockpit, which is studded with dozens of buttons.

Whilst there are many different support systems in place, and under normal conditions no pilot today needs to actively ensure that the aircraft remains stable at cruising altitude or flies in the correct direction, civil aviation is still a long way from full automation. The same applies to the implementation of modern AI systems on board. However, automation and the risks associated with it already begin with much simpler systems. As a sector where errors can cost human lives, the aviation industry has always taken a critical approach to the issue of safety. For example, the take-off of every commercial aircraft is still carried out manually; this is because there are aspects that we cannot (yet) hand over to machines – situations where one or more people are needed and complex decisions must be made. 

Who is the decision-maker in the sky?

Decisions in the air are never, by principle, made alone. There are at least two people in the cockpit: the captain and the first officer, also known as the co-pilot. Whilst the captain has the final say, up to that point both (with a few exceptions) have equal authority to make decisions. Who is responsible for what is not determined by rank, but by a division of roles agreed upon before each individual flight – between the pilot flying and the pilot monitoring. This ensures that both are capable of performing both roles. However, the two are not alone up there. To ensure accident-free coexistence with other aircraft, there are also air traffic controllers in large ground-based control centres who ensure sufficient separation between all aircraft by issuing instructions and granting clearances.

Autopilot on, brain off?

So what does the autopilot actually do? Functionally, there are (roughly speaking) two main functions (plus backups): ‘steering’ and ‘throttling’. To steer, the computer simply controls various flaps around the aircraft, which are positioned to face the wind in specific ways so that the aircraft rotates, banks or pitches along the three axes as required to get back on course. No magic, just physics. It’s not artificial intelligence, but rigidly programmed algorithms that follow clear rules and shut themselves down if data is missing (a faulty sensor) or something is wrong (too windy). This doesn’t reinvent flying, but predictably takes the strain off the crew and makes the flight more efficient. 

And what is left for the pilot(s) to do? Even when the autopilot is engaged, pilots have their hands full. They monitor its ‘actions’, check fuel consumption, review alternative airports, keep an eye on the weather and maintain contact with air traffic control on the ground. This is anything but trivial and, despite the autopilot, remains a demanding task. When the aircraft is on final approach, the autopilot is usually switched off. This is because an automatic landing requires such extensive preparations that, paradoxically, it is ‘easier’ to land manually when visibility is good.

Human in the loop, all good?

The main cause of aviation accidents remains, in most cases, human error (IATA, 2026). The machine performs its tasks reliably, but only those for which it was built. Everything else must necessarily be handled by people. This makes it all the more important that these people are empowered to fulfil this role. Otherwise, they would merely serve as scapegoats if something goes wrong (Crootof et al., 2023). Yet this is precisely what can happen when systems are designed in such a way that, in the end, it is always the operator who is to blame and never the technology. True to the saying: ‘The problem is sitting in front of the computer.’ This is not merely a legally relevant issue. Above all, it is about how we design processes. In the following, we use three lessons learned to illustrate, in part, exactly how the aviation industry deals with this and what other sectors can learn from it.

Learning 1: More than the sum of its parts

In air traffic control centres, air traffic controllers work together in large halls, each responsible for specific airspaces. Whilst the handover of aircraft between airspaces is generally carried out strictly in accordance with flight plans, changes often occur due to weather or delays. In such cases, if there is any doubt, arrangements must be made to divert an aircraft from one sector to another. In the past, this was done by talking or via a phone call. Today, a system handles this with a simple yes-or-no pop-up window. Quick, efficient, clear – but too simple? Replacing human interaction with a yes-no window ignores tone of voice, choice of words and pitch (Rieth, 2022). These are all aspects that help to contextualise a statement: Does it need to be done quickly? Is it a ‘could you’ or a ‘you must’? Is there an immediate danger? None of this is conveyed by this pop-up window. It is not without reason that we frequently use emojis in chats; because they help to contextualise what we say. It is therefore not surprising that, despite the system, air traffic controllers still pick up the phone or call out to the neighbouring desk when it really matters. That is not the only reason why they still sit together in the same room to this day.

This example highlights a fundamental problem: automation is intended to ease the burden on people by having a system take over one or even several tasks at once. However, by bundling these tasks together, the individual steps do not simply vanish into thin air; instead, they form a new, more complex whole. If I am the ‘human-in-the-loop’ overseeing this process, I must – in order to fulfil this role – be able to understand what is happening in detail, or at least notice when something goes wrong. To this end, the system must, first and foremost, inform me in a comprehensible manner exactly what is happening at any given moment.

It is therefore not enough simply to add up the strengths and weaknesses of humans and machines. It is precisely this hybrid collaboration that gives rise to new, difficult-to-predict problems of its own; above all, misunderstandings. The supposed simplification can make processes more complicated and thus harder to control. Machines are, and will remain, tools. They should therefore be designed and used accordingly, to reduce blind spots rather than increase them. Automated understanding is more important than automated decision-making. 

Learning 2: Check, check

One of the most important principles in aviation is redundancy. There are two turbines, half a dozen on-board computers from different manufacturers and three separate hydraulic systems for the flaps. Theoretically, only one of each would be needed. Yet redundancy is better not only from a technical point of view, but also in terms of human interaction. In our interviews, communication in the cockpit was compared to a theatre play. From reconciling fuel calculations, through the take-off procedure, to emergency situations such as a cabin fire, there are detailed checklists and scripts for everything that must be strictly followed. The overarching practice here is the ‘check’ principle: one person says something, the other confirms it; sometimes by repeating it word for word if the specific content is particularly important. Below 10,000 feet, there is even a ban on small talk (EU, 2015); this means that only matters directly related to the flight (or aircraft) may be discussed. There is no place for weekend plans here. The rules are so clear that even deviating from them can in itself be a statement. If a step is missing from the process, the procedure is aborted in case of doubt.

The medical profession has adopted this practice from the aviation industry: if a swab is missing, the patient is not sutured (Haynes et al., 2009). Particularly when automation means that more and more responsibilities are concentrated on fewer and fewer people, crystal-clear communication is essential to ensure that no one loses track of the situation. And as long as one person remains essential, two are needed.

Learning 3: New, old skills

Deeply ingrained in the aviation industry is the determination to learn from every mistake, particularly accidents. These are systematically analysed, and the findings and recommendations for improving procedures are communicated openly and proactively. This was also the case with Air France Flight 447, when an Airbus A330 crashed into the Atlantic Ocean in 2009 (BEA, 2012). The aircraft encountered severe weather, causing a sensor to freeze and the autopilot to disengage. This led to a cascade of poor decisions, culminating in the crash of the aircraft. The crew bore only limited responsibility for this, as they had not been trained to handle such unusual flight conditions. At the time, it was believed that, from a technical standpoint, a modern aircraft could no longer find itself in such situations. That was a misconception. The incident was reason enough to make sweeping and lasting changes to training. Today, every pilot must pass (at least) one simulator training session every year, during which deliberately absurd flight manoeuvres are tested; otherwise, they lose their licence. If I lack practice in the basic task – flying – but then have to intervene in complicated emergency situations, something is going wrong.

Automation often puts us in a position where we have to do less, yet be capable of more. The psychologist Lisanne Bainbridge recognised this as early as 1983 in her much-cited article ‘Ironies of Automation’. However, a ‘human-in-the-loop’ approach is only effective if the person can take over manually when in doubt; or at least bring the system to a halt with minimal risk – which tends to be difficult whilst in the air. In the words of a pilot instructor from our interviews: “When in doubt, revert to basic airmanship.” This is exactly why so much training in aviation takes place in simulators. The fact that this is no mere formality was underscored by a recent court ruling in France against Airbus and Air France regarding the aforementioned accident: manslaughter – due to inadequate training (Reuters, 2026).

However, it is not just existing skills that need to be preserved; automation is also changing perceptions of roles, and new skills are becoming necessary. Humans-in-the-loop, for example, fulfil ‘friction roles’ (i.e. I deliberately slow down a process, for instance to identify risks in good time) or ‘mediation roles’ (e.g. when, as a doctor, I act as an intermediary between my patients and the medical systems) (Crootof et al., 2023). For pilots, this means they are now less like aerial acrobats and more like system managers. One pilot told us in an interview, ‘I’m the worst pilot I’ve ever been.’ He probably shouldn’t repeat that when greeting passengers, but it hits the nail on the head. The more extensive the automation, the more challenging the starting point for a human who has to adapt, stay alert and be ready to intervene. Yes, I still have to be able to do everything myself and remain fully responsible – and yes, I have to learn new skills and change my self-perception. Striking this balance is the fate of the ‘HILOs’.

Automation is not an end in itself

What can we learn from this? Decisions must not be automated blindly, simply because it is technically feasible or because it offers a particularly significant opportunity to cut staff costs. The aim should be to improve processes in a way that is harmonious, supportive, promotes safety and builds trust. What exactly that means depends, of course, on the individual case. Not every process within a company can be as clearly structured, standardised and safeguarded as processes in the aviation industry – that much is certain. After all, fortunately, mistakes do not immediately result in the deaths of 300 people in every sector. But perhaps that should not be the benchmark either. Nor is it about reducing risk to zero. If that were the case, we’d have to stay on the ground. However, if we want to assign a person a role within a process that improves the system, we must not ignore the conditions that need to be met for that to actually be the case. The three lessons outlined above could be a starting point for this:

1. Automation combines tasks into a new, more complex process: whoever is to oversee this process must be able to understand it, which is why automated understanding is more important than automated decision-making.
2. Automation concentrates responsibility on fewer people: to ensure that we do not lose track of things, we need clear, structured communication that keeps everyone up to date.
3. Automation requires us to do less but be capable of more: core skills must be retained, and it must be possible to take over (where appropriate) at any time, whilst at the same time people grow into new roles.

The most important question for successful automation, therefore, is not simply who controls what and to what extent. It is: How do people and machines get along with one another? (Crootof, 2023).

1. Of the 394 fatalities mentioned, 241 alone were caused by the crash of Air India Flight 171 on 12 June 2025. Consequently, the total annual figure is actually significantly higher than in previous years. ↩︎
2. Based on an empirical study by Grimholt et al. (2019) involving a representative sample of the general adult population in Norway (N = 5,500). ↩︎

References

ADAC. (2026, 11. August). Verkehrstote 2025: Mehr tödliche Unfälle auf Deutschlands Straßen! https://www.adac.de/news/bilanz-verkehrstote/ 

Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6), 775–779. https://doi.org/10.1016/0005-1098(83)90046-8 

Bureau d’Enquêtes et d’Analyses pour la sécurité de l’aviation civile (BEA). (2012). Final report on the accident on 1st June 2009 to the Airbus A330-203 registered F-GZCP operated by Air France flight AF 447 Rio de Janeiro – Paris. https://www.faa.gov/sites/faa.gov/files/AirFrance447_BEA.pdf

Commission Regulation (EU) 2015/140 of 29 January 2015 amending Regulation (EU) No 965/2012 as regards sterile flight crew compartment and correcting that Regulation, OJ L 24, 30.1.2015, p. 5–14. https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32015R0140

Crootof, R., Kaminski, M. E., & Price, W. N., II. (2023). Humans in the loop. Vanderbilt Law Review, 76(2), 429–510. https://scholarship.law.vanderbilt.edu/vlr/vol76/iss2/2

European Union Aviation Safety Agency (EASA). (2023). Easy access rules for large aeroplanes (CS-25) (Revision of January 2023). CS 25.1309. https://www.easa.europa.eu/en/document-library/easy-access-rules/online-publications/easy-access-rules-large-aeroplanes-cs-25?page=41 

Grimholt, T.K., Bonsaksen, T., Schou-Bredal, T., Heir, T., Lerdal, A., Skogstad, L., Ekeberg, Ø. (2019). Flight anxiety reported from 1986 to 2015. Aerospace Medicine and Human Performance, 90(4), 384–388. https://doi.org/10.3357/AMHP.5125.2019 

Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A.-H. S., Dellinger, E. P., Herbosa, T., Joseph, S., Kibatala, P. L., Lapitan, M. C. M., Merry, A. F., Moorthy, K., Reznick, R. K., Taylor, B., & Gawande, A. A. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine, 360(5), 491–499. https://doi.org/10.1056/NEJMsa0810119

Mahlow, P., & Mosene, K. (2025, 8. Dezember). Der Human in the Loop bei der automatisierten Kreditvergabe – Menschliche Expertise für größere Fairness. Digital Society Blog. https://doi.org/10.5281/zenodo.17857675

International Air Transport Association (IATA). (2026). Interactive safety report. https://www.iata.org/en/publications/safety-report/interactive-safety-report/ 

Reuters. (2026, 22. Mai). Airbus, Air France found guilty of manslaughter over 2009 Atlantic crash. CNBC. https://www.cnbc.com/2026/05/22/airbus-air-france-found-guilty-of-manslaughter-over-2009-atlantic-crash.html

Rieth, M. (2022). Auswirkungen zunehmender Automatisierung auf Beschäftigte in High Reliability Organizations – Eine Analyse der veränderten Arbeitsrolle und Anforderungen unter Berücksichtigung der Arbeitsgestaltung am Beispiel des Berufsbildes der Fluglots:innen. Pabst Science Publishers.