{"id":120898,"date":"2026-10-28T12:22:16","date_gmt":"2026-10-28T11:22:16","guid":{"rendered":"https:\/\/www.hiig.de\/?p=120898"},"modified":"2026-10-28T18:14:08","modified_gmt":"2026-10-28T17:14:08","slug":"autopilot-on-brain-off","status":"publish","type":"post","link":"https:\/\/www.hiig.de\/en\/autopilot-on-brain-off\/","title":{"rendered":"Autopilot on, brain off? What we can learn from aviation about automation"},"content":{"rendered":"\n

Ever since the launch of AI assistants such as ChatGPT and Claude, numerous companies at various levels have been considering automating more of their operations \u2013 from emails to decisions and even entire processes. The aim is to make workflows more efficient \u2013 and, as an added bonus, save on human resources. A win-win? Not so fast, because EU law requires that a human must remain \u2018in the loop\u2019 when it comes to significant decisions. But why is that, actually? And is it really enough simply to throw any person into a process anywhere? It\u2019s not quite that simple. If you want to automate effectively, you need more than just a human equivalent of a scarecrow. To understand what\u2019s required, it\u2019s worth looking up. After all, in aviation, automation has been an integral part of everyday practice for decades. As a best-practice example, the sector demonstrates what really matters: whether it\u2019s clear communication or the continuous development of training programmes. In this blog post, we\u2019ll look at what we can learn from aviation about the harmonious interaction between humans and machines for other sectors that are only just beginning to grapple with automation.<\/strong><\/p>\n\n\n\n

Flying is the safest form of civil transport there is. This becomes apparent right from the certification stage: the acceptable probability of a \u2018catastrophic failure\u2019 \u2013 that is, an acute risk of a crash \u2013 is 10\u207b\u2079 (EASA, 2003, CS-25.1309<\/a>); this equates to one such failure in every billion flight hours. The actual accident rate in 2025 stood at 1.32 per million flights, corresponding to 49 accidents with a total of 394 fatalities worldwide (IATA, 2026<\/a>).1<\/a><\/sup> These figures are no less saddening for that, but the 2,832 road deaths in the same year in Germany alone certainly put them into perspective (ADAC, 2026<\/a>). And yet, around 3 per cent of people suffer from a fear of flying (Grimholt et al., 2019<\/a>).2<\/a><\/sup> Do we not trust the pilots to get us safely from A to B? Or do we not trust the aeroplane to do what it is supposed to do? Doesn\u2019t the AI autopilot do all that anyway these days? Not quite.<\/p>\n\n\n\n

Automation in the air<\/strong><\/h2>\n\n\n\n

The role of technology in assisting humans has been considered in aviation for a long time (Bainbridge, 1983<\/a>). Landing, for example, has been automatable since the 1950s. By way of comparison: back then, computers were still larger than people and could not even remotely do what even the simplest, tiniest computers are capable of today. A great deal has changed technologically since then, yet flying still involves a surprising amount of manual work, as you can see if you take a look inside the cockpit, which is studded with dozens of buttons.<\/p>\n\n\n\n

Whilst there are many different support systems in place, and under normal conditions no pilot today needs to actively ensure that the aircraft remains stable at cruising altitude or flies in the correct direction, civil aviation is still a long way from full automation. The same applies to the implementation of modern AI systems on board. However, automation and the risks associated with it already begin with much simpler systems. As a sector where errors can cost human lives, the aviation industry has always taken a critical approach to the issue of safety. For example, the take-off of every commercial aircraft is still carried out manually; this is because there are aspects that we cannot (yet) hand over to machines \u2013 situations where one or more people are needed and complex decisions must be made. <\/p>\n\n\n\n

<\/svg>

The following learnings excerpt insights from a case study on aviation conducted as part of our research project \u2018Human in the Loop?<\/a>\u2019. The term \u2018Human-in-the-Loop\u2019 (HILO) is frequently used to describe people who are involved in a (partially) automated decision-making process in order to review, confirm, correct or halt machine-generated (preliminary) decisions. This could involve a decision on a loan application, a social media post or a police intervention (Mahlow & Mosene, 2025<\/a>). To find out whether \u2013 and under what conditions \u2013 we humans can truly be such a magic ingredient for better decisions, the team is investigating, through various case studies, how a human can make meaningful contributions. To this end, we have interviewed various experts \u2013 ranging from cockpit crews and human factors specialists to organisational psychologists \u2013 to understand workflows and analyse empirical findings. Always with the aim of identifying factors that influence collaborative decision-making between humans and machines.<\/p><\/div>\n\n\n\n

Who is the decision-maker in the sky?<\/strong><\/h2>\n\n\n\n

Decisions in the air are never, by principle, made alone. There are at least two people in the cockpit: the captain and the first officer, also known as the co-pilot. Whilst the captain has the final say, up to that point both (with a few exceptions) have equal authority to make decisions. Who is responsible for what is not determined by rank, but by a division of roles agreed upon before each individual flight \u2013 between the pilot flying and the pilot monitoring. This ensures that both are capable of performing both roles. However, the two are not alone up there. To ensure accident-free coexistence with other aircraft, there are also air traffic controllers in large ground-based control centres who ensure sufficient separation between all aircraft by issuing instructions and granting clearances.<\/p>\n\n\n\n

Autopilot on, brain off?<\/strong><\/h2>\n\n\n\n

So what does the autopilot actually do? Functionally, there are (roughly speaking) two main functions (plus backups): \u2018steering\u2019 and \u2018throttling\u2019. To steer, the computer simply controls various flaps around the aircraft, which are positioned to face the wind in specific ways so that the aircraft rotates, banks or pitches along the three axes as required to get back on course. No magic, just physics. It\u2019s not artificial intelligence, but rigidly programmed algorithms that follow clear rules and shut themselves down if data is missing (a faulty sensor) or something is wrong (too windy). This doesn\u2019t reinvent flying, but predictably takes the strain off the crew and makes the flight more efficient. <\/p>\n\n\n\n

And what is left for the pilot(s) to do? Even when the autopilot is engaged, pilots have their hands full. They monitor its \u2018actions\u2019, check fuel consumption, review alternative airports, keep an eye on the weather and maintain contact with air traffic control on the ground. This is anything but trivial and, despite the autopilot, remains a demanding task. When the aircraft is on final approach, the autopilot is usually switched off. This is because an automatic landing requires such extensive preparations that, paradoxically, it is \u2018easier\u2019 to land manually when visibility is good.<\/p>\n\n\n\n

Human in the loop, all good?<\/strong><\/h2>\n\n\n\n

The main cause of aviation accidents remains, in most cases, human error (IATA, 2026<\/a>). The machine performs its tasks reliably, but only those for which it was built. Everything else must necessarily be handled by people. This makes it all the more important that these people are empowered to fulfil this role. Otherwise, they would merely serve as scapegoats if something goes wrong (Crootof et al., 2023<\/a>). Yet this is precisely what can happen when systems are designed in such a way that, in the end, it is always the operator who is to blame and never the technology. True to the saying: \u2018The problem is sitting in front of the computer.\u2019 This is not merely a legally relevant issue. Above all, it is about how we design processes. In the following, we use three lessons learned to illustrate, in part, exactly how the aviation industry deals with this and what other sectors can learn from it.<\/p>\n\n\n\n

<\/div>\n\n\n\n

Learning 1: More than the sum of its parts<\/strong><\/h3>\n\n\n\n

In air traffic control centres, air traffic controllers work together in large halls, each responsible for specific airspaces. Whilst the handover of aircraft between airspaces is generally carried out strictly in accordance with flight plans, changes often occur due to weather or delays. In such cases, if there is any doubt, arrangements must be made to divert an aircraft from one sector to another. In the past, this was done by talking or via a phone call. Today, a system handles this with a simple yes-or-no pop-up window. Quick, efficient, clear \u2013 but too simple? Replacing human interaction with a yes-no window ignores tone of voice, choice of words and pitch (Rieth, 2022). These are all aspects that help to contextualise a statement: Does it need to be done quickly? Is it a \u2018could you\u2019 or a \u2018you must\u2019? Is there an immediate danger? None of this is conveyed by this pop-up window. It is not without reason that we frequently use emojis in chats; because they help to contextualise what we say. It is therefore not surprising that, despite the system, air traffic controllers still pick up the phone or call out to the neighbouring desk when it really matters. That is not the only reason why they still sit together in the same room to this day.<\/p>\n\n\n\n

This example highlights a fundamental problem: automation is intended to ease the burden on people by having a system take over one or even several tasks at once. However, by bundling these tasks together, the individual steps do not simply vanish into thin air; instead, they form a new, more complex whole. If I am the \u2018human-in-the-loop\u2019 overseeing this process, I must \u2013 in order to fulfil this role \u2013 be able to understand what is happening in detail, or at least notice when something goes wrong. To this end, the system must, first and foremost, inform me in a comprehensible manner exactly what is happening at any given moment.<\/p>\n\n\n\n

It is therefore not enough simply to add up the strengths and weaknesses of humans and machines. It is precisely this hybrid collaboration that gives rise to new, difficult-to-predict problems of its own; above all, misunderstandings. The supposed simplification can make processes more complicated and thus harder to control. Machines are, and will remain, tools. They should therefore be designed and used accordingly, to reduce blind spots rather than increase them. Automated understanding is more important than automated decision-making. <\/p>\n\n\n\n

<\/div>\n\n\n\n

Learning 2: Check, check<\/strong><\/h3>\n\n\n\n

One of the most important principles in aviation is redundancy. There are two turbines, half a dozen on-board computers from different manufacturers and three separate hydraulic systems for the flaps. Theoretically, only one of each would be needed. Yet redundancy is better not only from a technical point of view, but also in terms of human interaction. In our interviews, communication in the cockpit was compared to a theatre play. From reconciling fuel calculations, through the take-off procedure, to emergency situations such as a cabin fire, there are detailed checklists and scripts for everything that must be strictly followed. The overarching practice here is the \u2018check\u2019 principle: one person says something, the other confirms it; sometimes by repeating it word for word if the specific content is particularly important. Below 10,000 feet, there is even a ban on small talk (EU, 2015<\/a>); this means that only matters directly related to the flight (or aircraft) may be discussed. There is no place for weekend plans here. The rules are so clear that even deviating from them can in itself be a statement. If a step is missing from the process, the procedure is aborted in case of doubt.<\/p>\n\n\n\n

The medical profession has adopted this practice from the aviation industry: if a swab is missing, the patient is not sutured (Haynes et al., 2009<\/a>). Particularly when automation means that more and more responsibilities are concentrated on fewer and fewer people, crystal-clear communication is essential to ensure that no one loses track of the situation. And as long as one person remains essential, two are needed.<\/p>\n\n\n\n

<\/div>\n\n\n\n

Learning 3: New, old skills<\/strong><\/h3>\n\n\n\n

Deeply ingrained in the aviation industry is the determination to learn from every mistake, particularly accidents. These are systematically analysed, and the findings and recommendations for improving procedures are communicated openly and proactively. This was also the case with Air France Flight 447, when an Airbus A330 crashed into the Atlantic Ocean in 2009 (BEA, 2012<\/a>). The aircraft encountered severe weather, causing a sensor to freeze and the autopilot to disengage. This led to a cascade of poor decisions, culminating in the crash of the aircraft. The crew bore only limited responsibility for this, as they had not been trained to handle such unusual flight conditions. At the time, it was believed that, from a technical standpoint, a modern aircraft could no longer find itself in such situations. That was a misconception. The incident was reason enough to make sweeping and lasting changes to training. Today, every pilot must pass (at least) one simulator training session every year, during which deliberately absurd flight manoeuvres are tested; otherwise, they lose their licence. If I lack practice in the basic task \u2013 flying \u2013 but then have to intervene in complicated emergency situations, something is going wrong.<\/p>\n\n\n\n

Automation often puts us in a position where we have to do less, yet be capable of more. The psychologist Lisanne Bainbridge recognised this as early as 1983 in her much-cited article \u2018Ironies of Automation\u2019<\/a>. However, a \u2018human-in-the-loop\u2019 approach is only effective if the person can take over manually when in doubt; or at least bring the system to a halt with minimal risk \u2013 which tends to be difficult whilst in the air. In the words of a pilot instructor from our interviews: \u201cWhen in doubt, revert to basic airmanship.\u201d This is exactly why so much training in aviation takes place in simulators. The fact that this is no mere formality was underscored by a recent court ruling in France against Airbus and Air France regarding the aforementioned accident: manslaughter \u2013 due to inadequate training (Reuters, 2026<\/a>).<\/p>\n\n\n\n

However, it is not just existing skills that need to be preserved; automation is also changing perceptions of roles, and new skills are becoming necessary. Humans-in-the-loop, for example, fulfil \u2018friction roles\u2019 (i.e. I deliberately slow down a process, for instance to identify risks in good time) or \u2018mediation roles\u2019 (e.g. when, as a doctor, I act as an intermediary between my patients and the medical systems) (Crootof et al., 2023<\/a>). For pilots, this means they are now less like aerial acrobats and more like system managers. One pilot told us in an interview, \u2018I\u2019m the worst pilot I\u2019ve ever been.\u2019 He probably shouldn\u2019t repeat that when greeting passengers, but it hits the nail on the head. The more extensive the automation, the more challenging the starting point for a human who has to adapt, stay alert and be ready to intervene. Yes, I still have to be able to do everything myself and remain fully responsible \u2013 and yes, I have to learn new skills and change my self-perception. Striking this balance is the fate of the \u2018HILOs\u2019.<\/p>\n\n\n\n

Automation is not an end in itself<\/strong><\/h2>\n\n\n\n

What can we learn from this? Decisions must not be automated blindly, simply because it is technically feasible or because it offers a particularly significant opportunity to cut staff costs. The aim should be to improve processes in a way that is harmonious, supportive, promotes safety and builds trust. What exactly that means depends, of course, on the individual case. Not every process within a company can be as clearly structured, standardised and safeguarded as processes in the aviation industry \u2013 that much is certain. After all, fortunately, mistakes do not immediately result in the deaths of 300 people in every sector. But perhaps that should not be the benchmark either. Nor is it about reducing risk to zero. If that were the case, we\u2019d have to stay on the ground. However, if we want to assign a person a role within a process that improves the system, we must not ignore the conditions that need to be met for that to actually be the case. The three lessons outlined above could be a starting point for this:<\/p>\n\n\n\n

    \n
  1. Automation combines tasks into a new, more complex process<\/strong>:<\/em> whoever is to oversee this process must be able to understand it, which is why automated understanding is more important than automated decision-making.<\/li>\n\n\n\n
  2. Automation concentrates responsibility on fewer people<\/strong>:<\/em> to ensure that we do not lose track of things, we need clear, structured communication that keeps everyone up to date.<\/li>\n\n\n\n
  3. Automation requires us to do less but be capable of more<\/strong>:<\/em> core skills must be retained, and it must be possible to take over (where appropriate) at any time, whilst at the same time people grow into new roles.<\/li>\n<\/ol>\n\n\n\n

    The most important question for successful automation, therefore, is not simply who controls what and to what extent. It is: How do people and machines get along with one another? (Crootof, 2023<\/a>).<\/p>\n\n\n\n

    <\/div>\n\n\n\n
    \n\n\n
    1. Of the 394 fatalities mentioned, 241 alone were caused by the crash of Air India Flight 171 on 12 June 2025. Consequently, the total annual figure is actually significantly higher than in previous years. \u21a9\ufe0e<\/a><\/li>
    2. Based on an empirical study by Grimholt et al. (2019) involving a representative sample of the general adult population in Norway (N = 5,500). \u21a9\ufe0e<\/a><\/li><\/ol>\n\n\n
      \n\n\n\n

      References<\/strong><\/h2>\n\n\n\n

      ADAC. (2026, 11. August). Verkehrstote 2025: Mehr t\u00f6dliche Unf\u00e4lle auf Deutschlands Stra\u00dfen!<\/em> https:\/\/www.adac.de\/news\/bilanz-verkehrstote\/<\/a> <\/p>\n\n\n\n

      Bainbridge, L. (1983). Ironies of automation. Automatica<\/em>, 19<\/em>(6), 775\u2013779. https:\/\/doi.org\/10.1016\/0005-1098(83)90046-8<\/a> <\/p>\n\n\n\n

      Bureau d\u2019Enqu\u00eates et d\u2019Analyses pour la s\u00e9curit\u00e9 de l\u2019aviation civile (BEA). (2012). Final report on the accident on 1st June 2009 to the Airbus A330-203 registered F-GZCP operated by Air France flight AF 447 Rio de Janeiro \u2013 Paris<\/em>. https:\/\/www.faa.gov\/sites\/faa.gov\/files\/AirFrance447_BEA.pdf<\/a><\/p>\n\n\n\n

      Commission Regulation (EU) 2015\/140 of 29 January 2015 amending Regulation (EU) No 965\/2012 as regards sterile flight crew compartment and correcting that Regulation, OJ L 24, 30.1.2015, p. 5\u201314. https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/HTML\/?uri=CELEX:32015R0140<\/a><\/p>\n\n\n\n

      Crootof, R., Kaminski, M. E., & Price, W. N., II. (2023). Humans in the loop. Vanderbilt Law Review<\/em>, 76<\/em>(2), 429\u2013510. https:\/\/scholarship.law.vanderbilt.edu\/vlr\/vol76\/iss2\/2<\/a><\/p>\n\n\n\n

      European Union Aviation Safety Agency (EASA). (2023). Easy access rules for large aeroplanes (CS-25) (Revision of January 2023). CS 25.1309. https:\/\/www.easa.europa.eu\/en\/document-library\/easy-access-rules\/online-publications\/easy-access-rules-large-aeroplanes-cs-25?page=41<\/a> <\/p>\n\n\n\n

      Grimholt, T.K., Bonsaksen, T., Schou-Bredal, T., Heir, T., Lerdal, A., Skogstad, L., Ekeberg, \u00d8. (2019). Flight anxiety reported from 1986 to 2015. Aerospace Medicine and Human Performance<\/em>, 90<\/em>(4), 384\u2013388. https:\/\/doi.org\/10.3357\/AMHP.5125.2019<\/a> <\/p>\n\n\n\n

      Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A.-H. S., Dellinger, E. P., Herbosa, T., Joseph, S., Kibatala, P. L., Lapitan, M. C. M., Merry, A. F., Moorthy, K., Reznick, R. K., Taylor, B., & Gawande, A. A. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine<\/em>, 360<\/em>(5), 491\u2013499. https:\/\/doi.org\/10.1056\/NEJMsa0810119<\/a><\/p>\n\n\n\n

      Mahlow, P., & Mosene, K. (2025, 8. Dezember). Der Human in the Loop bei der automatisierten Kreditvergabe \u2013 Menschliche Expertise f\u00fcr gr\u00f6\u00dfere Fairness<\/em>. Digital Society Blog. https:\/\/doi.org\/10.5281\/zenodo.17857675<\/a><\/p>\n\n\n\n

      International Air Transport Association (IATA). (2026). Interactive safety report<\/em>. https:\/\/www.iata.org\/en\/publications\/safety-report\/interactive-safety-report\/<\/a> <\/p>\n\n\n\n

      Reuters. (2026, 22. Mai). Airbus, Air France found guilty of manslaughter over 2009 Atlantic crash<\/em>. CNBC. https:\/\/www.cnbc.com\/2026\/05\/22\/airbus-air-france-found-guilty-of-manslaughter-over-2009-atlantic-crash.html<\/a><\/p>\n\n\n\n

      Rieth, M. (2022). Auswirkungen zunehmender Automatisierung auf Besch\u00e4ftigte in High Reliability Organizations \u2013 Eine Analyse der ver\u00e4nderten Arbeitsrolle und Anforderungen unter Ber\u00fccksichtigung der Arbeitsgestaltung am Beispiel des Berufsbildes der Fluglots:innen<\/em>. Pabst Science Publishers.<\/p>\n