The apparent diagnostic success of the MYCIN expert system rests on an indispensable, non-formalizable contribution from human physicians, whose holistic judgment of the seriousness of a patient’s illness cannot be reduced to heuristic rules, showing that even sophisticated AI systems depend on intuitive human understanding at crucial decision points.

By Hubert L. Dreyfus, from What Computers Can't Do

Key Arguments

  • To achieve its reported level of performance in prescribing powerful but dangerous drugs, MYCIN requires that human experts supply an evaluation of 'the seriousness of the patient's illness', rather than computing it from laboratory data.
  • This global evaluation 'cannot be computed from a battery of medical tests but must be arrived at by an experienced phsyician with a holistic grasp of the patient's overall condition', indicating reliance on gestalt, situation‑sensitive clinical understanding.
  • The 'knowledge engineers have wisely not even tried to reduce this intuitive aspect of the diagnosis to heuristics rules', implicitly acknowledging that this core aspect of medical expertise does not lend itself to representation as rule‑based knowledge.
  • Thus, MYCIN’s success does not demonstrate that expert medical judgment has been formalized; it shows instead that AI can only operate effectively when supplemented by non‑formal, embodied human expertise.

Source Quotes

1023. Before being overly impressed by these statistics, however, the reader should realize that to achieve this performance the program requires the aid of human experts. Feigenbaum does not mention that in order to know when to prescribe powerful but dangerous drugs the program must be given an evaluation of the seriousness of the patient's illness.
Before being overly impressed by these statistics, however, the reader should realize that to achieve this performance the program requires the aid of human experts. Feigenbaum does not mention that in order to know when to prescribe powerful but dangerous drugs the program must be given an evaluation of the seriousness of the patient's illness. This judgment cannot be computed from a battery of medical tests but must be arrived at by an experienced phsyician with a holistic grasp of the patient's overall condition.
Feigenbaum does not mention that in order to know when to prescribe powerful but dangerous drugs the program must be given an evaluation of the seriousness of the patient's illness. This judgment cannot be computed from a battery of medical tests but must be arrived at by an experienced phsyician with a holistic grasp of the patient's overall condition. The knowledge engineers have wisely not even tried to reduce this intuitive aspect of the diagnosis to heuristics rules.
This judgment cannot be computed from a battery of medical tests but must be arrived at by an experienced phsyician with a holistic grasp of the patient's overall condition. The knowledge engineers have wisely not even tried to reduce this intuitive aspect of the diagnosis to heuristics rules. 72.

Key Concepts

  • Before being overly impressed by these statistics, however, the reader should realize that to achieve this performance the program requires the aid of human experts.
  • Feigenbaum does not mention that in order to know when to prescribe powerful but dangerous drugs the program must be given an evaluation of the seriousness of the patient's illness.
  • This judgment cannot be computed from a battery of medical tests but must be arrived at by an experienced phsyician with a holistic grasp of the patient's overall condition.
  • The knowledge engineers have wisely not even tried to reduce this intuitive aspect of the diagnosis to heuristics rules.

Context

Endnote 71 in the Introduction to the Revised Edition, where Dreyfus comments critically on Feigenbaum’s presentation of MYCIN’s performance statistics and uses the system as an example of AI’s dependence on irreducibly intuitive human judgment.