Expert Systems as General-Use Advisory Tools: An Examination of Moral Responsibility Kimberly Cass Marquette University College of Business P.O. Box 1881 Milwaukee, WI 53201-1881 Abstract The idea of creating and maintaining a repository of organizational expertise to be accessed by anyone needing it within the firm and applying it in diverse and innovative ways sounds appealing. Human experts would no longer have to be burdened and distracted by ad hoc problem-solving inquiries from those requiring their expertise and could focus their time and attention on more interesting and important questions in their domain area. People requiring expertise would no longer be dependent upon the human expert's availability, and could access computer-based expertise at any time and for any reason to arrive at a recommendation or solution to their inquiry. Thus, expertise could be integrated into organizational problem-solving without over-taxing the human expert and everyone in the organization would have the potential to incorporate the firm's expertise into their individual projects and tasks. However this vision of expert systems (ES) usage contrasts with that historical experience. Early ES were developed as systems for experts and typically implemented and used by small groups of people in limited settings. The initial idea was to provide a computer-based expert that a human expert could consult as a "colleague" in order to gain a different perspective on a problem-solving procedure. Additionally, expert systems were to be employed by those who were training to become experts in a given field, to assist and mentor them in the problem-solving processes of the domain area that they were learning. Early in the history of ES, the roles of expert, knowledge engineer, designer, implementer, maintainer, and user overlapped. Hence, the ethical issues involved with such systems were relatively self-contained. More recently, these roles tend to be assumed by separate individuals, and the ES themselves are available to broader classes of users. This paper presents a model of the roles that manager, human expert, knowledge engineer, ES designer, ES builder, ES maintainer, and user play in a "general use" expert systems environment. It examines the issue of moral responsibility and culpability of each role in a case in which something goes wrong due to a course of action which was recommended by an ES consultation. Introduction For an expert system (ES) to be designed and developed, management must make a decision to capture and codify expertise in a given domain to be used in a particular environment for a specific purpose. As ES in many ways allow the leveraging of expertise, systems must demonstrate both economic and performance benefits (Gill, 1995). Additionally, Gill (1995) stresses the importance of addressing both technical and organizational impacts and perspectives when deciding to implement an ES. DeLone and McLean (1992) have identified the following organizational factors necessary for ES success: that the system does what it was designed to do, that it creates usable output, that potential users begin to regularly use the system, that users are satisfied with the system, and that the ES affects the behavior of users and has a positive effect on organizational performance. With ES use an organization or department can enforce consistency and quality in decision- making. ES have been shown to increase problem-solving speed and can help users perform tasks faster. Early on in their history, Watermann (1986) stressed that ES would yield lower error rates when used by knowledgeable and skilled users. Early ES were designed to be systems for experts and typically implemented and used by small groups of people in limited settings. The initial idea was to provide a computer-based expert that a human expert could consult as a "colleague" in order to gain a different perspective on a problem-solving procedure. Additionally, ES were employed by those who were training to become experts in a given field, to assist and mentor them in the problem-solving processes of the domain area that they were learning. Hence, the ethical issues involved with such systems were relatively self contained. Recently, ES have been proposed as vehicles for retaining and disseminating an organization's repository of expertise (Hammer and Champy, 1993). Expertise can by accessed by anyone needing it and applied in diverse and innovative ways during problem-solving and decision-making. Human experts would no longer have to be burdened and distracted by ad hoc problem-solving inquiries from those requiring their expertise and could focus their time and attention on more interesting and important questions in their domain area. People requiring expertise would no longer be dependent upon the human expert's availability, and could access computer-based expertise at any time and for any reason to arrive at a recommendation or solution to their inquiry. By changing the manner in which ES are used, we must identify the assumptions implied by that change: That the ES consultation is equivalent to a consultation with a human expert, and that expertise is generally understandable and transferrable. Borgmann (1984) would argue that by removing the human expert, the robust and "forgiving" structure of that consultation would be diminished, and hence the quality of the consultation. Wittgenstein (1953) questions the possibility of "contextless expertise." Without a framework in which to contextualize information, it is impossible to distinguish what is important from what is irrelevant. Without this structure there is no way to organize perceptions of reality so that they can yield meaningful revelations about one's world. Technical Roles Managers, experts, knowledge engineers, designers, implementers, maintainers, and users all have their prescribed roles in the development, use and maintenance of ES. We briefly discuss these roles in terms of technology to facilitate the consideration of the inherent ethical issues. Manager: A departmental manager identifies the task area in which the ES will be developed and used. She identifies and sanctions an expert and the technical representation of domain expertise. Most importantly, the manager must support the project, viewing it as an important activity to which the expert, knowledge engineer, and others can give priority. Additionally, the manager must anticipate how the ES will be used in order to integrate its use into the standard operating procedures for the department. Management should articulate ES use policies as well as maintenance and update policies. Expert: An expert possesses domain expertise and knows how to apply it. She understands how to interpret the environment in order to apply her expertise correctly and usefully. An expert is aware of the boundaries and limitations of her abilities, and factors this into her judgements. An expert constantly gathers and "connects" new information in her domain area. This knowledge and expertise accrual process takes an organic shape as the expert may identify information from other domains as pertinent to her area and arrive at insights therefrom. Lastly, and most importantly, an expert knows more than she can easily convey. Dreyfus and Dreyfus (1986) identify a phenomenon they term the "paradox of expertise." Through prolonged experience with problem-solving in a domain area, an expert develops non- rational and extra-linguistic intuitions about how to approach problems. When attempting to explain what she does in a problem-scenario, an expert inadvertently resorts to "beginner" explanations that do not accurately reflect what she does; the descriptions have little correspondence to the actual decision processes. To address this paradox, the next role was developed. Knowledge Engineer: A knowledge engineer attempts to capture, codify, and represent an expert's expertise and decision-making processes. With the goal of breaking through the expert's "paradox of expertise," the knowledge engineer carefully scrutinizes how the expert goes about solving the problem to discover the content and sequencing to which the expert attends and refers. The knowledge engineer tests his model by observing the expert undertake a series of problem-solving tasks in various settings. Additionally, the knowledge engineer develops a series of cases for the expert to solve under observation while "thinking outloud." The knowledge engineer also employs in-depth interviews to construct the expert's conceptual framework for contextualizing the expertise and for eliciting further domain-specific expertise. The justification for the knowledge engineer's role is based on the early artificial intelligence assumption that there are general problem-solving principles that are used by all domain areas. The knowledge engineer is accomplished at using these principles, and from these, he can identify similar patterns in the expert's discipline and "flesh them out" with domain-specific details. Although a knowledge engineer has general problem-solving experience, it can be argued that his lack of background and experience in the domain area may inhibit the proper interpretation and contextualization of the expert's knowledge. Additionally, with a lack of background in the domain area, it may be difficult to validate and verify the ES model of the expert's expertise. Validation determines that the model of expertise agrees with the expertise itself, whereas verification determines that the articulation or implementation accurately reflects the model. Expert System Designer: The system designer articulates the purpose of the ES and determines what it should do. The designer selects an appropriate development tool or expert system shell with which to develop the system. Most importantly, she designs the user interface, which in most systems accounts for over half of the written code. The designer works closely with users to discover the purpose of the system, and once that is determined, develops plans for the best way to accomplish that purpose. The designer usually works with the users to arrive at an acceptable and easy-to-use interface. The designer must validate the design, that it accurately reflects the expertise and problem-solving that was captured by the knowledge engineer. Expert System Implementer: The ES implementer translates the knowledge and procedural information gathered by the knowledge engineer and the design guidelines developed by the designer into a workable computer-based system. The implementer must verify that the implementation aligns with and is a true expression of the validated design. The implementer works with the designer, expert, and knowledge engineer to test the ES and to determine that it functions at an acceptable level of performance before it is deployed. Expert System Maintainer: The task of the expert system maintainer is to keep current the knowledge and rules represented in the ES. ES are designed and developed to make changes and updates to their knowledge bases as effortless as possible. Just as an expert constantly gathers information in his field, integrates and applies it in problem-solving, an ES maintainer must identify, capture, represent, and integrate this evolving expertise into the ES. Failure to do so would result in an ES that had a stagnant representation of a changing body of knowledge; the further away in time from its inception, the more out of alignment it would be with the expert's perception of the domain. An ES is not a closed, complete environment, but rather an open, and evolving system. New insights and knowledge must be added to the system if it is to function as intended. User: The ES user has a problem or scenario that requires expertise to resolve. The user must be able to translate his perception of "reality" or the "domain state of affairs" into statements, values, or attributes in response to the ES's questions. The user must be capable of determining what the ES requires as input and providing it. Implications of Changing How ES are Used The above roles assume a capable knowledge engineer and a domain-competent user, that is, another domain expert who will consult with the ES to get a second opinion, or at least someone who is apprenticing in domain area. When the application of ES broadens to include users without extensive knowledge and experience in the domain, certain contrasts must be noted, and with them the assumptions they entail. A user who is a domain novice lacks the experience or conceptual framework with which to critique or evaluate the problem-solving process. In some cases this lack of experience may reduce his ability to discern relevant information in the problem environment. Lacking a framework for the problem-solving process, the domain-novice user may misinterpret phenomena in the environment, and be incapable of noticing erroneous responses during the problem solving process. Thus the novice may be unable to understand his own impact on the recommended solution. Additionally, unlike computer-based systems that query the user for values to incorporate into a fixed model, the ES tailors its decision-process to the user's input; the user co-determines the problem-solving process. Thus, an erroneous response may cause the ES's problem-solving mechanism to abandon a correct line of reasoning, or to pursue an erroneous one. This may go unnoticed and unquestioned by a novice user, who would then arrive at an incorrect result. Contrary to a domain expert who has both problem-solving process and outcome knowledge, a domain novice has achieving the outcome or result as his goal. Having no basis for an expected result, the novice may accept an incorrect result to be correct and incorporate it into his purposes. As an ES is developed to off-load some of the problem-solving tasks that take up a human expert's time, the question must be raised whether an ES is equivalent to consulting with an expert. It may be argued that an ES impoverishes the problem-solving process by replacing the expert's intellectual and experiential flexibility with a static representation of his expertise. With a human expert, both the expert and consultee together create a joint context in which to explore specific situations, scenarios, and possible options for solutions. They can identify and talk about the ramifications of trade-offs and risks involved in the problem and various solutions. They can both bring to the problem-solving setting extra- domain experience and information that may enhance the process. During a face-to-face consultation with a human expert, errors, misrepresentations, and misinterpretations can be addressed and corrected as they occur and not impact the result. Ethical Concerns of ES Usage What happens when expertise is disembodied from the expert and commodicized into one of many "bricks" of a diffuse decision-making "edifice?" Here we must address the moral ramifications of interjecting ES technology into the decision-making process. How does the process change? How do roles and responsibilities change? Might human experts be left out of the process in order to avoid issues of moral responsibility in the use of expertise? Can we assume that it is possible to articulate expertise so clearly as to eliminate ambiguity that it can be communicated to anyone without misinterpretation? Can we assume that expertise functions as "immutable substance," incapable of being misunderstood or misapplied? Once expertise is represented, can we assume that it is perfectly correct and that nothing can go wrong, that there are no conditions or qualifications to consider before applying it, that there are no possibilities of error? Suppose something does go wrong as a result of a decision based in part on an ES recommendation, and in analyzing what went wrong, the ES result is determined to be the cause. In contrast to other types of computer-based systems, ES can never be fully tested (Watermann, 1986). Just because an ES is approved by the expert in terms of its function, it is not possible to account for all the ways in which the system can be used. Certain safeguards can be incorporated to prevent errors in input, but all possible errors can neither be identified in advance nor corrected by countermeasures. Can responsibility be assigned to the user? the expert? the designer? the knowledge engineer? the manager? Does ES-mediated expertise lessen the impact of moral and psychological responsibility on those involved in its design, inception, and application? In order to reduce the problems outlined above, we must define moral responsibility, and incorporate it into the technical definitions of ES roles in order to heighten people's awareness of their obligations and culpability in the ES-mediated problem-solving process. If something goes wrong with a course of action, someone is morally responsible to the extent the following three conditions occur: 1).The person is the causal agent of the action. She is capable of performing the action and intentionally performed it. Her action was motivated internally, rather than compelled by external coercion. She acted voluntarily. 2).She had knowledge of the consequences and impacts of her behavior and decisions. 3).She could have acted differently in the situation (Aristotle, 1941). When ES are positioned as general-use advisory tools to be employed by potential users who have little or no training in or knowledge about the domain area, we have an obligation to consider the moral ramifications of ES from idea to inception to use. The following is intended to be used in addition to the technical descriptions of the ES roles in order to make all parties involved aware of the moral aspect of their job roles and function. The next section will suggest some considerations that each role might incorporate into its job function to address issues of moral responsibility. Toward an articulation of moral responsibility of ES roles Now the ES roles will be defined from an ethical perspective. By examining each role with respect to the three conditions of moral responsibility, we can incorporate these characteristics into the technical role definitions to assure that ES are designed, developed, maintained and used ethically and responsibly. Manager: The manager is a causal agent in that, without his original vision and thrust, the ES would not have been designed and developed to provide expertise within the department. By invoking policy that would require using an ES instead of conferring with a human expert, the manager revamps the way expertise is encountered and disseminated within the department. The manager must evaluate the impact of decisions and behaviors resulting from an ES consultation. Since the manager has identified the task and set up some standard operating procedures for the ES, he needs also to research and determine the ramifications of replacing a human expert with an ES for the given task. The manager is responsible for defining the contextually "normal" situation of ES use, and for determining what is reasonable to expect from the ES in that situation. The manager has little actual control over errors resulting from ES use. However the manager can identify circumstances in which the ES can be misused, create a policy of "correct use" and develop and enforce sanctions incorrect use. Since the human expert, who in many cases would deter misuse of her expertise, is removed from the system, the responsibility for knowing when, how, for what purpose, and by whom the system will be used falls upon the manager. Perhaps capturing consultation sessions in a log file would impose a deterrent to system misuse. At the same time, this could provide important information for the ES designer and implementer for assessing system performance and identifying problem areas in consultations. This would also provide a historical log that would reflect the consultations and results, so that if something went wrong with a decision based upon an ES session, that the problem could be pinpointed. Provided that the manager thoroughly researched the effects of automating domain expertise on the decision making process, considered the concerns about and cautions about the ES raised by others involved with the inception/design/implementation process, and institute policies and procedures for correct use, he would not be morally culpable for a problem traced to the ES consultation. However, if the manager does not adequately research the impact of ES or ignores information about potential problems with the ES's intended use, he would have knowledge of problematic outcomes, and could have halted or revamped the project, and in this case be held morally responsible for problems that occur. Expert: The expert would be a causal agent if she deliberately misrepresented or withheld critical aspects of that expertise, and these introduced errors into the problem-solving process. In ES development, the expert is responsible for a full disclosure and clear articulation of her skills and expertise to be represented by the system. The expert is responsible for providing the best information possible about her problem-solving procedure. Also, the expert must state the assumptions, biases and values that are embedded in the ES to counteract the user assumption that they are receiving unbiased, objective, value-free expertise. The domain expert has the responsibility to candidly assess the possible problems that could arise from people consulting with a representation of expertise rather than directly with the expert. The expert has the responsibility to identify limitations in applying the expertise, and the ways it might be abused or misused. The expert must develop countermeasures to assure minimal misuse and misunderstanding of the expertise. The expert must be part of the team who decides how the ES will be used and by whom. Whereas the expert can not control the outcome of an ES consultation, she can control what is represented by the ES. If there are areas the expert deems areas to be too subtle or too confusing for non-experts, these should be explicitly documented and the potential difficulties that could occur should be described. If the expert believes that certain parts of the process are wrought with problems, she must state that, and question the reasonableness of representing it with an ES. The expert has the ability and responsibility to inform the ES design/implementation team of difficulties inherent in the problem-solving process. If the expert requires input that is difficult if not impossible to represent in statements, then she must refuse to attempt to characterize that part of the process. An ES requiring too much nuance and subtlety from domain novices may invite input errors. If the expert foresees too many obstacles to error- free use, she must oppose the project. This may be difficult if management views such opposition as insubordination. Nonetheless, the expert is culpable to the degree that she knows how the ES will be used and the problems users may encounter in understanding and/or applying the domain expertise. Knowledge Engineer: The knowledge engineer can be a causal agent if he misunderstands or misrepresents the expert's problem-solving process. As the knowledge engineer's role is to capture and represent the expert's expertise, he must assess the limitations and adequacy of doing this with each new knowledge domain. Some areas of expertise are so arcane that a high level of background knowledge is necessary to convey the most basic of ideas. The difficulty of knowledge acquisition is increased when people with diverse experiential backgrounds attempt to communicate. Additionally, knowledge engineers encounter the aforementioned "paradox of expertise," that experts develop non-rational and extra-linguistic intuitions that can not be articulated. Problem-solving processes in a domain are not limited to the rules and information in the area. An expert can draw from many sources to reach a conclusion. Since the thought process that brings information from outside the domain to a problem-setting is unconscious, it is difficult to determine the influence of extra-domain input and perspectives on domain issues. Human reasoning, unlike ES, is not neatly "partitionable" into problem-solving areas. Realizing the potential dangers of misunderstanding or misrepresenting the expert's ideas, the knowledge engineer must always be conscious of the limitations involved in eliciting knowledge. The knowledge engineer should document any uncertainties in the process and verify them with the expert. Due to the subtlety of errors in communication, we cannot expect the knowledge engineer to be absolutely aware of the limits of his understanding on representing expertise. We can not expect knowledge engineers to be conscious of their unconscious communication errors nor expect them to be responsible for problems resulting from them. Assuming that the knowledge engineer has received the expert's approval on his interpretation of the expertise and verified and clarified all uncertainties with the expert, he would not be morally responsible for an error. Expert System Designer: The ES designer is a causal agent if her design does not accurately reflect and employ the expert's expertise and problem-solving process in the ES. Additionally, the designer is responsible for shaping the user interface. The difficulty of this task is compounded if the domain experience of potential users varies widely. Such features as help, explanation, cautions, reasoning facilities, and rationale for approach must accommodate different user levels and function as safeguards against error and misuse. The interface design must allow all levels of users to supply accurate input to the system. The designer must question and oppose any foreseeable consultation scenario in which the ES might be misused at any level. The designer must accurately incorporate the expert's vision of system limitations and boundaries to counteract possible overconfident assumptions brought by users. The design must reinforce the importance of the user's role in defining the problem and in supplying accurate data. It is critical that the interface be tested and refined using suggestions from representatives of different domain experience levels. The designer of a system for experts could assume users have a high level of background knowledge and experience. However, the interface must somehow augment the non-expert user's lack of domain knowledge so that the ES can be employed correctly. The ES must be designed to underscore the attentive and conscious participation of users. The designer ultimately has neither control over who uses the ES nor how potential users will interpret the statements presented by the ES interface. Designers must articulate the dangers of potential misuse and propose countermeasures to prevent them. As much as possible, designers need to incorporate functions and checks that will prevent the misuse of the ES. If the designer identifies types of users that are prone to misuse the system, she should advocate a policy to exclude them from using the ES or to have them use the system with assistance. The designer is morally responsible if she does not adequately validate her design model or if the interface is poorly designed and leads to user error. Expert System Implementer: The ES implementer is the causal agent when the system he builds does not accurately reflect the designer's representation of the expert's knowledge-base and problem-solving process. Hence, the ES functions differently than its design. The ES implementer must rigorously validate that the ES's function aligns with its design. As previously indicated, an inherent problem with ES is that they cannot be fully tested; all the possible scenarios of a consultation can not be exhausted. Knowing this, the implementer must develop a testing plan that will cover the major types of consultations. The implementer lacks complete control if the system malfunctions and can not be morally responsible if the ES malfunctions in an unforeseen way. Expert System Maintainer: The ES maintainer is the causal agent if an error occurred because a modification to the knowledge base or interface was not made. The ES maintainer's role assures that the ES "keeps learning" along with the human expert it models. The maintainer must adhere to the proposed time frame for reviewing the system with the expert so that the ES reflects the expert's current thinking and knowledge in the domain area. The ES maintainer must judge whether the scheduled reviews are sufficient to keep the system current. The ES maintainer must carefully monitor the system and assess any problems the users have or suggestions that they convey. The maintainer must be concerned with incorporating clarifications and modifications that will make the system easier to use. The maintainer knows that failure to perform updates and modifications will result in a deficient ES. Therefore, she should be aware of the impact that action or lack of action will have on the ES's performance. Problems caused by using erroneous or dated information or techniques or by problematic aspects of the ES interface suggest maintenance error. The ES maintainer is morally responsible when an error results because she either neglected to modify the ES in response to expert or user information, or neglected to conduct a scheduled ES review with the expert. User: If something goes wrong because of a user's ES recommendation, then the user would be the causal agent because he applied the expertise in the real world. However, a user's domain knowledge impacts on his culpability when employing ES expertise. This section will examine the extreme cases of domain novices and domain experts to highlight the issues and responsibilities involved with each user type. The more moderate positions will not be discussed in this paper, but should be imagined to entail aspects of the examined extreme positions. The user's domain experience affects the way in which he relates to the ES. Johnson and Mulvey (1995) apply Bayles's (1981) models of professional-client relationship to the users and designers in the systems design process in order to clarify the roles and responsibilities. The models can be used to highlight the different relationships to ES that occur when domain experience level varies among users. A fiduciary relationship allows both client and professional to disclose information and to have input into the decision-making process. Both parties discuss, evaluate and weigh alternative solutions and thus share responsibility in the decision-making process and for the decision reached. This model reflects an expert-ES consultation as well as a non-expert-human expert consultation. In both cases the consultee takes responsibility to assure that expertise will be applied correctly for his specific purposes. The richness, flexibility and robustness of the dynamic allows the consultee to actively participate in the problem-solving process posing questions, raising issues or providing contextual information when the need arises. In a paternalistic relationship, the client gains the benefit of the professional's expertise, yet is not included in the decision-making process that applies the expertise to a specific situation. This model reflects a novice-ES consultation. Although an ES can offer explanations and justifications for presenting its questions and making its recommendations, these facilities are designed for general cases and cannot provide content information for a specific problem context. The ES dictates the novice's participation in the consultation, and due to its nature precludes the novice from questions or explanations outside of its implementation. Thus, the novice "does what he is told" rather than uniquely co-create the problem- solving process. If the user is a domain expert, and something goes wrong as the result of the ES recommendation, then the user is the causal agent because he impacted "the world." As a domain expert, he has the ability to comprehend and critique the problem-solving process as it progresses because he shares the same domain knowledge, conceptual framework and world-view represented by the ES. The user brings to the consultation vast domain experience and a conceptual framework in which to critically participate in the problem-solving process. When the ES solicits information, the expert has the capacity to understand why, as well as the ability to discern the underlying problem-solving model. Given this awareness, the expert can critically evaluate assumptions inherent in the process, identify biases represented in the ES, and can accept, reject, or question the ES's recommendation given. Also, the expert consulting the ES has experience with interpreting data in the environment according to the domain's conceptual framework. This increases the possibility that the expert knows what to look for and how to accurately characterize it. Thus, the expert user employs the ES to generate an alternative opinion to his own; he is capable of arriving at a result independent of the ES; he has both process and outcome knowledge. The domain expert has the capacity to question assumptions, values, procedure and even the result. He is capable of discerning when something goes wrong with the process and result, enough so that if a questionable result is obtained, he can re-check the flow of the problem-solving process and identify problems. Because of his expert experience, he is also capable of disputing a result. The expert modelled in the ES may evaluate variables quite differently than an expert user. When the expert user arrives at a different result than the ES recommends, a dilemma occurs. One of the benefits of ES is that they enforce consistent decision- making. However, if an expert disagrees with an ES recommendation, there must be some mechanism to register that disagreement. Ideally there would be a policy governing use of the ES. If the ES recommendations are to be enacted without dispute, and the expert disagrees, there must be some way to articulate his uneasiness with the ES recommendation, possibly suspend action, and argue why that course of action should not be followed. If policy requires enacting the ES's recommendation, then the consulting expert would not be responsible for resulting problems. If the consulting expert agrees with the ES recommendation, and something goes wrong as a result of the recommendation, the certainty of the environment must be considered. If the recommendation is reasonable and sound, yet environmental idiosyncracies cause the course of action to fail, the consulting expert has no control over the outcome and thus is not responsible for the failure. A novice's lack of a conceptual framework in the domain area precludes him from understanding and evaluating the problem-solving process as it unfolds. With a sketchy idea of the process, a novice may not detect when he inputs a wrong value. In some cases, this deficiency of experience may reduce the ability to discern relevant information in the problem environment. Lacking domain knowledge, a novice cannot question ES assumptions or arrive at alternative results on his own. He cannot critique the process dimension of the consultation. The novice may misunderstand nuances in domain terminology. This would go unnoticed and unquestioned by a novice, who would arrive at an incorrect result. A domain novice lacks problem-solving process knowledge, while holding the outcome or result as his goal. Having no basis for a reasonable outcome expectation, the novice may accept an incorrect result and implement the ES recommendation. The novice would be a causal agent if something went awry as a result of following the ES recommendation. However, since the novice has little experience in the domain area, he can use the tool to the best of his ability and unknowingly make grave errors. The novice's knowledge deficiency precludes his ability to discern his errors. Even when an ES is equipped with the most rigorous help and explanation facilities, the novice must be aware of the need to use them. Having little experience or understanding of the problem-solving process, a novice may be incapable of determining when he is in trouble and needs help. There may be an assumption that the ES will be able to reach a correct result regardless of the novice's ability to express his needs. Additionally, a novice is not able to conceive of possible consequences of their decisions, since he lacks an in-depth understanding of the problem-solving process. Assuming that he used the system as intended to the best of his ability, the novice would not be responsible if a problem resulted from their consultation. Conclusion This paper presents ways of viewing the roles in an ES environment from the standpoint of moral responsibility. By changing both the environment in which they are used and the domain experience-level of their users from those originally intended, the possibility of system misuse and system error increases. ES must be understood in terms of a larger context which incorporates an ethical dimension of their use. Potential pitfalls and difficulties with their use must be acknowledged and addressed in any situation in which the ES contributes to the decision-making process. Perhaps those involved with or affected by decisions involving ES should be notified of ES contribution to the decision process. Each role in the ES inception/development/use/maintenance process has the potential to be the causal agent of a problem that results from an ES inquiry. Checks and balances must be developed to assure that the constraints and limitations voiced by the expert and designer are conserved and reflected throughout the different phases of ES development. That an ES models human expertise is no reason to assume away the ethical considerations in its use. By replacing human experts in the decision process, ES make ethical issues more prominent. By taking the human expert out of the consultation process more attention must be given to assuring that ES and the expertise they represent are used in a responsible manner. References Aristotle, Ethica Nicomachea, in The Basic Works of Aristotle, Random House, New York, 1941. Bayles, M., Professional Ethics, Wadsworth, Belmont, CA, 1981. Borgmann, A., Technology and the Character of Contemporary Life: A Philosophical Inquiry, University of Chicago Press, Chicago, IL, 1984. DeLone, W.H. and McLean, E. R., "Information Systems Success: The Quest for the Dependent Variable," Information Systems Research, (3:1), March 1992, pp. 60-95. Dreyfus, H. and Dreyfus, S. "Why Computers May Never Think Like People," Technology Review, (89:1), pp. 42-61. Gill, T.G., "Early Expert Systems: Where Are They Now?," Management Information Systems Quarterly, (19:1), March 1995, pp. 51-81. Hammer, M. and Champy, J. Reengineering the Corporation: A Manifesto for Business Revolution, HarperBusiness, New York, NY, 1993. Johnson, D.G. and Mulvey, J.M., "Accountability and Computer Decision Systems," Communications of the ACM, (38:12), December 1995, pp. 58-64. Watermann, D.A. A Guide to Expert Systems, Addison Wesley, Reading, MA, 1986. Wittgenstein, L., Philosophical Investigations, Macmillian, New York, 1953.