Hierarchical Neural Networks for Partial Diagnosis in Medicine
Автор работы: Пользователь скрыл имя, 06 Апреля 2013 в 21:46, реферат
Краткое описание
Various domains require hierarchical classification. In medicine, learning partial diagnoses can be helpful when
time and information constraints are present. Hierarchical neural networks provide a good means to perform partial
diagnosis. We implemented a hierarchical backpropagation-based model for the domain of thyroid diseases, and com-
pared the results against those of nonhierarchical networks in terms of sensitivities and specificities.
Прикрепленные файлы: 1 файл
Hierarchical Neural Networks for Partial Diagnosis in Medicine
Lucila Ohno-Machado and Mark A. Musen
Section on Medical Informatics, Stanford School of Medicine
Medical School Office Building X-215, Stanford University, Stanford, CA 94305
machado@camis.stanford.edu, musen@camis.stanford.edu
Abstract
Various domains require hierarchical classification. In medicine, learning partial diagnoses can be helpful when
time and information constraints are present. Hierarchical neural networks provide a good means to perform partial
diagnosis. We implemented a hierarchical backpropagation-based model for the domain of thyroid diseases, and com-
pared the results against those of nonhierarchical networks in terms of sensitivities and specificities. In our system,
high-level neural networks filter instances that are relevant for use in specialized neural networks. The hierarchical
model required fewer epochs to be trained and yielded a higher classification rate in the test set than did the nonhier-
archical one. The hierarchical model also had the advantage that fewer data attributes for each instance were required
at higher levels. Therefore, using this model decreases the problem of dealing with missing values, and provides a
framework to establish a parsimonious sequence of tests for diagnosing thyroid diseases.
1. Background
In most real-life situations, medical decision making is done in absence of complete information. Diagnostic tests
may be ordered to decrease uncertainty, but actions take place before all results become available. The actions (which
could be ordering of new tests, or prescribing a treatment) may change the course of the disease. Cases that are
resolved in this initial phase may never be assigned a final diagnosis. Conversely, further investigation may yield a
more precise diagnosis. The diagnostic process is then repeated, until no additional information is necessary. Yet, the
decisions made early in the diagnostic process -- usually in the absence of complete information -- play a key role on
patient outcomes. These decisions are based on partial diagnoses derived from a limited set of observations. Partial
diagnoses are key components in medical reasoning [Pople, 1982], usually consisting of syndromic, rather than etio-
logic, diagnoses.
Thyroid diseases are classified in two major classes: hypothyroidism and hyperthyroidism. Each of these classes
can be further divided according to the etiology of the disease: hypothyroidism can be divided in primary, secondary,
and so on. We have built a computer program to help physicians decide whether a patient has hypothyroidism, hyper-
thyroidism, or normal thyroid function by interpreting the results of the patient's laboratory tests, and defining a par-
tial diagnosis. Such a partial diagnosis may be useful in explaining some of the patient's findings, in helping a
clinician to make decisions regarding what diagnostic tests to order next, and in helping the physician decide which
medications may be appropriate (even though this partial diagnosis may not be sufficient to allow the clinician to
decide on the optimal therapy). The system produces useful results early in the course of the investigation, when only
scarce information is available. In cases where the system determines that the patient’s thyroid function is not normal,
further processing occurs, and a final diagnosis is suggested.
Many taxonomies of diseases (nosologies) are structured in a hierarchical fashion [Gara, Rosenberg, and Gold-
berg, 1992]. This type of classification not only is easier to understand than a flat list of diseases, but also provides a
basis that guides the differential diagnosis. It is therefore natural to use a hierarchical classification system to perform
medical diagnosis. Several authors have used this approach when building medical expert systems, or rule-based sys-
tems [Weiss, Kulikowski, Amarel, and Safir, 1978]. Although performance may be acceptable, problems with expert
systems usually occur during the knowledge-acquisition phase, when a great amount of time is spent on extracting
information from the expert [Forsythe and Buchanan, 1989]. Furthermore, expert judgment may contain biases
[Tversky and Kahneman, 1974], a problem that machine-learning approaches, by extracting information from evi-
dence, may also avoid.
Hierarchies of neural networks are not new. Ballard proposed them as a solution to the problem of building large
networks [Ballard, 1990]. He developed a modification of the backpropagation algorithm to be applied to these hier-
archies; he reported that preliminary computer experiments showed that his approach resulted in a better performance
in large problems in terms of time and accuracy than did the approach that uses the backpropagation algorithm with
several internal levels. He did not report specific results of these studies. Our motivation for using hierarchical net-
works was somewhat different. Although we were concerned with the scaling problem, the objective of our project
was to develop a hierarchical network that would perform partial diagnosis accurately and parsimoniously. We
applied the backpropagation algorithm to a sequence of networks, so that each network was trained in a supervised
way. The first level of this hierarchical system was presented with fewer data attributes than were given to the more
specialized level. The purpose of this first network was to establish a partial diagnosis of hypothyroidism, hyperthy-
roidism, other conditions, or no disease.
Curry and Rumelhart used hierarchical networks to classify mass spectra [Curry and Rumelhart, 1990]. Our sys-
tem is based on their architecture, except that we did not incorporate extra units for representing the degree of confi-
dence in the top-level network’s results. Curry and Rumelhart did not compare the hierarchical system with its
nonhierachical counterpart. Other approaches using neural networks involve preprocessing of data by several statisti-
cal techniques, usually in a nonsupervised manner [Hrycej, 1992]. Frean has proposed a method for constructing the
hierarchical networks dynamically, but concepts associated with each intermediate level did not have a specific mean-
ing, as they do in our system [Frean, 1990]. Alternatives to building a supervised hierarchical classifier outside the
field of neural connectionist systems include piecewise linear machines, as described by Nilsson [1965], and classifi-
cation trees [Breiman, Friedman, Olshen, and Stone, 1984].
Hierarchical classification in medical domains has been done in a few cases. Ash and Hayes-Roth have studied the
use of action-based hierarchies in a surgical intensive-care unit [Ash and Hayes-Roth, 1993], and there are rule-based
systems that rely on hierarchical classification [Weiss, Kulikowski, Amarel, and Safir, 1978].
2. Material and Methods
We used the set of cases of thyroid diseases provided by Quinlan [1987], and distributed by the University of Cal-
ifornia at Irvine [Murphy and Aha, 1992]. It consists of more than 9000 instances, each with 29 attributes. A previous
version of this database was used by Quinlan to show the implementation of decision trees [Quinlan, 1986]. There are
continuous and discrete values, as well as many missing values. Input consists mainly of values for laboratory-test
results. There are 20 classes for output, which can be grouped in at least four superclasses. Data were collected from
1984 to 1987 in an Australian medical institution. Similar data were also used previously in a neural-network imple-
mentation [Schiffmann, Joost, and Werner, 1992]. The authors described the difficulty that the system had in learning
the patterns. They tried different variations of backpropagation, and studied the variability of learning associated with
variation in learning rate and momentum. As in Quinlan's experiments, their problem was just to classify whether or
not the patient had hypothyroidism. The authors were not concerned with learning both partial and final diagnoses.
Weiss also used a similar set of data to compare different machine-learning algorithms in the domain of thyroid dis-
eases, showing that the smaller error rates in the testing set were associated with neural networks of nine hidden units,
trained by a backpropagation variant [Weiss and Kulikowski, 1990].
2.1. Multiple Neural-Network Architecture
Two top-level networks that determined partial diagnoses (triage neural networks) consisted of multilayered per-
ceptrons (MLPs), with inputs provided by the reduced set of data attributes (20 inputs in the case of the first partial
networks), or the complete set of data attributes (23 inputs in the case of the other networks). We varied the number of
input attributes to measure the importance of the three additional attributes to the determination of the partial diagno-
sis. The attributes were laboratory values that could be left out in the first clinical assessment of thyroid diseases (T3,
T4, and TBG). Figure 1 shows the architecture of the triage networks, and Figure 2 shows the architecture of the spe-
cialized network. The complete set of data (23 inputs) was presented to the generic network, in which the final diag-
noses corresponded to output units. Figure 3 shows the architecture of the generic network.
FIGURE 1. Triage network. Inputs are clinical and labora-
tory data; outputs are first partial diagnoses.
TSH is the thyroid-stimulating hormone, and T4U is the
thyroxine resin uptake, T3 is the triiodothyronine, TT4 is
the total thyroxine, and TBG is the thyroxine-binding glob-
ulin.
Hidden
layer
Patient
data
Partial
diagnoses
TSH
T4U
Hyper-
thyroidism
Hypo-
thyroidism
Normal
Other
conditions
Clinical
finding
16
Clinical
finding
1
.
.
.
(5 or 10 units)
.
.
Patients
who will be
evaluated
further
Hidden
layer
Patient
data
Final
diagnoses
TSH
T4U
Clinical
finding
16
Clinical
finding
1
.
.
.
T3
TT4
TBG
.
.
(5 or 10 units)
Normal
Hypo-
Primary
hypothyroid
Compensated
hypothyroid
Secondary
hypothyroid
thyroid
Other
conditions
FIGURE 2. Specialized network for hypothyroidism.
The triage networks in the hierarchical model were trained to distinguish among four classes of diagnoses: (1)
normal, (2) hyperthyroid, (3) hypothyroid, and (4) other conditions. The specialized network was then trained on only
those cases considered hypothyroid by the triage network. It was trained to classify instances in the following final
diagnoses: (1) hypothyroid, not otherwise specified; (2) primary hypothyroid; (3) compensated hypothyroid; and (4)
secondary hypothyroid. In the generic and triage networks, the training set was composed of the first 4000 instances
in the database. The test set was composed of the remaining 5000 instances. In the specialized networks, the training
set was composed of all training instances considered hypothyroid by the corresponding triage network. The test set
was composed of all test instances considered hypothyroid in the corresponding triage network.
FIGURE 3. Generic neural network. Inputs are all labora-
tory data; outputs are final diagnoses. TSH is the thyroid-
stimulating hormone, T4U is the thyroxine resin uptake, T3
is the triiodothyronine, TT4 is the total thyroxine, and TBG
is the thyroxine-binding globulin.
Three triage networks were built in which there were (1) 20 input units and five hidden units, (2) 20 input units
and 10 hidden units, and (3) 23 input units and five hidden units. We built specialized networks with five and 10 hid-
den units, so that we had three systems for the hierarchical model: one with an overall of 20 hidden units, when 20
input units and 10 hidden units were used for the triage network; one with an overall of 10 hidden units, with 20 input
units and five hidden units in the triage network; and one with also an overall of 10 hidden units, but with 23 input
units and five hidden units in the triage network. The generic network was built with 23 input units, 10 hidden units,
and 10 hidden units.
We formatted all input files to provide one output unit for each desired diagnosis in each level of the hierarchies.
Continuous values were preserved, and true-false values were assigned "1" and "0," respectively. Missing values
were assigned to their means, and scaling was done for continuous values to provide inputs in the order of 100 to 10
-
4
. Of all attributes in Quinlan's set, we did not use those that flagged only whether continuous values were present
(e.g., "measurement": true or false), those referring to patients' identification numbers, those determining the referral
center, and the FTI (free thyroxine index). The latter is just the ratio between two other inputs, so the network should
be able to derive it. The ability of neural networks (as opposed to classification trees and other discriminants) to com-
bine inputs in this way was shown by Reibnegger and associates [Reibnegger, Weiss, Werner-Felmayer, Judmaier,
and Wachter, 1991], who dealt with the aminotransferases ratio in a neural net system used to diagnose liver diseases.
Each of the components of the hierarchical system, and the generic network learned using a standard backpropa-
gation algorithm [Rumelhart, Hinton, and Williams, 1986]. We started with a learning rate of 0.5, but we had to
decrease it to 0.01 to achieve a reasonable performance in both the training and the test sets. Since it is known that the
Hidden
layer
Patient
data
Final
diagnoses
TSH
T4U
Hyper-
thyroid
T3 Toxic
Normal
Clinical
finding
16
Clinical
finding
1
.
.
.
T3
TT4
TBG
.
.
(10 units)
Toxic
goiter
Secondarry
toxic
Hypothyroid
Primary
hypothyroid
Compens.
hypothyroid
Secondary
hypothyroid
Other
conditions
FIGURE 4. Number of epochs required.
prevalence of hypothyroid cases in the data set is 92 percent [Weiss and Kulikowski, 1990], this rate was a lower
bound to our classification-rate goal. The parameter momentum was set to 0. Weight updating was done by epochs,
rather than by patterns.The decision about when to stop training was not straightforward. One-hundred percent accu-
racy in the training set was not obtained even after 10,000 epochs in the generic model, which took approximately 48
hours on a shared SunSparc2. Since the total sum of squares (tss) decreased slowly after 10,000 epochs, we decided
to stop training then. By doing so, we could also compare our results to those of Schiffmann. All networks were
trained for 10,000 epochs. Performance of the test set (measured by the tss) was decreasing with the tss of the training
set, so there was no sign of overfitting at this point.
2.2. Evaluation
Although the goal of learning was to minimize the tss, we could not use this measure to evaluate the networks
because a different number of outputs were involved in each system. The classification rate provided a good means to
evaluate the percentage of correctly classified cases (or the accuracy of the classifiers), but in fact has little meaning
in medical practice. Sensitivities (the proportion of a test's true-positive results to all genuinely positive cases) and
specificities (the proportion of a test's true-negative results to all genuinely negative cases) are among the most
important measures of performance for a diagnostic test in medicine [Sox, Blatt, Higgins, and Marton, 1988]. It is
also usual to compare discriminating power of two tests by comparing the areas under each test's receiver operating
characteristic (ROC) curve over the whole range of cut-off values [Centor, 1991]. We compared classification perfor-
mance using sensitivities and specificities.
3. Results
Table 1 shows the classification rates for performing the partial diagnosis of hypothyroidism. We obtained the
numbers corresponding to the generic network by adding the numbers for each of the final diagnoses that can be clas-
sified as hypothyroidism. The classification rates for final diagnoses in the hierarchical systems correspond to the
results of the specialized networks for hypothyroidism. Table 2 shows the results for the specific final diagnosis of
primary hypothyroid. Table 3 shows the results for the final diagnosis of compensated hypothyroid.
Figure 4 shows how the tss decreased with learning in the triage networks, generic networks, and specialized net-
works. Note that it took more time to go through each epoch in the generic network, as compared to the combination
of triage networks and specialized networks, since there are many more diagnoses to be learned in the generic net-
work than in the triage network. The specialized networks were easy to train, since only about 300 instances were fil-
tered in the triage network.
The generic network had the least accuracy in all cases. In fact, this network could indicate compensated hypothy-
roid correctly in only 248 of 394 hypothyroid cases, and did not recognize any other type of hypothyroidism. Sensi-
tivity to primary hypothyroidism was, therefore, zero. Although not shown in the tables, the generic network could
not recognize any case of hyperthyroidism, whereas hierarchical systems with all 23 inputs units provided to the tri-
age network were the most accurate for classifying this superclass (accuracy 96.16 percent, sensitivity 80 percent,
and specificity 96.57 percent). The triage network that contained just 20 input nodes was also unable to recognize
cases of hyperthyroidism. The reason for these results may be that the three inputs that were left out in the 20-input
models were indeed essential to reach a diagnosis of hyperthyroidism.
The generic neural network was unable to learn patterns that were infrequent in the training set. One example of
an infrequent diagnosis was primary hypothyroidism, as mentioned above. The generic network usually yielded high
specificities (98.63, 99.24, and 96 for hypothyroidism, primary hypothyroidism, and compensated hypothyroidism,
respectively), whereas sensitivity for these different categories ranged from zero to 92.53 percent. There was no
important change in accuracy, sensitivity, or specificity when the triage and specialized networks were composed of
10 versus five hidden units.
3. Discussion
The degree of uncertainty in diagnosis depends on the penalty for false positives and false negatives. Therefore,
an accuracy of 99.99 percent may not mean much when high specificity is coupled with low sensitivity, or vice versa.
The sensitivity and specificity have to be considered by the health-care worker when she receives test results. If rare
patterns must not be missed (e.g., a treatable important disease), we can improve our results by lowering the threshold
of some output units, to ensure higher sensitivity. Preliminary results show that this procedure may be important to
calibrate the network to recognize rare, but important, patterns.
We have not considered physician or patient preferences in our system. Also, we have not addressed costs, risks,
severity of illness, or any measure of patients' well-being. We present sensitivities and specificities so that the appro-
priate misclassification penalties can be assigned. It is unrealistic to assume that costs for false positives and false
negatives are the same in the medical domain. For example, we can see in Tables 1 and 3 that, although the generic
network has a lower accuracy than does a simple "guess" that all patients do not have the disease, its sensitivity is far
from zero. Utility measures, although difficult to acquire, may provide additional insight on how to choose the best
classifier for this data set.
It is interesting to investigate whether the results conform the literature concerning thyroid diseases. The fact that
hierarchical systems could learn almost as well with fewer input units confirms certain guidelines of the American
Thyroid Association [Surks, Chopra, Mariash, Nicoloff, and Solomon. 1990] that suggest that few laboratory tests are
necessary to diagnose certain thyroid conditions. We are undertaking further research in this area. The use of back-
ground knowledge to choose which units to leave out for the triage networks needs further investigation. It speeded
up learning, since fewer weights had to be adjusted, yet it did not decrease accuracy. A physician needs to review
cases that were misclassified, to check whether they comprised mainly certain conditions that are more difficult to
diagnose, as in clinical practice.
Another interesting extension would be to use enhancing techniques, such as weight elimination and weight
decay, to improve performance. We also need to do rigorous evaluation before determining whether this classifier can
be useful in clinical settings. Our classification rates were different from those of previous works. Shiffmann reports a
best classification rate of 98.48 percent, whereas Weiss and Kulikowski reported results as good as 98.80 percent. Our
best result to diagnose the superclass hypothyroid was 98.54 percent correct classification with the hierarchical sys-
TABLE 1. Classification rates for superclass hypothyroid (percent)
System
Accuracy
train
Sensitivity
train
Specificity
train
Accuracy
test
Sensitivity
test
Specificity
test
Generic network
95.93
55.34
98.77
95.82
62.94
98.63
Hierarchical system:
20 hidden, 20 + 23 input, units
99.25
96.56
99.43
98.50
91.11
99.13
Hierarchical system:
10 hidden, 20 + 23 input, units
99.12
95.41
99.38
98.54
91.87
99.10
Hierarchical System:
10 hidden, 23 + 23 input, units
99.07
91.22
99.62
98.38
88.57
99.21
No computer-based system:
(assigning not hypothyroid to all)
93.45
0
100
92.12
0
100
TABLE 2. Classification rates for class primary hypothyroid (percent)
System
Accuracy
train
Sensitivity
train
Specificity
train
Accuracy
test
Sensitivity
test
Specificity
test
Generic network
96.55
0
99.38
96.90
0
99.24
Hierarchical system:
20 hidden, 20 + 23 input, units
99.25
85.96
99.64
97.90
68.64
98.61
Hierarchical system:
10 hidden, 20 + 23 input, units
99.27
82.46
99.77
98.38
67.72
99.11
Hierarchical system:
10 hidden, 23 + 23 input, units
98.90
73.68
99.64
98.08
61.86
98.96
No computerized system:
(assigning “not hypothyroid” to all)
97.15
0
100
97.64
0
100
TABLE 3. Classification rates for class compensated hypothyroid (percent)
System
Accuracy
train
Sensitivity
train
Specificity
train
Accuracy
test
Sensitivity
test
Specificity
test
Generic network
95.92
98.63
95.82
95.82
92.53
96.00
Hierarchical system:
20 hidden, 20 + 23 input, units
99.15
93.88
99.35
97.56
80.22
98.54
Hierarchical system:
10 hidden, 20 + 23 input, units
99.00
94.56
99.17
99.08
80.55
98.56
Hierarchical system:
10 hidden, 23 + 23 input, units
98.60
92.52
98.83
97.72
87.31
98.31
No computer-based system:
(assigning not hypothyroid to all)
96.33
0
100
94.64
0
100
tem.
Examples of the importance of partial diagnosis in medicine are common. Partial diagnoses that guide the diag-
nostic and treatment processes are relative. In certain centers, were the constraints on resources are strong, as in pri-
mary-care institutions, what is considered a final and specific diagnosis may correspond to a partial diagnosis in a
more specialized center. Furthermore, different patients may require different levels of refinement of diagnosis. These
issues play an important role in the decision regarding how partial a diagnosis should be. Triage of patients may ben-
efit from the use of hierarchical classification systems. Identifying patients who are suitable for research studies and
practice guidelines, for example, requires selecting individuals from a large population of patients. The degree of
selection accuracy may have a great influence on patients' satisfaction and on health care financial costs.
4. Conclusion
The hypothesis that hierarchical neural networks would perform more accurately than nonhierarchical ones,
because they "filtered" cases to be classified by more refined networks, was confirmed by this experiment. In all
cases, the overall accuracy of the hierarchical system with 20 inputs for the triage network, 23 inputs for the special-
ized network, and five hidden units for both networks was the highest. In one category (compensated hypothyroid-
ism), however, the sensitivity of the hierarchical system was lower than was that of the generic network. The generic
network was slower than the combination of triage and specialized networks.
Use of a hierarchical neural network classification system in a medical domain improved classification rates, com-
pared to nonhierarchical neural network system. In the limited domain of thyroid diseases, the system proved to be
accurate and practical. More important, results derived from this study may be valid across domains, so hierarchical
neural networks may be useful for many diagnostic tasks in which partial diagnoses are necessary.
References
Ash, D., and Hayes-Roth, B. A Comparison of Action-Based Hierarchies and Decision Trees for Real-Time Perfor-
mance. Proceedings of the Eleventh National Conference on Artificial Intelligence, MIT Press, Cambridge, MA,
1993.
Ballard, D. Modular Learning in Hierarchical Neural Networks. In Schwartz, E.L. (ed), Computational Neuroscience,
Bradford, London, 1990.
Centor, R.M. Signal Detectability: The Use of ROC Curves and Their Analyses. Medical Decision Making, 11, 102-
6, 1991.
Curry, B., and Rumelhart, D. MSnet: A Neural Network That Classifies Mass Spectra. Tetrahedron Computer Meth-
odology, 3, 213-37, 1990.
Frean, M. The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks. Neural
Computation, 2, 198-209, 1990.
Forsythe, D.E., and Buchanan, B.G. Knowledge Acquisition for Expert Systems: Some Pitfalls and Suggestions.
IEEE Transactions on Systems, Man, and Cybernetics, 19, 435-42, 1989.
Gara, M.A., Rosenberg, S., and Goldberg, L. DSM-III-R as a Taxonomy. Journal of Nervous and Mental Disease,
180, 11-9, 1992.
Hrycej, T. Modular Learning in Neural Networks. John Wiley and Sons, Inc, 1992.
Tversky, A., and Kahneman, D. Judgment Under Uncertainty: Heuristics and Biases. Science, 185, 1124-31.
Murphy, P.M., and Aha, D.W. UCI Repository of Machine Learning Databases (on-line directory), University of Cal-
ifornia at Irvine, Department of Information and Computer Science, Irvine, CA, 1992.
Nilsson, N. Learning Machines. McGraw-Hill, New York, 1965.
Pople, H.E. Heuristic Methods for Imposing Structure on Ill-Structured Problems: The Structuring of Medical Diag-
nosis. In Szolovits, P. (ed) Artificial Intelligence in Medicine. Westview Press, 1982.
Quinlan, J.R. Induction of Decision Trees. Machine Learning, 1, 81-106, 1986.
Quinlan, J.R. Simplifying Decision Trees. International Journal of Man-Machine Studies, 27, 221-34, 1987.
Reibnegger, G, Weiss, G., Werner-Felmayer, G., Judmaier, G, and Wachter H. Neural Networks as a Tool for Utiliz-
ing Laboratory Information: Comparison with Linear Discriminant Analysis and with Classification and Regression
Trees. Proc. Nat. Acad. Sci. USA, 88, 1146-30, 1991.
Rumelhart, D.E. Hinton, G.E., and Williams, R.J. Learning Internal Representation by Error Propagation. In Rumel-
hart, D.E., and McClelland, J.L. (eds) Parallel Distributed Processing. MIT Press, Cambridge, MA 1986.
Schiffmann, W., Joost, M., and Werner, R. Optimization of the Backpropagation Algorithm for Training Multilayer
Perceptrons. Universitat Koblenz, Technical Report 15/1992, Fachbericht Physik, 1992.
medicineK.I. Medical Decision Making. Butterworth, Stoneham, 1988.
Surks, M.I, Chopra, I.J., Mariash, C.N., Nicoloff, J.T., and Solomon, D.H. American Thyroid Association Guidelines
for Use of Laboratory Tests in Thyroid Disorders. JAMA, 263, 1529-32, 1990.
Weiss, S.M., Kulinowski, C.A., Amarel, S., and Safir, A. A Model-Based Method for Computer-Aided Medical Deci-
sion Making. Artificial Intelligence 11, 145-72, 1978.
Weiss, S. M., and Kulikowski, C.A. Computer Systems That Learn. Morgan Kaufmann, San Mateo, CA 1990.
Информация о работе Hierarchical Neural Networks for Partial Diagnosis in Medicine