Hints(3) for :

Lab 2: Implementing a Decision Tree Based Learning Agent in Java
http://www.cs.iastate.edu/~cs572/labs/lab2/lab2.html

Out: Nov 11, 2002
Due: Dec 3, 2002

ComS 572 Principles of Aritifical Intelligence
Dimitris Margaritis
Department of Computer Science
Iowa State University

TA, Jie Bao
Dept of Computer Science
Iowa State University
baojie@cs.iastate.edu
http://www.cs.iastate.edu/~baojie
Dec 03, 2002

Reminder: Due time & Grading policy for lab2

- A hard copy report is REQUIRED this time
- please use parameters as: C5 <namesFile> <trainsetFilename> <testsetFilename>. The split program should be separated

Hardcopy of report is due on Dec 4, Thur.
The code is due on 11:59pm Dec 3, Wed.

Grading policy is :
Code:60%
- A readme file for how to compile and run the program: 5%
- The code to randomly split the datasets: 5%
- Can compile and run with no errors or exceptions: 25%
- Gives results on all datasets:15% (5% for each datasets)
- Clear comments in source code: 10%

Report: 40%
- A list of your classes and functions with short description: 5%
- Printed decision trees: 5%
- Do the induced decision trees make sense given what you know about the domains we used? 5%
- Are you able to obtain any interesting insights about these two test domains by looking at the induced trees? 5%
- Do you see any places where pruning might have been useful had we implemented it? Please explain pruning first. 10%
- How do the learned trees compare to the very simple learning algorithm that simply determines the majority class in the training set, and then always guesses that class on the test set? (For this last question, compute the accuracy of this simple algorithm and compare it to the accuracy of C5.0): 10%

also available @ http://www.cs.iastate.edu/~baojie/acad/teach/ai/lab2/2002-11-11_lab2hints.htm

FAQ

Linear and nominal

Q1: In name files(arrhythmia.names), 'linear' means continuous that we need to scan first to decide five bins. 'nominal' means we need to scan first to know all the possible values. Is that correct? Those two keywords are not in the explanation of name file.

Q2: Can you tell me what is a nominal value and how does it differ from linear or discrete values?

A: Yes. nominal means the possible values are enumerable. Look http://www.cs.iastate.edu/~cs572/labs/lab2/arrhythmia.names for possible values for each attribute. You may modify the name file.

Missing Value

Q: I'm a little confused and am hoping you can help...
In the Hints(1) web page you recommend treating "?" in the data as follows...

Missing Value
There are missing values in some datasets. You need to substitute a default values for them. A natural substitution strategy is "majority is default"

But in the lab assignment at
H:\572AI\Lab2\Hints\PRINCIPLES OF ARTIFICIAL INTELLIGENCE.htm we're told to do it differently:

The character '?' indicates that this feature value is missing. We'll handle this as follows: for every feature, add ? as another valid value.

Can we do it either way, or is one preferable? The example output you sent us makes it look like we should do "majority default", but would it make more sense to randomly assign values to the "?"'s with a distribution proportional to the known values?

A: There are three ways to handle missing value

Assign most common value in tree
Assign most common value among training set
Assign probability to each possible value of A

You can use either of them.

Zero InfoGain

Q: My decision tree looks good on the first dataset and has a reasonable accuracy on different random splits. However for the 2nd and 3rd dataset I am runnning into a peculiar problem.

For nodes after a certain depth, none of the attributes can be used for splitting, This is because of all the unused attributes remaining at that node, none of them give any information gain. This is happening because at that node all the unused attributes have the same value in each record. So every attribute's info = goal entropy; hence Information gain = 0. So I cannot split those nodes any further; although they are not yet uniform in their goal class. In this case I am puzzled as to what goal class I should assign these nodes. This happens a few times in Arrhythmia dataset and many times in the audiology dataset.

I have chosen 5 buckets to discretize linear attributes and I replace missing values with majority value for that attribute.

Do you think by using more buckets or with a different replacement policy this problem could be reduced?

A: this can happen. Assuming everything in your implementation is correct, this means there is not enough information in the input features to completely separate the classes. In this case you should not descend any further i.e. do not split on any attribute with info-gain=0. The class people typically assign to the leaf in those cases is the majority one, with ties split randomly. Another possibility is to return a random class label to a (future query) with probability proportional to the size of each class (number of examples having that class label) in that leaf. But the simple majority rule would do fine for this lab's purposes.

Use more buckets may not help.

Logarithms base 2

Q: I just thought this might be useful; Since java.lang.Math defines natural logarithms (base e), here is a simple utility that gives logarithms to base 2.

public static double logBase2(double number){
        double base = 2;
        return Math.log(number)/Math.log(base);
    }

A: Yes, log₂(x) = ln x / ln 2

What's C5?

Q: Can you explain the main difference between ID3 and C5 algorithm?

A: C5 is proposed in 1998 as a enhancement to ID3. Before C5, there were also C4 and C4.5

see http://www.cse.unsw.edu.au/~billw/cs9414/notes/ml/06prop/id3/id3.html
In this project, although the class name is C5, you are actually required to implement ID3

This may also be helpful: http://www.cc.gatech.edu/classes/AY2002/cs4640_spring/decision-trees.pdf

indentifier

Q:I think in the name file for the third dataset, "audiology.standardized.names", we should add ": Label" behind the attribute "identifier" (obviously it is a spelling error using "indentifier").

A:"indentifier" is unique for each instance, So it's a linear attribute (integer).

Actually, it gives no help to build the decision tree, but your algorithm should can handle it - even when the resulting tree hasn't used it as splitting attribute anytime. It's a good test for your C5 about that if it can handle redundant attribute.

Name file format

Q：The parsing is becoming really difficult. I am having problems with the arrhythmia data file. It has datums like "-.1" . To solve that is not a problem, but as a general consideration, can we modify a bit the data and/or names files? If so, are you going to try our code in the data and names files we submit or yours? My parser will work for my versions of them only!

A: You can change the data files if you feel it can simplify your program. Just tell me what you have changed in your report and submit your modified files

Multiple lines in name file

Q: My names parser class works fine on the first two names file but on the last names file it does not get all goal values because of multiple lines in the goal value list. Could it be possible to make the format of all names files uniform such that all discrete values of an attribute occur on the same line to the right of ":"?

That would reduce the burden of writing the parser and enable us to focus more on ID3.

A: Do you mean in the 3rd name file, the classname has multiple lines?

( ClassName: cochlear_unknown,mixed_cochlear_age_fixation,poss_central,mixed_cochlear_age_otitis_media,
        mixed_poss_noise_om,cochlear_age,normal_ear,cochlear_poss_noise,cochlear_age_and_noise,
        acoustic_neuroma,mixed_cochlear_unk_ser_om,conductive_discontinuity,retrocochlear_unknown,
        conductive_fixation,bells_palsy,cochlear_noise_and_heredity,mixed_cochlear_unk_fixation,
        otitis_media,possible_menieres,possible_brainstem_disorder,cochlear_age_plus_poss_menieres,
        mixed_cochlear_age_s_om,mixed_cochlear_unk_discontinuity,mixed_poss_central_om
)

Only a small change to the name file is needed. You can reduce it to one line by deleting CR/LF(carriage return ) Be careful with the buffer size when the single line may be quite long.

Accuracy

Q: I've just realized that my algorithm classifies incorrectly the 100% of the instances of arrhythmia dataset. Is that possible? I have pretty nice results for the others (audiology 45%, Protein: 1.72% and HouseVotes: 6.09%). Please, if that is ok, could you let me know?

A: Yes. It's possible, even Weka only gives 63% on Arrhythmia dataset and 84% on audiology dataset.

Make sense?

Q: You mention in the Grading policy that we should explain if the induced DT make sense given what we know about the domain. Well, the only domain I may now something is the house-votes one, but I do not understand what the atttributes mean. For example, that "inmigration" = "y" means that the vote is in favor or against the what? In favor of more inmigration or less inmigration? I hope you can answer that question. Moreover, for the other datasets I have no Idea about the domain!!

A: I think you just need to answer this question on the first dataset, since nobody knows much about the last two.

As to the meaning of those politics terms, the source of the dataset may have it:

1. Title: 1984 United States Congressional Voting Records Database 2. Source: Congressional Quarterly Almanac, 98th Congress, 2nd session 1984, Volume XL: Congressional Quarterly Inc., ington, D.C., 1985 Donor: Jeff Schlimmer (Jeffrey.Schlimmer@a.gp.cs.cmu.edu) Date: 27 April 1987

However, it may not available to everyone in the libaray since it's too late. Online databases are also available:

- United States Congressional Roll Call Voting Records, 1789-1990: Reformatted Data http://www.icpsr.umich.edu/cgi/archive2.prl?num=9822
- BILL SUMMARY & STATUS IN THE 98th CONGRESS(1983-1984) http://thomas.loc.gov/bss/d098query.html

eg, handicapped-infants actually is H.R.808: A bill to amend the Child Abuse Prevention and Treatment Act to require the National Center on Child Abuse and Neglect to prevent practices in certain health care facilities which permit handicapped infants to die through deliberate neglect, to establish judicial remedies to prevent such practices, and for other purposes.

Try to find other information by yourself.

See how those Donkeis and Elephants abide by our decision tree :)

Sample decision tree

Those are results given by weka decision tree classifier on the 3 datasets. Compare it with your own result and find where your tree need pruning.

Weka (http://www.cs.waikato.ac.nz/~ml/) is very easy to use, what you need to do is convert your datasat into arff format (very easy). You can test your own dataset with weka and do comparison to your program.

Voting

=== Run information ===

Scheme:       weka.classifiers.j48.J48 -C 0.25 -M 2
Relation:     veto
Instances:    279
Attributes:   17
              handicapped-infants
              water-project-cost-sharing
              adoption-of-the-budget-resolution
              physician-fee-freeze
              el-salvador-aid
              religious-groups-in-schools
              anti-satellite-test-ban
              aid-to-nicaraguan-contras
              mx-missile
              immigration
              synfuels-corporation-cutback
              education-spending
              superfund-right-to-sue
              crime
              duty-free-exports
              export-administration-act-south-africa
              ClassName
Test mode:    user supplied test set: 158 instances

=== Classifier model (full training set) ===

J48 pruned tree
------------------

physician-fee-freeze = y
|   adoption-of-the-budget-resolution = y
|   |   anti-satellite-test-ban = y: republican (14.88/0.78)
|   |   anti-satellite-test-ban = n
|   |   |   water-project-cost-sharing = y: democrat (4.71/0.03)
|   |   |   water-project-cost-sharing = n: republican (3.02/1.0)
|   adoption-of-the-budget-resolution = n: republican (85.71/2.1)
physician-fee-freeze = n: democrat (170.67/1.22)

Number of Leaves : 5

Size of the tree : 9

Time taken to build model: 0.14 seconds

=== Evaluation on test set ===
=== Summary ===

Correctly Classified Instances         151               95.5696 %
Incorrectly Classified Instances         7                4.4304 %
Kappa statistic                          0.9074
K&B Relative Info Score              14187.9892 %
K&B Information Score                  134.0959 bits      0.8487 bits/instance
Class complexity | order 0             153.1027 bits      0.969 bits/instance
Class complexity | scheme               33.8895 bits      0.2145 bits/instance
Complexity improvement     (Sf)        119.2132 bits      0.7545 bits/instance
Mean absolute error                      0.0618
Root mean squared error                  0.1901
Relative absolute error                 13.1253 %
Root relative squared error             38.8552 %
Total Number of Instances              158

=== Detailed Accuracy By Class ===

TP Rate   FP Rate   Precision   Recall F-Measure   Class
  0.958     0.048      0.968     0.958     0.963    democrat
  0.952     0.042      0.937     0.952     0.944    republican

=== Confusion Matrix ===

  a b   <-- classified as
92 4 | a = democrat
  3 59 | b = republican

Arrhythmia

=== Run information ===

Scheme:       weka.classifiers.j48.J48 -C 0.25 -M 2
Relation:     arrhythmia
Instances:    278
Attributes:   280
              [list of attributes omitted]
Test mode:    user supplied test set: 174 instances

=== Classifier model (full training set) ===

J48 pruned tree
------------------

Q_wave_channel_AVR <= 92
|   Q_wave_Of_channel_V3 <= -2.5
|   |   Q_wave_channel_DIII <= 24: 3 (9.0/1.0)
|   |   Q_wave_channel_DIII > 24: 15 (2.0/1.0)
|   Q_wave_Of_channel_V3 > -2.5
|   |   Heart_rate <= 57
|   |   |   Q_wave_channel_V4 <= 0
|   |   |   |   R_wave_Of_channel_AVR <= 1.3: 6 (16.08/0.08)
|   |   |   |   R_wave_Of_channel_AVR > 1.3: 1 (2.0/1.0)
|   |   |   Q_wave_channel_V4 > 0: 1 (2.0/1.0)
|   |   Heart_rate > 57
|   |   |   Q_wave_channel_AVF <= 24
|   |   |   |   Heart_rate <= 101
|   |   |   |   |   Rp_wave_Of_channel_V1 <= 0.5
|   |   |   |   |   |   Q_wave_channel_V3 <= 20
|   |   |   |   |   |   |   Rp_wave_channel_DI <= 0
|   |   |   |   |   |   |   |   T_wave_Of_channel_V5 <= 0.3
|   |   |   |   |   |   |   |   |   T_wave_Of_channel_V3 <= 2.5
|   |   |   |   |   |   |   |   |   |   T_wave_Of_channel_DI <= 0.9
|   |   |   |   |   |   |   |   |   |   |   QRSA_Of_channel_V1 <= -32.6: 1 (2.0)
|   |   |   |   |   |   |   |   |   |   |   QRSA_Of_channel_V1 > -32.6: 2 (14.0/1.0)
|   |   |   |   |   |   |   |   |   |   T_wave_Of_channel_DI > 0.9: 1 (3.0)
|   |   |   |   |   |   |   |   |   T_wave_Of_channel_V3 > 2.5: 1 (3.0/2.0)
|   |   |   |   |   |   |   |   T_wave_Of_channel_V5 > 0.3
|   |   |   |   |   |   |   |   |   T_wave_Of_channel_AVF <= 3.3
|   |   |   |   |   |   |   |   |   |   Number_of_intrinsic_deflections_channel_V2 <= 36
|   |   |   |   |   |   |   |   |   |   |   P_wave_Of_channel_V1 <= 0.4: 1 (151.0/9.0)
|   |   |   |   |   |   |   |   |   |   |   P_wave_Of_channel_V1 > 0.4: 16 (4.0/1.0)
|   |   |   |   |   |   |   |   |   |   Number_of_intrinsic_deflections_channel_V2 > 36: 2 (2.0/1.0)
|   |   |   |   |   |   |   |   |   T_wave_Of_channel_AVF > 3.3: 10 (2.0)
|   |   |   |   |   |   |   Rp_wave_channel_DI > 0: 2 (3.0/1.0)
|   |   |   |   |   |   Q_wave_channel_V3 > 20: 2 (2.0/1.0)
|   |   |   |   |   Rp_wave_Of_channel_V1 > 0.5
|   |   |   |   |   |   S_wave_channel_DII <= 20: 1 (3.0/1.0)
|   |   |   |   |   |   S_wave_channel_DII > 20
|   |   |   |   |   |   |   R_wave_Of_channel_AVF <= 2: 1 (2.0/1.0)
|   |   |   |   |   |   |   R_wave_Of_channel_AVF > 2: 10 (18.0/1.0)
|   |   |   |   Heart_rate > 101: 5 (6.0/1.0)
|   |   |   Q_wave_channel_AVF > 24
|   |   |   |   T_wave_Of_channel_AVR <= -1.3
|   |   |   |   |   R_wave_channel_AVF <= 16: 1 (3.92/0.92)
|   |   |   |   |   R_wave_channel_AVF > 16: 8 (2.0/1.0)
|   |   |   |   T_wave_Of_channel_AVR > -1.3
|   |   |   |   |   T_wave_Of_channel_AVR <= 0.8
|   |   |   |   |   |   T_wave_Of_channel_V2 <= 1.3: 10 (2.0)
|   |   |   |   |   |   T_wave_Of_channel_V2 > 1.3: 4 (15.0/1.0)
|   |   |   |   |   T_wave_Of_channel_AVR > 0.8: 2 (4.0)
Q_wave_channel_AVR > 92: 9 (5.0)

Number of Leaves : 25

Size of the tree : 49

Time taken to build model: 3.88 seconds

=== Evaluation on test set ===
=== Summary ===

Correctly Classified Instances         110               63.2184 %
Incorrectly Classified Instances        64               36.7816 %
Kappa statistic                          0.4523
K&B Relative Info Score               8294.2151 %
K&B Information Score                  207.9585 bits      1.1952 bits/instance
Class complexity | order 0             450.1548 bits      2.5871 bits/instance
Class complexity | scheme            42009.649 bits    241.4348 bits/instance
Complexity improvement     (Sf)     -41559.4942 bits   -238.8477 bits/instance
Mean absolute error                      0.0496
Root mean squared error                  0.1953
Relative absolute error                 56.695 %
Root relative squared error             91.978 %
Total Number of Instances              174

=== Detailed Accuracy By Class ===

TP Rate   FP Rate   Precision   Recall F-Measure   Class
  0.837     0.375      0.686     0.837     0.754    1
  0.474     0.065      0.474     0.474     0.474    2
  1         0.006      0.875     1         0.933    3
  1         0.017      0.25      1         0.4      4
  0.833     0.018      0.625     0.833     0.714    5
  0.625     0.012      0.714     0.625     0.667    6
  0         0          0         0         0        7
  0         0.012      0         0         0        8
  0.75      0          1         0.75      0.857    9
  0.296     0.027      0.667     0.296     0.41     10
  0         0          0         0         0        11
  0         0          0         0         0        12
  0         0          0         0         0        13
  0         0          0         0         0        14
  0         0.006      0         0         0        15
  0         0.03       0         0         0        16

=== Confusion Matrix ===

  a b c d e f g h i j k l m n o p   <-- classified as
72 6 0 2 0 1 0 0 0 1 0 0 0 0 0 4 | a = 1
  7 9 0 0 0 0 0 1 0 1 0 0 0 0 0 1 | b = 2
  0 0 7 0 0 0 0 0 0 0 0 0 0 0 0 0 | c = 3
  0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 | d = 4
  1 0 0 0 5 0 0 0 0 0 0 0 0 0 0 0 | e = 5
  3 0 0 0 0 5 0 0 0 0 0 0 0 0 0 0 | f = 6
  1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 | g = 7
  1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | h = 8
  0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 | i = 9
12 3 0 0 2 1 0 1 0 8 0 0 0 0 0 0 | j = 10
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | k = 11
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | l = 12
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | m = 13
  1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | n = 14
  0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 | o = 15
  7 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 | p = 16

Hearing

=== Run information ===

Scheme:       weka.classifiers.j48.J48 -C 0.25 -M 2
Relation:     hearing
Instances:    200
Attributes:   70
              age_gt_60
              air()
              airBoneGap
              ar_c()
              ar_u()
              bone()
              boneAbnormal
              bser()
              history_buzzing
              history_dizziness
              history_fluctuating
              history_fullness
              history_heredity
              history_nausea
              history_noise
              history_recruitment
              history_ringing
              history_roaring
              history_vomiting
              late_wave_poor
              m_at_2k
              m_cond_lt_1k
              m_gt_1k
              m_m_gt_2k
              m_m_sn
              m_m_sn_gt_1k
              m_m_sn_gt_2k
              m_m_sn_gt_500
              m_p_sn_gt_2k
              m_s_gt_500
              m_s_sn
              m_s_sn_gt_1k
              m_s_sn_gt_2k
              m_s_sn_gt_3k
              m_s_sn_gt_4k
              m_sn_2_3k
              m_sn_gt_1k
              m_sn_gt_2k
              m_sn_gt_3k
              m_sn_gt_4k
              m_sn_gt_500
              m_sn_gt_6k
              m_sn_lt_1k
              m_sn_lt_2k
              m_sn_lt_3k
              middle_wave_poor
              mod_gt_4k
              mod_mixed
              mod_s_mixed
              mod_s_sn_gt_500
              mod_sn
              mod_sn_gt_1k
              mod_sn_gt_2k
              mod_sn_gt_3k
              mod_sn_gt_4k
              mod_sn_gt_500
              notch_4k
              notch_at_4k
              o_ar_c()
              o_ar_u()
              s_sn_gt_1k
              s_sn_gt_2k
              s_sn_gt_4k
              speech()
              static_normal
              tymp()
              viith_nerve_signs
              wave_V_delayed
              waveform_ItoV_prolonged
              ClassName
Test mode:    user supplied test set: 26 instances

=== Classifier model (full training set) ===

J48 pruned tree
------------------

tymp() = a
|   age_gt_60 = f
|   |   history_fluctuating = f
|   |   |   static_normal = f: possible_brainstem_disorder (5.0/1.0)
|   |   |   static_normal = t
|   |   |   |   notch_at_4k = f
|   |   |   |   |   history_heredity = f
|   |   |   |   |   |   history_noise = f
|   |   |   |   |   |   |   air() = mild: cochlear_unknown (25.0/3.0)
|   |   |   |   |   |   |   air() = moderate: cochlear_unknown (3.0/1.0)
|   |   |   |   |   |   |   air() = severe: cochlear_unknown (2.0)
|   |   |   |   |   |   |   air() = normal
|   |   |   |   |   |   |   |   notch_4k = f
|   |   |   |   |   |   |   |   |   boneAbnormal = f
|   |   |   |   |   |   |   |   |   |   history_nausea = f: normal_ear (21.0/4.0)
|   |   |   |   |   |   |   |   |   |   history_nausea = t: cochlear_unknown (2.0)
|   |   |   |   |   |   |   |   |   boneAbnormal = t: cochlear_unknown (3.0)
|   |   |   |   |   |   |   |   notch_4k = t: cochlear_unknown (7.0)
|   |   |   |   |   |   |   air() = profound: cochlear_unknown (1.0)
|   |   |   |   |   |   history_noise = t
|   |   |   |   |   |   |   notch_4k = f
|   |   |   |   |   |   |   |   boneAbnormal = f: cochlear_unknown (6.0/1.0)
|   |   |   |   |   |   |   |   boneAbnormal = t: cochlear_poss_noise (2.0)
|   |   |   |   |   |   |   notch_4k = t: cochlear_poss_noise (4.0)
|   |   |   |   |   history_heredity = t: cochlear_noise_and_heredity (2.0)
|   |   |   |   notch_at_4k = t: cochlear_poss_noise (11.0/1.0)
|   |   history_fluctuating = t: possible_menieres (6.0)
|   age_gt_60 = t
|   |   history_noise = f
|   |   |   history_nausea = f: cochlear_age (45.0/2.0)
|   |   |   history_nausea = t: normal_ear (2.0/1.0)
|   |   history_noise = t: cochlear_age_and_noise (22.0/4.0)
tymp() = as
|   bone() = mild: mixed_cochlear_unk_fixation (6.0/1.0)
|   bone() = moderate: conductive_fixation (0.0)
|   bone() = normal: conductive_fixation (6.0)
|   bone() = unmeasured: conductive_fixation (0.0)
tymp() = b
|   age_gt_60 = f
|   |   boneAbnormal = f: otitis_media (2.0)
|   |   boneAbnormal = t: mixed_cochlear_unk_ser_om (3.0)
|   age_gt_60 = t: mixed_cochlear_age_s_om (3.0/1.0)
tymp() = ad
|   airBoneGap = f: mixed_cochlear_unk_discontinuity (2.0)
|   airBoneGap = t: conductive_discontinuity (2.0)
tymp() = c
|   age_gt_60 = f
|   |   ar_c() = normal: mixed_poss_noise_om (0.0)
|   |   ar_c() = elevated: mixed_poss_noise_om (2.0)
|   |   ar_c() = absent: otitis_media (2.0/1.0)
|   age_gt_60 = t: mixed_cochlear_age_otitis_media (3.0)

Number of Leaves : 31

Size of the tree : 52

Time taken to build model: 0.16 seconds

=== Evaluation on test set ===
=== Summary ===

Correctly Classified Instances          22               84.6154 %
Incorrectly Classified Instances         4               15.3846 %
Kappa statistic                          0.7981
K&B Relative Info Score               1815.6376 %
K&B Information Score                   66.7227 bits      2.5663 bits/instance
Class complexity | order 0              88.4247 bits      3.401 bits/instance
Class complexity | scheme             4299.5268 bits    165.3664 bits/instance
Complexity improvement     (Sf)      -4211.1021 bits   -161.9655 bits/instance
Mean absolute error                      0.0201
Root mean squared error                  0.1133
Relative absolute error                 27.5385 %
Root relative squared error             59.514 %
Total Number of Instances               26

=== Detailed Accuracy By Class ===

TP Rate   FP Rate   Precision   Recall F-Measure   Class
  0         0.077      0         0         0        cochlear_unknown
  0         0          0         0         0        mixed_cochlear_age_fixation
  0         0          0         0         0        poss_central
  0         0          0         0         0        mixed_cochlear_age_otitis_media
  0         0          0         0         0        mixed_poss_noise_om
  1         0          1         1         1        cochlear_age
  1         0          1         1         1        normal_ear
  0.5       0          1         0.5       0.667    cochlear_poss_noise
  0.75      0          1         0.75      0.857    cochlear_age_and_noise
  0         0          0         0         0        acoustic_neuroma
  0         0          0         0         0        mixed_cochlear_unk_ser_om
  0         0          0         0         0        conductive_discontinuity
  0         0          0         0         0        retrocochlear_unknown
  0         0.077      0         0         0        conductive_fixation
  0         0          0         0         0        bells_palsy
  0         0          0         0         0        cochlear_noise_and_heredity
  1         0          1         1         1        mixed_cochlear_unk_fixation
  0         0          0         0         0        otitis_media
  0         0          0         0         0        possible_menieres
  0         0          0         0         0        possible_brainstem_disorder
  0         0          0         0         0        cochlear_age_plus_poss_menieres
  0         0          0         0         0        mixed_cochlear_age_s_om
  0         0          0         0         0        mixed_cochlear_unk_discontinuity
  0         0          0         0         0        mixed_poss_central_om

=== Confusion Matrix ===

  a b c d e f g h i j k l m n o p q r s t u v w x   <-- classified as
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | a = cochlear_unknown
  0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 | b = mixed_cochlear_age_fixation
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | c = poss_central
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | d = mixed_cochlear_age_otitis_media
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | e = mixed_poss_noise_om
  0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | f = cochlear_age
  0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | g = normal_ear
  2 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | h = cochlear_poss_noise
  0 0 0 0 0 0 0 0 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 | i = cochlear_age_and_noise
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | j = acoustic_neuroma
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | k = mixed_cochlear_unk_ser_om
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | l = conductive_discontinuity
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | m = retrocochlear_unknown
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | n = conductive_fixation
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | o = bells_palsy
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | p = cochlear_noise_and_heredity
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 | q = mixed_cochlear_unk_fixation
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | r = otitis_media
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | s = possible_menieres
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | t = possible_brainstem_disorder
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | u = cochlear_age_plus_poss_menieres
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | v = mixed_cochlear_age_s_om
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | w = mixed_cochlear_unk_discontinuity
  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | x = mixed_poss_central_om

[Return to Jie Bao's Homepage]