I'm not a statistician by education, I'm a software engineer. Yet statistics comes up a lot. In fact, questions specifically about Type I and Type II error are coming up a lot in the course of my studying for the Certified Software Development Associate exam (mathematics and statistics are 10% of the exam). I'm having trouble always coming up with the right definitions for Type I and Type II error - although I'm memorizing them now (and can remember them most of the time), I really don't want to freeze up on this exam trying to remember what the difference is.
I know that Type I Error is a false positive, or when you reject the null hypothesis and it's actually true and a Type II error is a false negative, or when you accept the null hypothesis and it's actually false.
Is there an easy way to remember what the difference is, such as a mnemonic? How do professional statisticians do it - is it just something that they know from using or discussing it often?
(Side Note: This question can probably use some better tags. One that I wanted to create was "terminology", but I don't have enough reputation to do it. If someone could add that, it would be great. Thanks.)
Since type two means "False negative" or sort of "false false", I remember it as the number of falses.
- Type I: "I falsely think hypothesis is true" (one false)
- Type II: "I falsely think hypothesis is false" (two falses)
When the village boy cried wolf ...
The first error the villagers did (when they believed him) was type 1 error.
The second error the villagers did (when they didn't believe him) was type 2 error.
The boy's cry was alternate hypothesis because a null hypothesis is no wolf ;)
Here's a handy way that happens to have some truth to it.
Young scientists commit Type-I because they want to find effects and jump the gun while old scientist commit Type-II because they refuse to change their beliefs.
(someone comment in a funnier version of that :) )
I was talking to a friend of mine about this and he kicked me a link to the Wikipedia article on type I and type II errors, where they apparently now provide a (somewhat unhelpful, in my opinion) mnemonic. I did, however, want to add it here just for the sake of completion. Although I didn't think it helped me, it might help someone else:
For those experiencing difficulty correctly identifying the two error types, the following mnemonic is based on the fact that (a) an "error" is false, and (b) the Initial letters of "Positive" and "Negative" are written with a different number of vertical lines:
- A Type I error is a false POSITIVE; and P has a single vertical line.
- A Type II error is a false NEGATIVE; and N has two vertical lines.
With this, you need to remember that a false positive means rejecting a true null hypothesis and a false negative is failing to reject a false null hypothesis.
This is by no means the best answer here, but I did want to throw it out there in the event someone finds this question and this can help them.
I make no apologies for posting such a ridiculous image, because that's exactly why it's easy to remember.
Image source: Ellis, P.D. (2010), “Effect Size FAQs,” website http://www.effectsizefaq.com, accessed on 12/18/2014.
You could reject the idea entirely.
Some authors (Andrew Gelman is one) are shifting to discussing Type S (sign) and Type M (magnitude) errors. You can infer the wrong effect direction (e.g., you believe the treatment group does better but actually does worse) or the wrong magnitude (e.g., you find a massive effect where there is only a tiny, or essentially no effect, or vice versa).
See more at Gelman's blog.
I'll try not to be redundant with other responses (although it seems a little bit what J. M. already suggested), but I generally like showing the following two pictures:
I use the "judicial" approach for remembering the difference between type I and type II: a judge committing a type I error sends an innocent man to jail, while a judge committing a type II error lets a guilty man walk free.
Based on the principle of Occam's razor, Type I errors (rejecting the null hypothesis when it is true) are "arguably" worse than Type II errors (not rejecting the null hypothesis when it is false).
If you believe such an argument:
- Type I errors are of primary concern
- Type II errors are of secondary concern
Note: I'm not endorsing this value judgement, but it does help me remember Type I from Type II.
Hurrah, a question non-technical enough so as I can answer it!
"Type one is a con" [rhyming]- i.e. fools you into thinking that a difference exists when it doesn't. Always works for me.
I used to think of it in terms of the usual picture of two Normal distributions (or bell curves). Going left to right, distribution 1 is the Null, and the distribution 2 is the Alternative. Type I (erroneously) rejects the first (Null) and Type II "rejects" the second (Alternative).
(Now you just need to remember that you're not actually rejecting the alternative, but erroneously accepting (or failing to reject) the Null -- i.e. restate everything in the form of the Null. Hey, it worked for me!)
(a bit joke answer I invented just a minute ago)
- A first class person thinks he is always right.
- A second class person thinks he is always wrong.
- The first class person can only make a type I error (because sometimes he will be wrong).
- The second class person can only make a type II error (because sometimes he will be right).
Here is one explanation that might help you remember the difference.
TYPE I ERROR: An alarm without a fire. TYPE II ERROR: A fire without an alarm.
Every cook knows how to avoid Type I Error - just remove the batteries. Unfortunately, this increases the incidences of Type II error. :)
Reducing the chances of Type II error would mean making the alarm hypersensitive, which in turn would increase the chances of Type I error.
Source: A Cartoon Guide to Statistics
My friend came up with this and I thought it was rather brilliant. She said that during the last two presidencies Republicans have committed both errors: President ONE was Bush who commited a type ONE error by saying there were weapons of mass destruction in Iraq when in fact..... Under president TWO, Obama, (some) Republicans are comitting a type TWO error arguing that climate change is a myth when in fact....
Whatever your views on politics or climate change, it's a pretty easy way to remember!!
I am surprised that noone has suggested the 'art/baf' mnemonic. Basically remember that $\alpha$ is the probability of the type I error and $\beta$ is the probability of a type II error (this is easy to remember because $\alpha$ is the 1st letter in the greek alphabet, so goes with the 1st error, $\beta$ is the 2nd letter and goes with the 2nd error). Now remember the word "art" or "$\alpha$rt" says that $\alpha$ is the probability of Rejecting a True null hypothesis and the psuedo word "baf" or "$\beta$af" says that $\beta$ is the probability of Accepting a False null hypothesis.
The "art" portion is fairly acceptable, the "baf" portion suffers from the fact that 1). it is not a real word, and 2). we are not supposed to accept the null, just fail to reject it. But if you can remember "art/baf" and the idea of Reject True is the R and T in art and the a/$\alpha$ links it to the type I error, then it is a pretty good mnemonic.
RAAR 'like a lion'= first part is *R*eject when we should *A*ccept (type I error) second part is *A*ccept when we should *R*eject (type II error)
This is the easiest way to remember it for me :)
Type 1 = Reject : this is a ONE-word expression Type 2 = Do not : this is a TWO-word expression
I remember it by thinking: What's the first thing I do when I do a null-hypothesis significance test? I set the criterion for the probability that I will make a false rejection. Thus, type 1 is this criterion and type 2 is the other probability of interest: the probability that I will fail to reject the null when the null is false. So, 1=first probability I set, 2=the other one.
Memorize “It’s Type I not II where the null is true” as it rhymes and figure the rest out while you are looking at the problem
Since you are making an error Type I - the null is true but you say it isn’t (reject it) - False positive Then Type II is where the null is not True but you say it is (Fail to reject it)- False Negative
Also, it helps to state what your Null and Alternative Hypothesis are BEFORE doing anything else
This is how I remember the difference between Type I and Type II errors
Type I is a false POSITIVE
Type II is a false NEGATIVE
Type I is so POSITIVE it jumps out of bed first, runs downstairs and finds a significant breakfast while Type II is so NEGATIVE it stays in bed all day so when it eventually crawls out all the food is gone. It can never find anything!
Type One error Reject null hypothesis when it is true
Tiny Overly Eager Raccoons Never Hide When It Is Teatime
Type Two Error Accept null hypothesis when it is false
Twelve Tan Elvis's Ate Nine Hams With Intelligent Irish Farmers
To a software engineer: How about associating Type I error (first of the two) with the term "S"erial "N"umber -- you find something "significant" but it's acutally "not." Type II error is just the opposite once you know what Type I error is.
Sometimes reading really old scientific papers help me to understand some ideas behind statistics.
...they identified "two sources of error", namely:
(a) the error of rejecting a hypothesis that should have been accepted, and
(b) the error of accepting a hypothesis that should have been rejected.
Original source: Neyman, J.; Pearson, E.S. (1967) . "On the Use and Interpretation of Certain Test Criteria for Purposes of Statistical Inference, Part I". Joint Statistical Papers. Cambridge University Press. pp. 1–66. http://biomet.oxfordjournals.org/content/20A/1-2/175.full.pdf+html
Here's how I do it: Type I is an Optimistic error. Type II is a Pessimistic error.
O, P: 1, 2. They're alphabetical.
I think that the usual table is confusing because it concatenates negation verbs. I found the following "verdict table" easier to remember an generalize:
H0 (fair) True False Positive False positive True positive Decision Type I error (Gilty) Negative True negative False negative Type II error
- the decision (positive/negative) matches the verdict name
- the verdicts with "false" are the errors
I True II False or I TRue II FAlse or I TR II FA or I T R. II F A or Type I error: True Ho is Rejected. Type II error: False Ho is Accepted.
I True II False