This article includes two simple and easy-to-understand examples to help grasp relevant statistical knowledge.

• Example 1: Hypothesis Testing for the Population Mean (σ² Unknown)

• Example 2: Type I and Type II errors

Continuation from the previous article, we introduced a simple hypothesis testing example using the Single Sample Mean (One-Sample t-Test) which σ² is known to understand the basic steps of hypothesis testing.

Step1. State the Hypotheses:

Step2. Choose the Significance Level and Appropriate Test

Step3. Calculate the Test Statistic

Step4. Make a Decision

1.Three Different Scenarios in Hypothesis Testing

Yes, all hypothesis tests follow these four steps, but beyond the basic steps, there are various different scenarios involving the sample and the population.

For instance, just looking at two-tailed tests, here are three different situations based on the relationship between the sample and the population:

In the previous 2.4 Sample vs Population_Inferential Statistics (3) , we used the Single Sample Mean (One-Sample t-Test). Then, in the context of the Single Sample Mean, we also need to further analyze the conditions of the data, for example...

Criterion 1: Determine if the population variance (σ²) is known.

Criterion 2: Evaluate whether the population follow a normal distribution (N(μ, σ²)) or follow a bivariate normal distribution.

Criterion 3: Consider whether the sample size (n) exceeds 30, distinguishing between large (n > 30) and small samples (n ≤ 30).

2.When to use Z-test vs T-test ?

For the Single Sample Mean (One-Sample t-Test),

To decide between using a Z-Test or a T-Test, consider the following criteria:

• Population Variance Known (σ²) vs. Unknown (s²)

• Population Normality or Large Sample Size

  • Sample Size (Large [n > 30] vs Small [n ≤ 30])

the summary table is as follows:

If it is now a Two-Sample Mean Comparison, additional considerations are needed.

Criterion 4: Assess if the population variances are homogeneous (σ₁² = σ₂²).

Criterion 5: Determine if the population variances are equal (σ₁² = σ₂²).

We will provide a detailed introduction in the next article template.

3.Difference Between one sample z test vs t test formula

Under the conditions where the population variance is unknown and the population is normal, we use the t-distribution for the hypothesis testing of the population mean rather than the standard normal distribution (Z-distribution) used previously.

In this case, we replace σ² in the original formula with s².

4.Example 1: Hypothesis Testing for the Population Mean (σ² Unknown)

Look at the following example:

Now, a sample of 30 teenagers from a certain region has been taken to conduct an IQ test. The data is shown in IQ_Score.csv, and the calculated mean of this sample is 106.64. The question is whether we can consider the average IQ of teenagers in this region to be 108? (Given a significance level α=0.05)

As we can see in the above situation, the population variance is unknown, meaning σ² is unknown.

However, we generally assume that IQ test scores follow a normal distribution. Therefore, we can use the t-distribution. In this case, we need to test the hypothesis H₀: μ = μ₀, that is:

• H₀: μ = 108, meaning the sample is drawn from a population with a mean of μ₀.

• H₁: μ ≠ 108, meaning the sample is not drawn from a population with a mean of μ₀, but from a population with a mean of μ₁.

(Generated by Prompts from Bayeslab.)

In 2.3 Probabilities and Significant level_Inferential Statistics (2)we briefly introduced this.

▶︎ Two-tailed test:

Divides α equally into two parts, with one rejection region on each side. Each rejection region corresponds to a probability of α/2.

▶︎ Left-tailed test:

Places the entire rejection region corresponding to α on the left side.

▶︎ Right-tailed test:

Places the entire rejection region corresponding to α on the right side.

▶︎ Region for acceptance:

The area where the null hypothesis is accepted.

▶︎ Region for rejection:

The area where the null hypothesis is rejected, located outside the acceptance region.

5.What are type I and type II errors in hypothesis testing?

When conducting hypothesis testing, it is not 100% accurate. Generally, the potential errors can be categorized into two types:

• When H₀ is true but we reject H₀, this is called a Type I error (α).

• When H₀ is false but we accept H₀, this is called a Type II error (β).

What is a Type I Error (α)?

As previously introduced, if we draw a sample of size n from a normal population X~N(μ,σ²), the sample mean is almost never expected to fall outside the range of (μ ± 1.96σ/√n).

If it does fall within this range, we call it a "low probability event."

However, "almost never" does not mean "never." The world is full of various random opportunities and coincidences.

So, if we happen to draw a sample and calculate a sample mean X̅ that originally belongs to X~N(μ,σ²) but falls outside the range of (μ ± 1.96σ/√n), meaning it falls into the rejection region, we then say that the mean represented by X̅ is significantly different from μ₀.

This implies that we consider it to be from a different distribution, no longer the original normal population with mean μ₀, but a new normal population with a different mean μ₁.

(Generated by Prompts from Bayeslab.)

In other words, we reject the null hypothesis H₀ and accept the alternative hypothesis H₁.

• When H₀ is true and we reject H₀, the error made in this situation is called a "Type I error."

• Since the probability of making this type of error is the same as the significance level α, it is also called an α error.

What is a Type II Error (β)?

The situation opposite to a Type I error (α) is when the null hypothesis H₀ is false, but we accept the corresponding null hypothesis H₀ and reject the correct alternative hypothesis H₁.

(Generated by Prompts from Bayeslab.)

In this case, the sample is indeed drawn from a population with a mean μ₁, not from a population with a mean μ₀.

However, due to the insufficient difference between the sample mean (μ = 106) and μ₀ = 108, the sample falls into the acceptance region of the null hypothesis H₀. This misjudgment leads to a Type II error (β), also known as a false negative error.

6.Type I and type II errors in statistics examples

A small analogy for Type I error (α) vs Type II error (β) could be:

Assume you are the person who is striving to improve your life and trying to stand out among your peers. Now, let's assume the outcomes of your efforts as below:

• H₀: You are a-millionaire

• H₁: You are not a multi-millionaire

type i and type ii errors table：

Situation 1: You have indeed made an effort and achieved certain success(H₀- null hypothesis is true), but at a class reunion, you happen to cosplay as a clown for your daughter's birthday . At this moment, your classmates mistakenly think you are still struggling, making a living by performing for kids.

Here, your classmates misunderstand the wealthy you and accept the incorrect H₁ alternative hypothesis, committing a Type I error (α), also called a "false positive error."

Situation 2: You have made an effort but without significant results(H₀- null hypothesis is False), still making a living as a designated driver Coincidentally, your classmates see you driving a client's Lamborghini on the road and mistakenly think you are wealthy.

Here, your classmates mistakenly believe you are rich and accept the incorrect H₀ null hypothesis, committing a Type II error (β), also called a "false negative error."

In the context of signal detection theory:

H₀: No signal

H₁: There is a signal

Clearly, a Type I error (α) is a false positive, indicating a signal when there is none, which is a False Alarm.

A Type II error (β) is a false negative, indicating no signal when there is one, which is a Miss.

We must understand that Type I error (α) and Type II error (β) always exist, whether in statistical analysis or in life. Completely eliminating errors is impossible.

We can set a smaller significance level α, for example, changing the original α=0.05 to α=0.01. This can reduce the probability of making a Type I error (α), but correspondingly, it will increase the probability of making a Type II error (β).

To an extreme, what if we set the significance level α to 0? Then the acceptance region would be the entire number line (-∞, +∞). What does this mean?

It means:

• If there is no chance to reject a "correct null hypothesis," there is also no chance to reject an "incorrect null hypothesis."

• Ultimately, right vs wrong, true vs false, goodness vs justice, would all lose their meaning.

This is the true purpose of learning hypothesis testing. We need to accept the inevitability of making mistakes in life through hypothesis testing, to understand and realize that unexpected and expected events are always inevitably coupled. By feasible methods, we can continuously improve the chances of making correct life decisions. However, embracing missteps with wisdom and swiftly adapting paves the path for renewed progress. Even though errors cannot be eliminated, they can be reduced.

In addition to setting the significance level α's value, we can generally also:

• Increase the sample size.

• Properly set the rejection region (two-tailed, left-tailed, right-tailed).

Both methods can to some extent reduce the probability of making a Type II error (β). We will go into more detail on how to set the rejection region in later templates, using AI Block visualization from Bayeslab for a better introduction.

To summarize today's content:

We studied the definitions of Type I and Type II errors:

• Type I error (α):

  • Rejecting a correct null hypothesis and accepting an incorrect alternative hypothesis, also known as an α error.

• Type II error (β):

  • Accepting an incorrect null hypothesis and rejecting a correct alternative hypothesis, also known as a β error.

Stay tuned, subscribe to Bayeslab , and let everyone master the wisdom of statistics at a low cost with AI Agent Online tool.