April 6th, 2026
What Is a t-Test?: Complete Guide With Formulas (+ Example)
By Zach Perkel · 17 min read
What Is a T-Test?
A t-test is a statistical test used to compare averages and determine whether the difference between them is statistically significant, meaning unlikely to have occurred by chance. It works by checking whether the difference between your groups is large compared to how much your data naturally varies.
Why is a t-test important?
A t-test is important because it gives you a structured way to check whether a difference in your data is likely due to random variation. Here’s where it helps in practice:
Validating business decisions: I’ve seen teams reallocate budgets based on differences in numbers that later turned out to be statistically insignificant. A t-test tells you whether a gap between two groups, like conversion rates, revenue figures, and customer scores, is statistically significant.
Removing subjectivity in comparisons: When you're comparing two campaigns, two product versions, or two time periods, eyeballing the numbers isn't enough. A t-test gives you a calculated result instead of relying only on a judgment call.
Catching false positives early: I find this one underrated. Differences that look meaningful in a small sample can disappear when you account for natural variation. A t-test flags that before you draw the wrong conclusion.
Supporting reproducible analysis: Because a t-test follows a defined formula, your team can run the same test on the same data and get the same result. That makes your findings easier to explain and share across your team.
When to use a t-test
A t-test works best under specific conditions, and knowing when to use it helps you avoid results that may not be reliable. Use a t-test when:
Your sample is relatively small: T-tests are often used for datasets with around 30 observations or fewer, though they can also work with larger samples, especially when the data is close to normally distributed. In my experience, this is the most common use case for teams working with survey data or pilot results.
You're comparing one or two groups: A t-test can compare a single group against a known value or compare two groups against each other. If you have three or more groups, you'll need a different test, like ANOVA.
Your data is numerical: T-tests work with continuous numeric data, like revenue, scores, and measurements. I'd double-check this upfront, because it's an easy one to overlook when you're working with mixed datasets.
Your data is roughly normally distributed: If your data has extreme outliers or a heavily skewed spread, the results may not be reliable. I'd always plot your data first to see whether the values cluster around a central point without too many extreme highs or lows.
You don't know the population standard deviation: In most real-world scenarios, you're working from a sample rather than a full population, which is what t-tests are built for. This is common in business and research contexts I've worked with.
3 Types of t-tests and when to use them (with formulas)
There are 3 types of t-tests, and the right one depends on how many groups you're working with and how your data is structured. Let’s walk through each one below:
One-sample t-test
A one-sample t-test compares the average of a single dataset against a fixed value you've chosen in advance. You’d use this when you have one group and a specific value to test against, like checking whether your team’s average deal size matches an industry benchmark.
The formula is:
t = (x̄ - μ) / (s / √n)
Where:
x̄ is your sample mean
μ is your hypothesized mean
s is your sample standard deviation
n is your sample size
Tip: You can check out our one-sample t-test guide to learn more.
Independent t-test (two-sample t-test)
An independent t-test compares the averages of two separate groups that are not related. I see this come up often in business contexts, like comparing conversion rates between two audience segments or revenue across two store locations.
The formula is:
t = (x̄1 - x̄2) / (Sp × √(1/n1 + 1/n2))
Where:
x̄1 and x̄2 are the means of your two groups
Sp is the pooled standard deviation
n1 and n2 are the sample sizes of each group
Paired t-test (dependent t-test)
A paired t-test compares two sets of measurements from the same group, often taken at two different points in time. Use this when the same subjects appear in both groups, such as measuring employee output before and after a training program.
The formula is:
t = d̄ / (sd / √n)
Where:
d̄ is the mean of the differences between your paired observations
sd is the standard deviation of those differences
n is the number of paired observations
Key Assumptions for Conducting a What your data needs before you run a t-testT-Test
Not all datasets are t-test-ready, and it’s worth confirming your data qualifies before you invest time in the analysis. Here's what to check:
Normal distribution: Your data should follow a roughly bell-shaped curve, meaning most values cluster around the average with fewer values at the extremes. If your data is heavily skewed or has extreme outliers, the t-test results can be less reliable. I’d plot your data first to get a quick visual sense of its shape.
Numerical data: T-tests only work with continuous numeric data, like sales figures, test scores, or response times. They don't work directly with categories or text-based variables.
Independent observations: Each data point in your sample should be independent, meaning one observation shouldn't influence another. If your data points are connected in some way, your results can be unreliable.
Similar variance between groups: For the independent t-test specifically, the spread of data within each group should be roughly similar. A large difference in variance between your two groups can affect your results.
Random sampling: Your sample should be drawn randomly from the larger population you're trying to understand. A non-random sample introduces bias that a t-test doesn’t account for.
How to run a t-test (with examples)
Step 1: Collect and prepare your data
Start by gathering your data and organizing it into clearly labeled groups. Make sure it’s numerical, cleanly formatted, and free of errors that could affect your results.
In our example, you've run two ad campaigns over the same 30-day period and recorded the daily conversion rate for each one, giving you two sets of 20 daily observations.
Step 2: Set your hypotheses
Every t-test starts with two hypotheses. Your null hypothesis states that there’s no difference between the groups. Your alternative hypothesis states that a difference does exist.
For our example campaigns, the null hypothesis is that Campaign A and Campaign B have the same average daily conversion rate. The alternative hypothesis is that their averages differ.
Step 3: Choose the right t-test type
Use the conditions we covered earlier to choose the right test. Are you comparing one group against a fixed value, two independent groups, or the same group at two different points in time?
Since we're comparing two separate, unrelated groups of daily observations with no overlap between the people who saw each ad, this is an independent t-test.
Step 4: Calculate your t-statistic
Your t-statistic measures how far apart your two group averages are relative to the natural variation within your data. The larger the t-statistic, the stronger the evidence that the difference between your groups is not due to random variation.
For an independent t-test, plug your numbers into this formula:
t = (x̄1 - x̄2) / (Sp × √(1/n1 + 1/n2))
Tip: You can use a tool like Julius to run the calculation directly from your data. If you don't have a dataset ready, Julius can also search for relevant public data and compile it for you, so you can start with a question instead of a file.
Step 5: Find your p-value
Your p-value tells you how likely it is to see a difference this large if there were actually no difference between your groups. Most analyses use a threshold of 0.05. A p-value below that means your result is considered statistically significant.
In practice, many analysts use statistical software or an online calculator to find the p-value rather than working through a t-distribution table by hand. For our campaigns, you'd input both sets of conversion rate data and let the software calculate it alongside your t-statistic.
Step 6: Interpret your results
Compare your p-value to your significance threshold. This is the cutoff point you set before running the test, and 0.05 is the most commonly used value. A p-value below 0.05 means you can reject the null hypothesis and conclude the difference is statistically significant. A p-value above it means you don’t have enough evidence to conclude there’s a difference between your groups.
In our example, a p-value of 0.03 tells you the gap in conversion rates between the two campaigns is statistically significant, and you can use that finding as one input when deciding which campaign to scale.
Common that can skew your results
T-tests are straightforward once you know the steps, but a few missteps can quietly compromise your results. These are the ones I see come up most often:
Using the wrong t-test type: Each of the three types has a specific use case, and swapping them around can produce results that don't actually answer your question. An independent t-test on paired data, for example, can give you a t-statistic that doesn’t reflect your actual comparison.
Skipping the assumptions check: Running a t-test on data that isn't normally distributed or has unequal variance can lead to misleading results. I'd verify your data meets the conditions we covered earlier before you start.
Treating a small p-value as proof: A p-value below 0.05 is typically considered statistically significant, but it doesn't tell you the difference is large enough to matter in practice. Statistical significance and practical significance aren't the same thing.
Using too small a sample: The smaller your sample, the harder it is for a t-test to detect a difference even when one exists. I’d aim for around 20 to 30 observations per group, where possible.
Ignoring outliers: A single extreme value can pull your group average in a direction that doesn't reflect your data as a whole, which can affect your t-statistic and p-value.
Running multiple t-tests instead of ANOVA: If you're comparing three or more groups, running separate t-tests between each pair increases your chances of a false positive. ANOVA is a better fit for that scenario.
Want to run a t-test without the manual math? Try Julius
T-tests are a lot more manageable when you don't have to work through the formulas by hand. Julius is an AI-powered data analysis tool that can connect to your own data or pull public data from a prompt. We designed it to help you calculate your t-statistic, p-value, and results in plain English, without the spreadsheet gymnastics.
Here’s how Julius helps:
Data search: Type your question, and Julius can search for relevant public data or pull live financial market data for over 17,000 companies through its Financial Datasets integration, so you don’t need a file or database connection to begin.
Direct connections: Link databases like PostgreSQL, Snowflake, and BigQuery, or integrate with Google Ads and other business tools. You can also upload CSV or Excel files. Your analysis can reflect live data, so you’re less likely to rely on outdated spreadsheets.
Repeatable Notebooks: Save an analysis as a notebook and run it again with fresh data whenever you need. You can also schedule notebooks to send updated results to email or Slack.
Smarter over time: Julius includes a Learning Sub Agent, an AI that adapts to your database structure over time. It learns table relationships and column meanings as you work with your data, which can help improve result accuracy.
Quick single-metric checks: Ask for an average, spread, or distribution, and Julius shows you the numbers with an easy-to-read chart.
Built-in visualization: Get histograms, box plots, and bar charts on the spot instead of jumping into another tool to build them.
One-click sharing: Turn an analysis into a PDF report you can share without extra formatting.
Ready to run your next t-test without the manual calculations? Try Julius for free today.
Frequently asked questions
What are some alternatives to a t-test?
The most common alternatives are ANOVA, the Mann-Whitney U test, and the z-test. You'd use ANOVA when you're comparing three or more groups, the Mann-Whitney U test when your data isn't normally distributed, and the z-test when you have a large sample size and already know the population standard deviation.
What's the difference between a one-tailed and two-tailed t-test?
A t-test result tells you whether the difference between your group averages is statistically significant, meaning unlikely to have occurred by chance. It gives you a t-statistic and a p-value. The t-statistic measures the size of the difference relative to the variation in your data, and the p-value tells you how likely that difference is to be random.
