🧢 Coach T's Recap
Not every shift in your data is meaningful. Statistical significance helps you understand which movements are real signals versus normal variation, so you can focus on changes that actually matter.
What is statistical significance?
Statistical significance helps you determine whether a change in your brand data is likely to be real or whether it could have happened by chance.
If a result is statistically significant, it means the change is large enough, and supported by enough data, that it’s unlikely to be random. In other words, it’s a shift worth paying attention to.
How statistical significance works in Tracksuit
Whenever you view movement in your metrics across any Timeline or Comparison page, Tracksuit runs a statistical test behind the scenes.
It takes into account:
The sample size
The starting value of the metric
The size of the change
With more responses, even small changes can be meaningful. With fewer responses (for example, when filtering by demographics), changes need to be larger to confidently rule out normal variation.
If a shift is mathematically meaningful, we flag it in the dashboard as statistically significant so you know it's a signal worth paying attention to.
💡 Tip
Understand meaningful shifts in the data at a glance with our color-coded arrows. Significant uplifts are displayed as green arrows and significant declines are displayed as red arrows.
How to interpret a statistically significant change
When something’s flagged as significant in the dashboard, that’s your cue to get curious.
Here are some smart questions to guide your thinking 👇
What’s changed recently?
Connect the data to real-world context.
Have there been changes in media spend, messaging, or targeting?
Did a competitor launch, reposition, or enter the market?
Could seasonality or broader market conditions be influencing the result?
Which segment is driving the shift?
Zoom into the data to understand who is behind the movement:
Is it driven by a specific age group, region, or demographic segment?
Are new audiences discovering your brand, or are existing customers leaning in?
Does this align with our strategy?
Now bring it back to your goals.
Is the movement aligned with what you’ve been trying to achieve?
Does it validate recent activity, or suggest the need to adjust course?
Is growth happening at the right stage of the funnel?
What if something isn’t significant?
That doesn’t mean it’s not interesting, it just means we don’t have strong enough evidence (yet) to call it a real change. Keep tracking, look for patterns, and use these movements as directional signals, not definitive conclusions.
At Tracksuit, we take care of the significance testing so you can stay focused on what matters most: sustainable brand building.