🧢 Coach T's Recap
Not every shift in your data means something. Statistical significance helps you spot the meaningful shifts so you can focus your energy on changes that matter.
When you’re tracking brand health over time, it’s natural to look for movement in the numbers. But how do you know which shifts are meaningful?
That’s where statistical significance comes in.
It sounds technical, but here’s the gist 👇
Statistical significance is a way of knowing whether a change in your brand data is real, or if it could’ve just happened by chance. If something is statistically significant, it means the shift is big enough (and backed by enough data) that it’s worth paying attention to.
How it works
Whenever you view movement in your metrics across any Timeline or Comparison page, Tracksuit runs a statistical test behind the scenes.
It takes into account:
The sample size
The starting value of the metric
The size of the change
With more responses, even small changes can be meaningful. With fewer responses e.g. demographic filtering, the change needs to be bigger to make sure it’s not just normal variation.
If the shift is mathematically meaningful, we flag it in the dashboard as statistically significant so you know it's a signal worth paying attention to.
💡 Tip
Understand meaningful shifts in the data at a glance with our color-coded arrows. Significant uplifts are displayed as green arrows and significant declines are displayed as red arrows.
What to ask yourself when you see a significant shift
When something’s flagged as significant in the dashboard, that’s your cue to get curious.
Here are some smart questions to guide your thinking 👇
What’s changed recently?
Think about any brand or category activity that could have influenced the result, including Milestones you’ve marked in the dashboard.
Has there been a change in media spend, messaging, or targeting?
Was there a competitor move - a new entrant, product launch, or price drop?
Could seasonality or market context be playing a role?
Connecting the shift to real-world context helps turn numbers into narrative.
Which segment is driving the shift?
Zoom into the data to understand who is behind the movement:
Is it driven by a particular age group, region, or income level?
Are you seeing traction with a previously low-engagement group?
Is it loyal customers leaning in or new audiences discovering your brand?
Identifying the source helps inform where to lean in or double down.
Does this align with our strategy?
Now connect it back to your plan.
Is this change in line with the objectives we've been working towards?
Does it validate our recent efforts or suggest a need to pivot?
Are we seeing growth at the right stage of the funnel?
Use these signals to course-correct or continue building momentum.
What if something isn’t significant?
That doesn’t mean it’s not interesting, it just means we don’t have strong enough evidence (yet) to call it a real change. Keep tracking, look for patterns, and use those signals to guide your next steps.
At Tracksuit, we take care of the significance testing so you can stay focused on what matters most: sustainable brand building.