The audience size conundrum: Can you ever know exactly how many users you have?

A hot topic in tech right now is Elon Musk’s on then off purchase of Twitter.

One of the central reasons Musk is giving for attempting to pull out of the transaction is because he claims Twitter has not been truthful about the number of ‘spam accounts’ it has on its platform.

His claims insinuate Twitter is not truthful about the size of its audience.

I’ll leave the spambot investigations to the billionaire detective agency. But Musk’s claims did make me think about an interesting question:

• Can you really know exactly how many users you have?

Let’s do a thought exercise with Twitter

How would you measure the exact number of people that used Twitter in a month?

You could count the number of accounts that were active in that month, that feels like a logical place to start.

But this gives you active accounts per month. Not users. The number of accounts is likely to be far higher than the number of users as Twitter even says you can have as many accounts as you want, but only up to 5 can be linked to a single email address.

Okay so let’s go there next. Let’s count the number of unique email addresses that were active in a given month.

Well that’s probably a bit better, but still not going to give you what you are after. There is no restriction on how many email addresses one person can have. Again we are likely overestimating real people.

So let’s try another route, one that is very commonly used when you see companies talking about active users. Devices.

Let’s count the distinct number of devices that accessed Twitter in a set period. Let’s even be a bit clever here and combine it with email addresses, so the same email address on multiple devices counts as one user.

Again problems. The average household in America has over 7 internet connected ‘screens’. It would be very normal for one person, particularly an engaged Twitter user, to have multiple accounts across multiple devices.

Again we are going to overestimate the number of users we have, and that’s before we even begin to open the spambot can of worms.

We could keep this exercise going all day, but the point I want to make is this:

Counting users is hard. Very often the metrics you hear such as Monthly Active Users (MAUs) used to define audience size are not close to a count of real people at all. Despite the implication of an active user being the equivalent of 1 distinct human person.

Instead these metrics often count some combination of devices, accounts or email addresses. But this will very rarely be the same as real people.

So we can’t know exactly how many people are using a product, but we have good proxies

Does any of this really matter?

Any metric can be useful if we find the right context. Therefore ‘MAUs’ or other KPIs used to communicate audiences numbers can still be insightful as long as we understand their limitations.

Perhaps sometimes it makes sense to look at numbers of accounts. Maybe you want to look at total number of devices for a specific investigation into the scale of a certain product on various platforms.

My point is it probably doesn’t actually matter that we can’t exactly count the number of unique people using a given product or platform, as we can still likely find ways to answer most questions we may have.

What does matter is when you are not transparent about what a specific metric is actually measuring.

As long as we have an understanding of what a KPI is actually measuring, we can then determine their usefulness and limitations for different contexts.

In summary

Can you ever know exactly how many users you have?

Almost certainly not.

So does this really mater?

Not as long as we are aware of the limitations of the proxy measurements being used.

When being presented with any KPI like ‘Monthly Active Users’ it is important to think critically about what the metric you’re being presented is actually measuring.