TweetLevel has been developed by Edelman, one of the world’s leading PR agencies, using a unique algorithm which takes into account the quality and quantity of someone’s tweets, how engaged and trusted a tweeter is, as well as how popular they are.

By entering their Twitter details into Edelman’s free online tool at www.tweetlevel.com, individuals can measure their own importance and rank themselves against a range of factors including influence, engagement, trust and popularity.

TweetLevel allows individuals the opportunity to compare their own importance in the Twittersphere to that of their friends, colleagues and others they choose to ‘follow’.

Well, here – reposted from the original at Jonny Bentwood’s Technobabble blog, is the low down on TweetLevel, how it works, what it’s for and why it matters. May the debate begin.

Picture1

TweetLevel is a new measurement tool that calculates someone’s importance on Twitter. In other words it is the Twitter equivalent to Google’s PageRank.

The explosive growth of Twitter has massively impacted the nature of communications. Initially the domain of the tech-minority, it has now reached a maturity level where it has branched out to multiple demographics.

Picture2

Picture3

(Tables: Nielsen Twitter usage increase Feb 08 to Feb 09 and Nielsen Twitter usage age groups)

My personal view is that as long as Twitter is the tool that key audiences use then I will use it too – if everyone changes next week to product X, then I will too. However, as a communications professional, it is my job to take part in the conversations that  my key audiences are having no matter where they take place.

Jeremiah Owyang of Altimeter Group sums it up:

“I’ll be very clear on this. If you want to influence me, be in a conversation and dialog with me, in person, online, and wherever I go.”

One of the common questions Edelman is asked is ‘who should I spend my limited time with?’. It is this question that has forced companies to tier their audiences depending upon their impact within the wider community. One of the common problems associated with ranking twitter users is the misconception that popularity = influence. This couldn’t be further from the truth. Popularity has its place but it should not be the only metric that defines ‘importance’.

TweetLevel aims to answer this question.

This unique tool compiles twitter data from over 30 sources and feeds the data through an algorithm to rank an individual according to four weightings:

  1. Popularity (i.e. How many people follow you)
  2. Influence (i.e. What you say is interesting, relevant and many people listen)
  3. Engaged (i.e. You actively participate within your community)
  4. Trusted (i.e. People believe what you say)

Of course, the explanation above is a simplified definition of a complex algorithm(the full methodology of this is shown at the bottom of this post).

To illustrate how this works, I have selected four individuals who each excel in different areas. What this shows is not a measure of an individuals ‘importance’ in the world or even social media but simply how they use Twitter.

Popular CNN Breaking News

Picture4

With close to 2.8 million followers this account is exceptionally popular. However, by contrast its engagement score is very low – this is because all the account does is broadcast its opinion (and links) without engaging in conversations with other people. However, without doubt it is exceptionally important as people believe what it says (as shown by its trust score) and its content is retweeted frequently.

InfluentialCalvin Lee

Picture5

Calvin Lee at Mayhem Studios has a high influence score, but a low number of followers as compared to other people in the top 20 TweetLevel names ranked by influence. With less than 40,000 followers Calvin understands that within Twitter it is micro-communities that count not necessarily who can win the popularity contest. Calvin also has a very high engagement score as a result of interactively discussing frequent, relevant and interesting content.

EngagementT-Mobile USA

Picture6

Congratulations to T-Mobile for understanding that the key to using Twitter as a tool is to have two-way conversations that are informative with their community. They have mastered this medium and have been ranked top over a list of millions when it comes to who has the most engaged Twitter account.

Just as PR has gone to public engagement, and AR has gone to analyst engagement, it is only natural that I believe that instead of mass-broadcast (shown by popularity) or mass-amplification (shown by influence), I believe that the purest form of interaction is via multiple targeted micro-conversations where people actively engage and interact with the niche community. This is why when scores are weighted for ‘engagement’ the ‘involvement index’ becomes the most important factor.

Robert Scoble in his recent post discussing the merits of Twitter’s Suggested Users touches on engagement. He explains that the number of followers someone has (i.e. their popularity on Twitter) should be turned off and the answer… engagement.

Turn off follower counts for everyone and come up with a new “engagement score” that is more focused on how you use Twitter and how people engage with you.

Couldn’t agree more which is why TweetLevel has such a high weighting on this metric.

TrustPaulo Coehlo

Picture7

The extra measurement of trust has been included as this factor is the key to action. When people are reviewing how they should act, they may take into consideration someone who is popular, influential or engaged – however, this final metric is the decisive or tipping point. This view is backed-up by the Edelman trust barometer which stated that:

77% of people refused to buy products or services from a company they distrusted

It is this final measure of trust that has been the most discussed during beta testing. After all how do you measure trust on Twitter? Although not the only factor that decides this ranking, in my opinion if someone is prepared to associate what someone else has said with them (via retweeting for example), then this indicates a measure of trust. Whereas some Twitter users achieve this via posting news (such as CNN), others such as Paulo Coehlo have achieved a magnificent trust score by tweeting content that other people wish to spread throughout the twittersphere.

As a final point, I know that when discussing this people tend to be far more interested in ‘influence’ rather than engagement or trust. My view was nicely echoed by AdAge when commenting on Ashton Kutcher’s use of bill board advertising when trying to win the (in)formal ‘first to 1million followers’ against CNN competition. In this piece they quoted a New York commenter who goes by Stevewax:

Seems to me what’s useful with Twitter is creating a small, two-way community with people who aren’t busy running a Twitter team and who have time to SHARE ideas. Rather than broadcast them.

Find out what your TweetLevel is

Simply go to: www.tweetlevel.com now and find out your score. While you are at it, try a few other people and see how you compare to them. I welcome your feedback as to how this can be improved or your comments in general.

Algorithm and Methodology

Picture8

Following – Twitter lists the number of people each user follows. The tendency for most celebrities is to only follow a few individuals. The more people that someone follows, there is an increased likelihood of them actively participating in conversations with the community instead of simply broadcasting to it. Following ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 30) that was used as part of the algorithm. Note: Twitter opened its API to TweetLevel so that data could be sourced easily and quickly to benefit the user.

Followers – Twitter lists the number of people that follow each user. Like subscribing to a feed, this is a clear indication of ‘popularity’ as it requires someone to actively request participation. Even though TweetLevel has a ranking of people based upon popularity, it is influence, engagement and trust that is more important. Due to the nature of logarithmic ranges, a change in the number of people that follow someone, such as from 500 – 1000, will give a far higher change in score than a move from 180K – 200K. Following ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 30) that was used as part of the algorithm. Note: Twitter opened its API to TweetLevel so that data could be sourced easily and quickly to benefit the user.</p>

Twitter Lists – without a doubt this feature addition to Twitter will significantly change the influence score. Even though Twitter has released their API to us, this particular metric is not yet included. When it is, a TweetLevel score will increase depending upon the number of times a user is included in a list, the number of people who follow that list and the authority of those people.

Updates – How often does someone update what they are doing. This number is purely objective as it scores someone highly no matter what the content of their post (i.e. how relevant is it). Nevertheless it is assumed that if someone posts frequently but has poor content then their ‘followers’ will decrease. Update ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 30) that was used as part of the algorithm.

Name Pointing – e.g. @name – How many people engage in conversation with a celebrity or point to their name. The clearest way to establish this is to run a search on the number of people who reference @username in a message. This calculation is based upon a one month period combined with a 24 hour period. The number of times this happens is calculated with each range was assigned a number (0 to 30) – again this was then used as part of the algorithm.

Retweets – Has a tweet caused sufficient interest that it is worth re-submitting by others? Despite a great deal of ‘noise’ (i.e. posts that are not relevant or interesting), when someone sees something that is of high interest, their post can be re-tweeted. The clearest way to establish this is to run a search on the number of people who reference RT @username in a message. This calculation is based upon a one month period combined with a 24 hour period. The number of times this happens is calculated with each range was assigned a number (0 to 50) – again this was then used as part of the algorithm.

Twitalyzer – “This is a unique (and online) tool to evaluate the activity of any Twitter user and report on relative influence, signal-to-noise ratio, generosity, velocity, clout, and other useful measures of success in social media.” This 3rd party tool is a useful method to combine automated metrics dependent upon criteria within posts and publicly available numbers. Where tools such as this are available, we incorporate them into the algorithm to achieve a more confident score. Twitalyzer gives users scores from 0 to 100. Ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Twitalyzer noise to signal ratioSignal-to-noise ratio is a measure of the tendency for people to pass information, as opposed to anecdote. Signal can be references to other people (defined by the use of “@” followed by text), links to URLs you can visit (defined by the use of “http://&#8221; followed by text), hashtags you can explore and participate with (defined by the use of “#” followed by text), retweets of other people, passing along information (defined by the use of “rt”, “r/t/”, “retweet” or “via”). If you take the sum of these four elements and divide that by the number of updates published, you get the “signal to noise” ratio. Twitalyzer gives users scores from 0 to 100. Ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Twinfluence RankTwinfluence is an automated 3rd party tool that uses APIs to measure influence. For example: “Imagine Twitterer1, who has 10,000 followers – most of which are bots and inactives with no followers of their own. Now imagine Twitterer2, who only has 10 followers – but each of them has 5,000 followers. Who has the most real “influence?” Twitterer2, of course.” As with Twitalyzer, this index uses 3rd party tools to add greater confidence in the overall Twitter score. Similar to the other criteria, ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Twitter GraderTwitter Grader is the final automated tool to add greater confidence to the final index. This site creates a score by evaluating a twitter profile. Similar to the other criteria, ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Involvement Index – The Involvement Index is unique Edelman IP that calculates a score based upon how an individual engages with their community. It is calculated by analysing the content of an individual posts. People who score highest in this category have frequent, relevant, high-quality content that actively involved the twitter community (asking questions, posting links or commenting on discussions) and did not purely consist of broadcasting. Ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Velocity Index – As more people engage on Twitter, it may become harder to keep activity going. The velocity index measures changes on a regular basis and assigns a score based on increased or decreased participation. Ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Weighting - Each specific variable listed above was given a standard score out of 10. Using a weighting scale I varied the importance of the each metric to establish an individual’s total score.

Weighted for Popularity – the key variable is the number of people someone has following them. There are many online tools that show this such as Twitterholic.

Weighted for Engagement – the key variables are an individual’s participation with the Twitter community (as measured by the Involvement Index), with additional emphasis on the frequency of people name pointing an individual (via @username), the numbers of followers and the signal to noise ratio. Other attributes were included in the final score but were given a lower weighting.

Weighted for Influence – the key variables in this instance is a combination of the number and authority of someone’s followers together with the frequency of people name pointing an individual (via @username) and the how many times and individuals posts are re-tweeted. Other attributes were included in the final score but were given a lower weighting.

Weighted for trust – the best measure of trust is whether an in individual is will to ‘trust’ what someone else has said sufficiently that they are also prepared to have what they tweeted associated with them. The key metric in this instance are a combination of retweets and number of followers. Other attributes were included in the final score but were given a lower weighting.

In the true spirit of ‘open sourcing’ this work, I welcome your comments, views and criticisms in how this approach can be as accurate as possible. Whereas I don’t believe for one moment that TweetLevel has found the holy grail of social media measurement, I think it is a good step forward and look forward to discussing this with you.

9 Responses to “Tweetlevel”


  1. Can I ask, is the weighting linear? Thanks.


  2. All weighting is logarithmic and variable depending upon whether the key ranking metric you want to analyse is popularity, influence, engagement or trust. For example, if I wanted to find out who was the most popular I would have a far greater weighting on number of followers and lists inclusions/followers-of, whereas I would have less significance placed on how effectively someone communicates with other people.


  3. Interesting. Why logarithmic vs. linear?


  4. The principle behind logarithmic weighting is that the majority of people have a few followers compared to the handful of superstars who have millions. A logarithmic scale will reward the people who have a far more significant rise on the lower end compared to a jump of 1m+ at the high end.


  5. Absolutely, but aren’t you just upweighting what logically are less influential upticks at the lower end. In other words, why are you exagerating an increase of a few hundred at the lower end compared to a few thousand at the upper end?


  6. The simple answer is that if you didn’t use a logarithmic scale then 99% of the twittersphere will have a tweetlevel score of less than 1.00

    By using this scale, it rewards people for overcoming early obstacles when their visibility is very low.

    TweetLevel is not the only mechanism that uses this thought process – just think how a Google PageRank works…


  7. Thanks Jonny, I think the idea that TweetLevel rewards those who are starting out is lovely. But I think my question is ‘being kind to the newbies’ the logic which should define the weighting? Is there any other reason that we should use a logarithmic scale vs. linear?


  8. I think you are missing the point – its not to rewards newbies is that whenever you try and rank anything that has a very large range (from earthquakes, sound and yes twitter influence) then you need to have a logarithmic scale.

    What tweetlevel does is help people find important people within micro communities. I don’t care who has 7 million followers – I care who engages well within a specific niche topic area. If you have a linear scale then important people by area would be impossible to find as they all would have a score of less than 1.00 as the superstars and their huge number of followers would take up 99% of the scale despite them only accounting for less than 1% of the twittersphere.


  9. Quite possibly I am, it’s much easier to talk about log functions face-to-face!

    I think you are right, there is some precedent for the use of the log for large scales, which seems in some part to be based around Steven’s power law: http://en.wikipedia.org/wiki/Stevens%27_power_law

    But I would also argue that there are many examples of large scales which are kept linear (e.g. time, distance, currency).

    I think it is an interesting assumption to make (and gut feel probably the right one), but I guess I am looking for some extra ideas as to why the relationship between the overall influencer score and the raw data should be non-linear…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 30 other followers