TweetLevel has been developed by Edelman, one of the world’s leading PR agencies, using a unique algorithm which takes into account the quality and quantity of someone’s tweets, how engaged and trusted a tweeter is, as well as how popular they are.
By entering their Twitter details into Edelman’s free online tool at www.tweetlevel.com, individuals can measure their own importance and rank themselves against a range of factors including influence, engagement, trust and popularity.
TweetLevel allows individuals the opportunity to compare their own importance in the Twittersphere to that of their friends, colleagues and others they choose to ‘follow’.
Well, here – reposted from the original at Jonny Bentwood’s Technobabble blog, is the low down on TweetLevel, how it works, what it’s for and why it matters. May the debate begin.
The explosive growth of Twitter has massively impacted the nature of communications. Initially the domain of the tech-minority, it has now reached a maturity level where it has branched out to multiple demographics.
My personal view is that as long as Twitter is the tool that key audiences use then I will use it too – if everyone changes next week to product X, then I will too. However, as a communications professional, it is my job to take part in the conversations that my key audiences are having no matter where they take place.
Jeremiah Owyang of Altimeter Group sums it up:
“I’ll be very clear on this. If you want to influence me, be in a conversation and dialog with me, in person, online, and wherever I go.”
One of the common questions Edelman is asked is ‘who should I spend my limited time with?’. It is this question that has forced companies to tier their audiences depending upon their impact within the wider community. One of the common problems associated with ranking twitter users is the misconception that popularity = influence. This couldn’t be further from the truth. Popularity has its place but it should not be the only metric that defines ‘importance’.
TweetLevel aims to answer this question.
This unique tool compiles twitter data from over 30 sources and feeds the data through an algorithm to rank an individual according to four weightings:
- Popularity (i.e. How many people follow you)
- Influence (i.e. What you say is interesting, relevant and many people listen)
- Engaged (i.e. You actively participate within your community)
- Trusted (i.e. People believe what you say)
Of course, the explanation above is a simplified definition of a complex algorithm(the full methodology of this is shown at the bottom of this post).
To illustrate how this works, I have selected four individuals who each excel in different areas. What this shows is not a measure of an individuals ‘importance’ in the world or even social media but simply how they use Twitter.
Popular – CNN Breaking News
With close to 2.8 million followers this account is exceptionally popular. However, by contrast its engagement score is very low – this is because all the account does is broadcast its opinion (and links) without engaging in conversations with other people. However, without doubt it is exceptionally important as people believe what it says (as shown by its trust score) and its content is retweeted frequently.
Influential – Calvin Lee
Calvin Lee at Mayhem Studios has a high influence score, but a low number of followers as compared to other people in the top 20 TweetLevel names ranked by influence. With less than 40,000 followers Calvin understands that within Twitter it is micro-communities that count not necessarily who can win the popularity contest. Calvin also has a very high engagement score as a result of interactively discussing frequent, relevant and interesting content.
Engagement – T-Mobile USA
Congratulations to T-Mobile for understanding that the key to using Twitter as a tool is to have two-way conversations that are informative with their community. They have mastered this medium and have been ranked top over a list of millions when it comes to who has the most engaged Twitter account.
Just as PR has gone to public engagement, and AR has gone to analyst engagement, it is only natural that I believe that instead of mass-broadcast (shown by popularity) or mass-amplification (shown by influence), I believe that the purest form of interaction is via multiple targeted micro-conversations where people actively engage and interact with the niche community. This is why when scores are weighted for ‘engagement’ the ‘involvement index’ becomes the most important factor.
Robert Scoble in his recent post discussing the merits of Twitter’s Suggested Users touches on engagement. He explains that the number of followers someone has (i.e. their popularity on Twitter) should be turned off and the answer… engagement.
Turn off follower counts for everyone and come up with a new “engagement score” that is more focused on how you use Twitter and how people engage with you.
Couldn’t agree more which is why TweetLevel has such a high weighting on this metric.
Trust – Paulo Coehlo
The extra measurement of trust has been included as this factor is the key to action. When people are reviewing how they should act, they may take into consideration someone who is popular, influential or engaged – however, this final metric is the decisive or tipping point. This view is backed-up by the Edelman trust barometer which stated that:
77% of people refused to buy products or services from a company they distrusted
It is this final measure of trust that has been the most discussed during beta testing. After all how do you measure trust on Twitter? Although not the only factor that decides this ranking, in my opinion if someone is prepared to associate what someone else has said with them (via retweeting for example), then this indicates a measure of trust. Whereas some Twitter users achieve this via posting news (such as CNN), others such as Paulo Coehlo have achieved a magnificent trust score by tweeting content that other people wish to spread throughout the twittersphere.
As a final point, I know that when discussing this people tend to be far more interested in ‘influence’ rather than engagement or trust. My view was nicely echoed by AdAge when commenting on Ashton Kutcher’s use of bill board advertising when trying to win the (in)formal ‘first to 1million followers’ against CNN competition. In this piece they quoted a New York commenter who goes by Stevewax:
Seems to me what’s useful with Twitter is creating a small, two-way community with people who aren’t busy running a Twitter team and who have time to SHARE ideas. Rather than broadcast them.
Find out what your TweetLevel is
Simply go to: www.tweetlevel.com now and find out your score. While you are at it, try a few other people and see how you compare to them. I welcome your feedback as to how this can be improved or your comments in general.
Algorithm and Methodology
Following – Twitter lists the number of people each user follows. The tendency for most celebrities is to only follow a few individuals. The more people that someone follows, there is an increased likelihood of them actively participating in conversations with the community instead of simply broadcasting to it. Following ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 30) that was used as part of the algorithm. Note: Twitter opened its API to TweetLevel so that data could be sourced easily and quickly to benefit the user.
Followers – Twitter lists the number of people that follow each user. Like subscribing to a feed, this is a clear indication of ‘popularity’ as it requires someone to actively request participation. Even though TweetLevel has a ranking of people based upon popularity, it is influence, engagement and trust that is more important. Due to the nature of logarithmic ranges, a change in the number of people that follow someone, such as from 500 – 1000, will give a far higher change in score than a move from 180K – 200K. Following ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 30) that was used as part of the algorithm. Note: Twitter opened its API to TweetLevel so that data could be sourced easily and quickly to benefit the user.</p>
Twitter Lists – without a doubt this feature addition to Twitter will significantly change the influence score. Even though Twitter has released their API to us, this particular metric is not yet included. When it is, a TweetLevel score will increase depending upon the number of times a user is included in a list, the number of people who follow that list and the authority of those people.
Updates – How often does someone update what they are doing. This number is purely objective as it scores someone highly no matter what the content of their post (i.e. how relevant is it). Nevertheless it is assumed that if someone posts frequently but has poor content then their ‘followers’ will decrease. Update ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 30) that was used as part of the algorithm.
Name Pointing – e.g. @name – How many people engage in conversation with a celebrity or point to their name. The clearest way to establish this is to run a search on the number of people who reference @username in a message. This calculation is based upon a one month period combined with a 24 hour period. The number of times this happens is calculated with each range was assigned a number (0 to 30) – again this was then used as part of the algorithm.
Retweets – Has a tweet caused sufficient interest that it is worth re-submitting by others? Despite a great deal of ‘noise’ (i.e. posts that are not relevant or interesting), when someone sees something that is of high interest, their post can be re-tweeted. The clearest way to establish this is to run a search on the number of people who reference RT @username in a message. This calculation is based upon a one month period combined with a 24 hour period. The number of times this happens is calculated with each range was assigned a number (0 to 50) – again this was then used as part of the algorithm.
Twitalyzer – “This is a unique (and online) tool to evaluate the activity of any Twitter user and report on relative influence, signal-to-noise ratio, generosity, velocity, clout, and other useful measures of success in social media.” This 3rd party tool is a useful method to combine automated metrics dependent upon criteria within posts and publicly available numbers. Where tools such as this are available, we incorporate them into the algorithm to achieve a more confident score. Twitalyzer gives users scores from 0 to 100. Ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.
Twitalyzer noise to signal ratio – Signal-to-noise ratio is a measure of the tendency for people to pass information, as opposed to anecdote. Signal can be references to other people (defined by the use of “@” followed by text), links to URLs you can visit (defined by the use of “http://” followed by text), hashtags you can explore and participate with (defined by the use of “#” followed by text), retweets of other people, passing along information (defined by the use of “rt”, “r/t/”, “retweet” or “via”). If you take the sum of these four elements and divide that by the number of updates published, you get the “signal to noise” ratio. Twitalyzer gives users scores from 0 to 100. Ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.
Twinfluence Rank – Twinfluence is an automated 3rd party tool that uses APIs to measure influence. For example: “Imagine Twitterer1, who has 10,000 followers – most of which are bots and inactives with no followers of their own. Now imagine Twitterer2, who only has 10 followers – but each of them has 5,000 followers. Who has the most real “influence?” Twitterer2, of course.” As with Twitalyzer, this index uses 3rd party tools to add greater confidence in the overall Twitter score. Similar to the other criteria, ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.
Twitter Grader – Twitter Grader is the final automated tool to add greater confidence to the final index. This site creates a score by evaluating a twitter profile. Similar to the other criteria, ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.
Involvement Index – The Involvement Index is unique Edelman IP that calculates a score based upon how an individual engages with their community. It is calculated by analysing the content of an individual posts. People who score highest in this category have frequent, relevant, high-quality content that actively involved the twitter community (asking questions, posting links or commenting on discussions) and did not purely consist of broadcasting. Ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.
Velocity Index – As more people engage on Twitter, it may become harder to keep activity going. The velocity index measures changes on a regular basis and assigns a score based on increased or decreased participation. Ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.
Weighting – Each specific variable listed above was given a standard score out of 10. Using a weighting scale I varied the importance of the each metric to establish an individual’s total score.
Weighted for Popularity – the key variable is the number of people someone has following them. There are many online tools that show this such as Twitterholic.
Weighted for Engagement – the key variables are an individual’s participation with the Twitter community (as measured by the Involvement Index), with additional emphasis on the frequency of people name pointing an individual (via @username), the numbers of followers and the signal to noise ratio. Other attributes were included in the final score but were given a lower weighting.
Weighted for Influence – the key variables in this instance is a combination of the number and authority of someone’s followers together with the frequency of people name pointing an individual (via @username) and the how many times and individuals posts are re-tweeted. Other attributes were included in the final score but were given a lower weighting.
Weighted for trust – the best measure of trust is whether an in individual is will to ‘trust’ what someone else has said sufficiently that they are also prepared to have what they tweeted associated with them. The key metric in this instance are a combination of retweets and number of followers. Other attributes were included in the final score but were given a lower weighting.
In the true spirit of ‘open sourcing’ this work, I welcome your comments, views and criticisms in how this approach can be as accurate as possible. Whereas I don’t believe for one moment that TweetLevel has found the holy grail of social media measurement, I think it is a good step forward and look forward to discussing this with you.