adamyntium

My experiment testing user engagement on Facebook, Friendfeed, and Twitter

What did I do?!
I posted an identically-phrased note on Facebook, Friendfeed, and Twitter at around 1:30am PDT Friday morning. Specifically, I posted this: “Could you kindly help me with a super-quick experiment (takes less than 30 seconds)? I’ll share results šŸ™‚ Thanks!”

Why?
I was curious to see which set of friends/subscribers (henceforth referred to as “contacts”) would be more apt to read my note and reply.


What happened?
As of nearly 40 hours after posting…

So does this mean Facebook is better than Friendfeed and Twitter?
No. Is a particular service a better fit or a more powerful promotional vehicle for some people or for some needs or interests? Probably. But my experiment doesn’t prove that. This is based upon my sets of contacts, and was limited to a single test. I know it would make for a far more popular blog entry to trumpet this with a title of “[servicename] the [other servicename] Killer?!?!?” or “[servicename] Beats the Pants Off [other service name]” or — best yet — “[servicename] Set to Trounce Google?!?” — but I refuse to support such memes or pageview-increasing tactics. At least until I receive a very lucratic offer and then decide to sell-out :D.

So what does your little test suggest?
It means that — with my sets of contacts — I’m significantly more likely to get engagement and actions from my contacts on Facebook.

Huh?

When looking at contact interaction, I think we have to take a few things into account:
  • What’s the contact acquisition rate? e.g., how many folks subscribe to / follow you each day?
  • What’s the contact retention rate? How many people stick with you (vs. defriend or unsubscribe)
  • What’s the attention rate? How many actually read what you post?
  • What’s the engagement rate? How many click on your links?
  • And lastly, what’s the action rate? This is just subtly different than engagement, but I mean this to distinguish between clicking on a blog post link and actually posting a comment there.
And, from my limited test and experiences, here’s the more detailed comparison:
  • Acquisition: I’ve found that I acquire contacts on Twitter far more rapidly than on either of the other services.
I’ve been gaining followers at a much faster rate on Twitter than on Friendfeed… typically more than 7-10 a day on Twitter vs. 1 a day on FF. In fact, I even plugged Friendfeed to nearly 100 of my buddies via a friendly (albeit form) e-mail, and got a sum total of zero friends subscribing to my FF feed from that. Bummer :-(. FF’s a much harder sell, at least amongst my non-geek friends, than I previously assumed. Balancing that, though, two friends I personally referred to FF a while back are now two of the service’s more active users :-D. Also, note that my Twitter subscriber count got a HUGE boost because I was recently subscribed to by the wildly popular Google account on Twitter.
  • Retention is a bit harder to assess.
For a while, I used the third party service Twitterless to let me know who unsubscribed from my feed each day. Seemed like I lost about a follower a day on Twitter, which was a little depressing, so I’m glad that feature of Twitterless ceased to function a while back. Though I’ve culled my own Facebook friend list, I haven’t really noticed if/when my friends have unfriended me there. And I’ve also not tracked/noticed people unsubscribing from me on Friendfeed, though I’m sure it’s happened, despite the consistently scintillating quality of my FF posts (HAH!).
  • Attention: Not sure how I could possibly measure that. There’s no user-available “analytics for Friendfeed / Facebook / Twitter” that I know of. Bummer.
  • Engagement, or click-thru rate… in the past, I’ve found that I’ve gotten proportionally the most clicks from Twitter contacts, followed by Facebook and then Friendfeed.
Contemplating Attention and Engagement… I’m guessing that Friendfeed’s generally-helpful/intriguing “Friend of Friend” option could nonetheless be substantially diluting the total attention that feeds-of-friends get. In other words, when someone subscribes to me on Friendfeed, they then start getting (by default) a stream of not only my content, but also the content of my friends’ items I comment on or Like. More to look at means, understandably, attention spread across more items = less time looking at my items. Then again, one could argue that this is balanced out by the fact that people who aren’t subscribed to me are likely to be seeing my items in their feeds when their friends Like or comment upon my entries. Hmm.
I’m not quite sure why Facebook engagement seems to be proportionally so much higher than on the other services, but I think it has to do with the friendship-vs-content orientation of my respective contacts. More specifically, I believe my FB network consists of more strong / moderate friendship ties, whereas people following me on Twitter and FF may be more apt to be reading my stuff because, well, they like my stuff (funny comments, links) vs. liking me personally. So given this, when I asked for a quick favor, it makes sense that I’d get a higher response rate from friends vs. fans.
  • Action is where things get a lot more complicated.
When looking at the magnitude of action — that is to say, getting a single reply (minimal action) vs spawning a lengthy thread of comments (extended action) — then the services are quite different from my experience. On Twitter and Facebook, I’ve found that I quite often get one reply or a small handful replies to my posts. On Friendfeed, more of my posts go without any comments, but… on Friendfeed, I’m more likely to see a post get a large collection of comments. This isn’t surprising to me. Though Facebook has moved more towards facilitating a Friendfeed style of item+comment, Friendfeed’s been IMHO by far the strongest service in town for conversations. In contrast, on Twitter it’s quite easy to post an @ response, but rather frustrating to follow a conversation. I think this explains why I tend to see more robust conversations on Friendfeed, but more frequent (albeit less voluminous) replies on Facebook and Twitter.
The tone and content of a post also plays a large role in determining the extent of replies for me by service. Examples of post-types that are most likely to elicit replies on the various services (again, for me; your mileage may vary!)

– Facebook: “Having a rotten day, could use a hug!” [expression of emotion, change in personal status, in-joke shared amongst friends]

– Friendfeed: “Whoa, check out this robot who recites poetry! WANT! You, too?” [early link to article highlighting a new geek toy or popular geek meme, profound observations or statements of concern, anything about the Kindle, Apple, or Obama]

– Twitter: “At big electronics store in Japan. Should I buy digital camera here or wait ’til Korea?” [questions that don’t demand a complicated response, simple but unexpected notes (e.g., “Now in Bora Bora for 3 hours!”, “Just got engaged!”)]



General caveats:

  • Interconnection: Lots of interconnection between the services! Twitter is integrated into FF and FB, for instance. However, I don’t import my twitter feed into my Facebook account, and I also immediately deleted the twitter-post in Friendfeed to help mitigate this issue.
  • Facebook UI change: Facebook just switched over to a new format. This could have increased or decreased attention to my link.
  • Timing: The timing wasn’t necessarily optimal. Posting it so late on Thu night meant that — by the time most people accessed their account on the various services — they likely already had a ton to look at… e.g., my post was no longer “fresh” at that point.

Personal caveats/notes:

  • Difference in contact symmetry:
    Anyone can follow me on Twitter and FF (assymetry / self-selection), but I pick (and am picky about) who I friend on FB (due to both its symmetrical friends model and my own preferences).

  • Difference in contact type:
    My contacts on FB are far less geeky than my contacts on the other services. They also tend to be typically personal friends rather than acquaintances or fans. In contrast, my contacts on Twitter seem to be largely online marketers, SEOs, and geeks. Same on FF, but with a much higher emphasis of online marketing / uber-geeky folks who are deeply excited about stuff online. Many of my FB friends just dabble a little bit online and most tend to be buddies from school, work, dance, etc.

  • Why Friendfeed / Facebook / Twitter and not [blah blah blah]?
    Because these are the social networking/broadcasting-type services I predominantly use. I have also tried Friendster, Myspace, Orkut, Tribe, Multiply, Jaiku, and likely many other services I’ve forgotten about, but the three above are the ones I’m active on.

And now for some notes from the respondents:

In addition to the main survey question asking people where they clicked on the link from, I also invited people to leave a freeform comment. I’m not sharing all of them (due to privacy concerns), but have excerpted (and replied to in brackets) some below:

  • Did worry it might implode my computer with malware, but hey, I’m leaving the company in two weeks! šŸ™‚[Yeah, I hadn’t even thought of how my impersonal-sounding click-here request might be misperceived. Wonder if that lowered the clickthru rate?]

  • You’re my hero, Adam!
    [Aw, and you wrote that even before you read this blatheringly long blog post. Hope you still feel the same way :-)]

  • Uh, it’s WAY too soon to be talking engagement — I mean you just confirmed we were FRIENDS yesterday!
    [What if I added you to my Top Friends app list in FB? Would that win your heart?]

  • When you write up the results, please keep emphasizing that these are just your friends and try not to generalize.:-D
    [I hope I’ve suitably emphasized that!]

  • Although I clicked through from FB because I have FB chat turned on in Pidgin and it shows me status updates right there.
    [That’s a very good point. I wasn’t thinking about how use of third party tools could skew this experiment.]
  • Good idea Adam, though I wonder if it may be slightly different results for others. After all, you are “the Adam Lasnik”. [I doubt my micro-celebrity status (in the webmaster world) would affect things one way or the other. Might be responsible for getting me more subscribers on Twitter and Friendfeed, but that’s why I listed proportional results above :-)].
I also got another interesting comment which further accentuates the complication involving the use of third party tools with these services… and also touches upon the frustration of data silos:

Hmm, I think you’re missing a subtlety. I selected Facebook because that was the source of the thing I saw. However, where I actually saw it was in Google Reader. I feel like I spend a lot of time trying to get Facebook stuff *out* of Facebook and into the applications I prefer to use. FB does not really make this as easy as it should be. Also, I usually end up getting stuck with two copies of things when someone, for example, imports their Twitter posts to FB. But at least in Reader I can really quickly scan all the updates in a list, skimming over the duplicate or uninteresting ones. (I just wish I could get a FB feed for a friends sub-list!)

Oh, and Vinny… thanks for the hat! šŸ™‚

Wow, that experiment was neato! Can I do that, too? Should I do it? Are you gonna repeat it to see how things change?!

Yep! Technically. Probably not. Unlikely.

Frankly, I’m guessing my friends would get highly annoyed with me if I identically repeated this experiment, and — worse yet — I bet that people in the webosphere would get really pissed at you (and me) if this experiment was duplicated ad nauseum. So sorry, I’ve got first-mover advantage. Take solace in the fact that I likely won’t get rich and famous from this, though. Unless I’m offered a book deal along the lines of, “A Completely Unscientific Experiment Exploring User Engagement With Three Darlings of the Interwebs — The Untold Story” for one MILLION dollars. But that also seems at least somewhat unlikely.

Thanks for reading, though! And hey, while you’re at it, go subscribe to my Friendfeed and Twitter streams šŸ˜€

* * *Ā 

P.S. — You’re welcome to check out the click-thru data of my original request via bit.ly :-D.

* * *


And now… YOUR turn!

Do my experiences match yours? Do you see similar demographic differences in your friend/follower sets amongst the services? What kind of response rates have YOU seen? Other thoughts?

Posted

in

,

by

Adam Lasnik

Tags:

Comments

16 responses to “My experiment testing user engagement on Facebook, Friendfeed, and Twitter”

  1. Brian T Avatar
    Brian T

    Thereā€™s also an interesting addendum to the ā€œattentionā€ bit (at least I think itā€™s interesting): FBā€™s home page feed doesnā€™t show you everything, whereas Twitterā€™s feed will (and I assume FFā€™s feed does as well, but I have no experience with it).

    With FB, when you post a status update, itā€™s not guaranteed that it will show up on all your contactsā€™ home page feeds.Ā  Your experiment link appeared on mine, but it may not have appeared on all of your contactsā€™.

    That didnā€™t appear to affect FBā€™s status as ā€œwinnerā€ in your test, but I think itā€™s notableā€¦

    Reply
  2. Adam Avatar
    Adam

    Actually, Brian, in the new Facebook UI, it seems to me that *all* my friends status updates are shown on the main page whereas before I noticed exactly what youā€™ve stated, that FB (previously) selected only a subset of friend actions to show.

    Whatā€™s interesting and frustrating for me is that this seems like quite a change for the worse.Ā  Whereas before, FB was seeming to do a pretty darn good job algorithmically filtering my homepage feed for me AND I could still see *all* updates via the Live Feed dropdownā€¦ now, thereā€™s no longer such helpful filtering.Ā  And worse yet, I canā€™t find a way to peruse ā€œSo-and-so is now engagedā€ and related update item types :(.

    Reply
  3. Brian T Avatar
    Brian T

    Yeah, I realised that shortly after I posted.Ā  However, I believe I saw your original status on my home page *before* the UI change filtered out to me.Ā  So itā€™s safe to assume that you really did catch it in mid-transition, with some people seeing your post through the old UI, and some through the new.

    I agree with your assessment of the new homepage.Ā  On the plus side, Iā€™m not missing things I otherwise would miss (the algorithm was decent, but far from perfect for me; Iā€™ve already noticed a few people show up in the new feed who unfortunately rarely showed up in the old), but now weā€™re back to the flood-of-info problem.Ā  I have enough ā€˜friendsā€™ that their updates overflow the homepage pretty quickly, and if I donā€™t check every few hours, Iā€™ll certainly miss stuff I *do* care about amidst the flood of stuff.Ā  (Yeah, I know, I could click on the ā€œMore Postsā€ link at the bottom, butā€¦ I shouldnā€™t have to.)

    Hopefully itā€™ll continue to evolve as it has in the pastā€¦ I wouldnā€™t be surprised if the FB guys work in some more smarts as time goes by.

    Reply
  4. Jaemi Avatar
    Jaemi

    Excellent post! Interesting results.

    At first, I was surprised by the lack of responses you got, but then I realized how easy it is to impact a survey of that sort.

    How busy were your collective contacts during the time the survey ran? Maybe they didnā€™t check any of those sites in that time period.

    How do people interact with said sites? The third party app users might actually have been a lot more likely to respond.

    Entries get so easily lost in Twitter-main when you have a bunch of subscriptions, and while FriendFeed gives you the list option, unless the list is short, it can still be hard to stay as on top of the stream as youā€™d like too. And old or new FB, even keeping up with simple status updates can be hard if your friends are active status users.

    I think what your survey shows really well is how hard it can be to follow someone on -any- service.

    FriendFeed is superb at conversations. But they ebb and flow based on the time the community has to put into them.

    For awhile I was in the top 500 most active users, and it was great. And then I barely had time to look at it at all. When I was more active, I got more activity. As I got busier elsewhere, the activity on my content basically came to a standstill, aside from sometime comments from people who personally like to keep up with what Iā€™m doing/saying.

    I rarely get any Twitter attention. Or if I do, it tends to come from other services. Replies used to come from FriendFeed, but I recently turned off the Twitter imports, since I now follow some ā€œfor funā€ Twitter accounts, and who needed to see all the @replies for my word scrambles?

    Facebook is by nature meant to get people to interact, so I think itā€™s no surprise that the most attention takes place here. The relationships do tend to be more personal. Which is likely to mean more attentiveness. At some point, if you have a friends list full of people you donā€™t interact with, youā€™ll clean up the list, or take them out of your feed, etc.

    I think the other thing your survey showed, or at least your write up on results, is that these services really do have different intents and purposes, and thatā€™s what people use them for. Itā€™s probably not possible to be everything to everyone, and people may not want you to try.

    People who want microblogging already Twitter, or something similar. I saw a lot of anger of FB trying to emulate that functionality. And FF is also very good at what -it- does, and Iā€™m not sure its functionality that needed to be in FB either. Conversations, OK. Likes? Not so sure.

    As these services continue to grow (or not) and change, there will likely be changes to how users interact with them.

    Reply
  5. Eszter Avatar
    Eszter

    Thanks for the write-up and for all the caveats.:-)Ā  Interesting idea. This wouldnā€™t work with too many people replicating it as folks would soon know (or assume to know) what itā€™s about, but this one time it was an interesting little test.

    Reply
  6. Jillian Avatar
    Jillian

    Hi Adam! Are you home now? Because this sounds like something I would do on that gawd-awful long flight back from Tokyo!

    I donā€™t have Twitter and am wondering why I am so reluctant to embrace it. I think I have already saturated my technology tolerance level or something. But your analysis makes me think I should sign up for Twitter and at least give it a try.

    I like your awareness that the tone and content of a post will drive your replies. That is so true.

    Anyway, if you are back in the States – welcome home!

    Reply
  7. Ian Avatar
    Ian

    Interesting stuff! I suspect the Twitter environment may do more to facilitate communication between weak ties in a social web. Facebook may implicitly discourage this, and IMO Friendfeed hasnā€™t built enough traction, but the explosive growth of Twitter combined with the relatively small amount of profile information required could suggest that Twitter ties may stay ā€œweakerā€ for longer. This doesnā€™t mean that I think itā€™s impossible to make real friends on Twitter, rather that there might be a higher ā€œfamiliarity breeds contemptā€ threshold than, say, Myspace. Anyway, much food for thought, thanks!

    Reply
  8. Markus Avatar
    Markus

    Very nice experiment with helpful results, indeed. The implications are awesome. I should really try this twitter stuff šŸ˜€

    Reply
  9. Grinder Avatar
    Grinder

    I havenā€™t tried this, but I would venture to say that I would get a much greater response from facebook than I would from Twitter (Iā€™m not on friendfeed). Most of the people I am friends with on Facebook I actually know personally, or at least I have met them before. On the other hand the majority of the people I follow on Twitter, and who follow me, I have never met before. I basically use Twitter as kind of a personalized news feed, but I use it some for social interaction.

    Reply
  10. Gates Avatar
    Gates

    Those are some interesting results. I think I would get a better response from Twitter. I do a lot of @xxxx on there and I seem to get responses back. Iā€™m not sure how good of a response I would get though if i just wrote an update and left it at that.

    Reply
  11. Murnau Avatar
    Murnau

    Wow! This is a very interesting result. I think this based at the private atmosphere in facebook. For example at xing I will never get such a great result.

    Reply
  12. Adam Avatar
    Adam

    Brian, I also trust that the FB engineers are smart.Ā  Hope the folks that manage them are similarly user-friendly šŸ™‚

    Jaemi, yep, I think there were a zillion factors at play here, any one of which could have substantially (and painfully) skewed the results.Ā  But I think above all, itā€™s started an interesting conversation, and for that Iā€™m happy šŸ˜€

    And yep, it will be fascinating to see changes in how people interact with these individual services in 5 years (if theyā€™re still around).Ā  In particular, I wonder how much the services can shape usersā€™ interests and habits (witness FBā€™s original Newsfeed) and vice versa.

    Eszterā€¦ great point.Ā  And indeed, Iā€™m thankful that it hasnā€™t been repeated, because Iā€™d hate to be the one blamed for starting a hugely annoying trend :-).

    Jillian, yep, Iā€™m home now (and was indeed home when you posted that comment), and very very very belatedly getting back to some of my old blog posts.Ā  Eeek!Ā  The Internet is too distracting.Ā  And darn job, taking up my free time šŸ˜›

    Ian, very good point.Ā  The function of weak ties vs. strong ties undeniably plays a role, and I agree with you that Twitter likely facilitates weak ties more than, say, Facebook.

    Markus, glad you liked this šŸ˜€Ā  Do try Twitter, but also Friendfeed šŸ˜‰

    Grinder, again and again, this segmentation comes upā€¦ people having different types of friends on each service.Ā  I wonder if eventually, as more people become involved in online social stuff, if the groups will meld, or if the distinctions will become even sharper.

    Gates, not sure either, but per my comment with Eszter, donā€™t try this experiment at home :-P.

    Mikefast, you SHOULD use Friendfeed šŸ˜›

    Murnau, you raise an interesting point; I was pretty Ameri-centric in my test here.Ā  I do know that Xing has quite a following in Europe, though as I understand it, itā€™s more business than social, right?

    APfB, yep, I agree that there could certainly be more optimal ways to have tested thisā€¦ but I think with so many variables (time zone differences, overlapping but not identical friends on each service, etc.), itā€™d be very hard to isolate stuff and draw more firm conclusions.Ā  Iā€™d be surprised, though, if at least some folks hadnā€™t been doing more substantive research into this realm than I have :-D.

    Reply
  13. spielo Avatar
    spielo

    this is indeed an awsome experiment. Did you studiy sociology or psychology?

    I was expecting, that twitter is rather a one time thing, means people sign up and check it and thatā€™s it. The same for checking other peopeles messages.

    Facebook is much more about friends and networking.

    Thank you very much for posting!

    Reply
  14. Ismet Dumlupinar Avatar
    Ismet Dumlupinar

    Adam, what a great analysis. I am starting to dig for new experiment for my blog and looking for improvements for social marketing. Looks like Facebook is more popular at Turkey. You can also brigde Facebook and Twitter with some applications which gave you more visitorsā€¦

    Reply
  15. Ismet Dumlupinar Avatar
    Ismet Dumlupinar

    Friend feed doesnā€™t give much traffic options for certain countries.

    Reply
  16. Kymaro Avatar
    Kymaro

    Facebook is undoubtedly the most powerful tool for promotion in terms of social media. Twitter being a close second.

    Reply

What do you think?