Measuring the value of communities of practice

Ed Mitchell over at Platform Neutral picks up on the issue of Return on Investment (ROI) for on-line communities, quite rightly identifying this as growing topic for debate and argument over the coming months. In Ed’s words:

"ROI
for ‘communities’ is going to appear on our horizons for proper this
year. We will have the debate from the sponsors asking if it’s worth
it, and another one about which way around it should be – ie: ROI for
whom? The sponsors, or the participants. Likewise there is much talk
about the ROI of ’social media’, and all of the projects I am working
on have measurement built into them at the strategy level
"

Personally, I haven’t quite cracked this nut for the virtual communities using the IDeA platform (240 of them at the last count). Since many of the communities are closed, self-organising networks, I only really see the platform-wide metrics. So, for example, I can see:

  • Total number of communities
  • Total registered users
  • Total contributors 
  • Members per community
  • Total topics 
  • Threads with responses
  • Responses per thread
  • Participating users
  • Total number of blogs
  • Total posts
  • Total comments
  • Total number of Wikis
  • Total edits
  • Articles per community
  • Total number of documents
  • Total number of document comments
  • Number of Documents per community
  • Total number of messages
  • Number of users sending messages
  • Number of users receiving messages 
  • Total events posted
  • Events per community

I can get the same raw data from each
individual community (but it’s a tedious process doing this for 240 communities!)
– though this does at least give a more accurate indication of the health of
the individual community (where my definition of ‘health’ means
‘activity’), but none of this information can provide me with
a ‘value’ or ROI – no matter how I slice and dice the data.

Hence the need for a more qualitative approach, in the form of membership and user surveys etc. Picking up on Ed’s comment that all of his community projects have measurement built into them at a strategy level – which sounds like the right approach – it should also be noted that even this can be difficult to quantify in terms of an ROI. The IDeA community strategy was developed to "improve local government services". I can infer from the platform statistics that the communities appear to be active, but I cannot yet connect a specific output (service improvement) to the work of any individual community.

So, allowing for the subjective nature of user surveys, I still think they are more likely to give an indication of ROI if not an absolute measure.  If I’m right, the next step is in asking the
right questions!

The Online Community Research Network are doing a survey on this, so please have a read, fill it in, and pass it on. The more responses the better.

2 Comments

  1. KD Paine said:

    ROI for social networks isn’t nearly that complicated. When Ragan started “MyRagan” and Melcrum started their network, I asked both of them how they were going to measure their success and they both said “membership.” I wondered how “membership” translates into revenues, but then both organizations do conferences and the rule for conferences is, the bigger the mailing list the more successful the conference. So the true ROI for those networks, is ultimately tied to the attendance at their conferences. It’s as simple as that. What’s the desired outcome to the organization? and then you take the cost of the network and subtract it from the revenue benefits.

    February 11, 2008
    Reply
  2. Dino said:

    Thanks for the marvelous posting! I seriously enjoyed reading it, you happen to be a great author.I will ensure that I bookmark
    your blog and may come back very soon. I want to encourage you to ultimately continue your great posts,
    have a nice weekend!

    March 9, 2016
    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *