1. SIKM | Main
  2. Topics

How do you measure successful knowledge sharing? #metrics #knowledge-sharing #value

 

Hi all,

I work at a multinational health technology company. We organize an internal, in-person meeting/event of global/remote scientists to discuss and work on technical projects. An important benefit of the meeting will be building trust, developing the network and understanding who has what knowledge - where the knowledge is. This will help us collaborate throughout the year to support local customers in the markets, as well as bring relevant knowledge into the business.

Besides the technical output (which is tangible), we would like to measure the value of the abovementioned benefit - in order to justify the meeting and support further similar meetings.

Any ideas on how this might be measured in a good way?

Thank you all so much!
Kind regards,
Jasper
 

Jasper, take a look at some of the previous discussions on this topic.  They may be helpful.
 

I'd suggest first defining the goals for the knowledge sharing. Then see how each goal can be translated into a measurable KPI. For example, if the goal is to improve time to completion of projects, you could measure the time it takes to complete projects in groups that practice knowledge sharing vs. in groups that don't.  
(Of course there are other elements that influence time to completion.  Some of them can be eliminated by statistical models.)

On Mon, Oct 4, 2021, 8:32 AM Jasper Lavertu <jlavertu@...> wrote:
Hi all,

I work at a multinational health technology company. We organize an internal, in-person meeting/event of global/remote scientists to discuss and work on technical projects. An important benefit of the meeting will be building trust, developing the network and understanding who has what knowledge - where the knowledge is. This will help us collaborate throughout the year to support local customers in the markets, as well as bring relevant knowledge into the business.

Besides the technical output (which is tangible), we would like to measure the value of the abovementioned benefit - in order to justify the meeting and support further similar meetings.

Any ideas on how this might be measured in a good way?

Thank you all so much!
Kind regards,
Jasper
 

As a start look into Corporate Longitude that I wrie some years ago. Add then the work in Germany on Wissenskapital. 

mån 4 okt. 2021 kl. 19:10 skrev Maria Svoisky Goldberg <arthemis28@...>:

I'd suggest first defining the goals for the knowledge sharing. Then see how each goal can be translated into a measurable KPI. For example, if the goal is to improve time to completion of projects, you could measure the time it takes to complete projects in groups that practice knowledge sharing vs. in groups that don't.  
(Of course there are other elements that influence time to completion.  Some of them can be eliminated by statistical models.)

On Mon, Oct 4, 2021, 8:32 AM Jasper Lavertu <jlavertu@...> wrote:
Hi all,

I work at a multinational health technology company. We organize an internal, in-person meeting/event of global/remote scientists to discuss and work on technical projects. An important benefit of the meeting will be building trust, developing the network and understanding who has what knowledge - where the knowledge is. This will help us collaborate throughout the year to support local customers in the markets, as well as bring relevant knowledge into the business.

Besides the technical output (which is tangible), we would like to measure the value of the abovementioned benefit - in order to justify the meeting and support further similar meetings.

Any ideas on how this might be measured in a good way?

Thank you all so much!
Kind regards,
Jasper
 

Hi
This paper also may be useful:

Mohammad


--
With best wishes
====================
Mohammad Hassanzadeh (Ph.D.)
Professor, Knowledge and Information Science (Knowledge Management)
Vice-Chancellor for Research and Technology Affairs, Faculty of Management and Economics, TMU
Editor-in-chief, Journal of Information Management. stim.qom.ac.ir
Editor-in-chief, International Journal of Digital Content Management (IJDCM). dcm.atu.ac.ir
Managing editor, International Journal of Knowledge Processing Studies (IJKPS) kps.artahub.ir 
Managing editor, International Journal of Learning Spaces Studies (IJLLS) lss.artahub.ir 
Head and Faculty member, Knowledge and Information science Dept.
Faculty of Management and Economics
Tarbiat Modares University, Tehran, Iran
Knowledge Management Senior Consultant
 

When it comes to measuring success, I try to come at it if possible from four different angles:

  1. Numerical: If you have raw numbers (number of views, shares, users of a KM platform, etc.) these can be helpful in their own right, but can also be tested statistically to see if there are correlations with other numbers such as KPIs, financial metrics, etc.

  2. Success Stories: Anecdotal "We saved this much" or "We were able to meet an unrealistic deadline" might be one-off stories, but they help showcase the value of KM, the implication being that there wouldn't have been that same success without it.  So if you have benchmark data already collected, you can say that at least in this case we saved this much time, money, improved accuracy, whatever, over the way things were done previously.

  3. Satisfaction Surveys: It might be good to develop a survey that asks questions such as on a scale of 1-7, "It's easy for me to find the information I need to do my job;" "When I have a question I know who to ask;" etc. -- whatever the focus of your KM system might be.  I have found it very useful to have questions generic enough that they can be used in repeated surveys over time, then set up the survey using something like SurveyMonkey or others that allow for conditional branching.  Then you can use a strategy similar to the Net Promoter Score.  I tend to do it this way:
    • Create survey questions like the ones I described above with a 1-7 rating.
    • For each question, if the response is 6 or 7, follow immediately with an open-ended question like "Why do you think it is so easy for you?"  If the response is a 1 or 2, follow with something like "Why do you think it is so hard?"  This combination is nice, because the numerical score gives you something consistent you can track over time with successive surveys to see if you are improving, while the open-ended responses give you suggestions for improvements to try between now and the next time you survey.
  4. Process Improvements: If you can take the knowledge that is captured in documents and embed it into a process, then the organization has learned without the need for all the individuals in it to learn -- all they need to do is follow the steps in the new process.  And if you have benchmarking data on your process before you improve it, you should be able to quantify the improvement.

Good luck!
Dennis
 

You might consider social network analysis (SNA) to create a baseline social graph and dataset before your get together, do a follow up SNA 6-12 months later, and compare the results. While incremental growth in the number and strength of social ties across a network is a normal result of working together on various projects, an event like the one you’re envisioning will likely accelerate the quantity and strength of connections significantly. 

There are lots of threads and papers available on the topic. 

--
-Tom
--

Tom Short Consulting
TSC
+1 415 300 7457

All of my previous SIKM Posts

 

Jasper - 

There have already been some excellent responses here. I agree with Maria's response - your first step should be clarifying your goals. Define exactly what you need to measure, and be clear on what is "good" or "bad".

Dennis has also stated pretty much what I wanted to say. There is no single best way to measure impacts. The work that I did used a similar blended approach - a mix of quantitative and qualitative techniques, and story collection. We also used the collected feedback to improve our KM processes. Some of these were periodical measures; some were continual.

I described this to some extent in my chapter in "TIMAF Information Management Best Practices – Volume 1" - see Step 11:

http://delarue.net/blog/wp-content/uploads/2011/02/03KeithDeLaRue.pdf

Regards,
 
 - Keith.
--------------------------------------------------------
Keith De La Rue
AcKnowledge Consulting
...acting on knowledge, communication and learning
email: keith@...
phone: +61 418 51 7676
blog: http://acknowledgeconsulting.com/
--------------------------------------------------------
 

On Mon, Oct 4, 2021 at 01:12 PM, Leif Edvinsson wrote: As a start look into Corporate Longitude that I wrote some years ago. And then the work in Germany on Wissenskapital.

Here are links:
 

Hi Jasper,
You might consider a results-based evaluation framework blending quantitative and qualitative elements.  I always liked using, and had some success with 'activity - insight - impact - outcome' as the evaluation elements set against your objectives and factors to be considered.  This framework allows for short-term, mid-term and longer-term results to be highlighted and by virtue of that, creates an impetus to continue the undertaking to identify the post short-term results. Success breeds continuation. 
Best
Paul


Paul McDowall
KnowHowWorks
Ottawa
 

 
I think a four-stage framework will be helping:
A. Defining key performance indicators such as "time saved while making a critical decision" (KPI_a, KPI_b, KPIn)
B. Assigning a weight (w) for each KPI (k), based on its priority  (this can be extracted through a qualitative approach) and calculating the total value based on the assigned weights  ( Σ K*( μ w(s))
C. Setting a threshold for value creation in each knowledge sharing event.
D. Making decisions based on the threshold value. 

Definition of informative and real KPIs are critical and must have their root in the previous achievements and your business objectives.
Regards
Mohammad

On Wed, Oct 6, 2021 at 3:45 PM Stan Garfield <stangarfield@...> wrote:
On Mon, Oct 4, 2021 at 01:12 PM, Leif Edvinsson wrote: As a start look into Corporate Longitude that I wrote some years ago. And then the work in Germany on Wissenskapital.

Here are links:



--
With best wishes
====================
Mohammad Hassanzadeh (Ph.D.)
Professor, Knowledge and Information Science (Knowledge Management)
Vice-Chancellor for Research and Technology Affairs, Faculty of Management and Economics, TMU
Editor-in-chief, Journal of Information Management. stim.qom.ac.ir
Editor-in-chief, International Journal of Digital Content Management (IJDCM). dcm.atu.ac.ir
Managing editor, International Journal of Knowledge Processing Studies (IJKPS) kps.artahub.ir 
Managing editor, International Journal of Learning Spaces Studies (IJLLS) lss.artahub.ir 
Head and Faculty member, Knowledge and Information science Dept.
Faculty of Management and Economics
Tarbiat Modares University, Tehran, Iran
Knowledge Management Senior Consultant
 

You may gain insights from a write-up on ROI of public libraries. They follow a similar approach where you can change your KPI to the ones you find valuable for your organization. The article has many references to studies on this subject and moreover cases.

On Oct 6, 2021, at 23:39, Mohammad Hasanzadeh <hzadehm@...> wrote:

 
I think a four-stage framework will be helping:
A. Defining key performance indicators such as "time saved while making a critical decision" (KPI_a, KPI_b, KPIn)
B. Assigning a weight (w) for each KPI (k), based on its priority  (this can be extracted through a qualitative approach) and calculating the total value based on the assigned weights  ( Σ K*(μ w(s))
C. Setting a threshold for value creation in each knowledge sharing event.
D. Making decisions based on the threshold value. 

Definition of informative and real KPIs are critical and must have their root in the previous achievements and your business objectives.
Regards
Mohammad

On Wed, Oct 6, 2021 at 3:45 PM Stan Garfield <stangarfield@...> wrote:
On Mon, Oct 4, 2021 at 01:12 PM, Leif Edvinsson wrote: As a start look into Corporate Longitude that I wrote some years ago. And then the work in Germany on Wissenskapital.

Here are links:




-- 
With best wishes
====================
Mohammad Hassanzadeh (Ph.D.)
Professor, Knowledge and Information Science (Knowledge Management)
Vice-Chancellor for Research and Technology Affairs, Faculty of Management and Economics, TMU
Editor-in-chief, Journal of Information Management. stim.qom.ac.ir
Editor-in-chief, International Journal of Digital Content Management (IJDCM). dcm.atu.ac.ir
Managing editor, International Journal of Knowledge Processing Studies (IJKPS) kps.artahub.ir 
Managing editor, International Journal of Learning Spaces Studies (IJLLS) lss.artahub.ir 
Head and Faculty member, Knowledge and Information science Dept.
Faculty of Management and Economics
Tarbiat Modares University, Tehran, Iran
Knowledge Management Senior Consultant
 

Hi Stan, thanks for your reply. I indeed searched for similar discussions related to my question prior to posting it. Although very interesting and valuable, I couldn't find something specifically related to the impact of knowledge sharing in a face-to-face meeting on building trust and developing the network. But, again, previous posts are very inspiring as well.
 

Hi All,

Wow, a lot of valuable ideas and suggestions so far.
@Maria, great suggestion to define goals and translate them to measurable KPIs. This is definitely a good starting point and something that we need to do, so thank you.

@Mohammad, thanks for sharing the link to the article (I have seen it before, in my search to solve my challenge, but still I appreciate it). And your 4-stage framework is very insightful and should be part of the solution. Part of the solution because from what I've learnt so far really tells me to also consider qualitative outcomes. 
Like the suggestions from @Dennis, @Keith and @Paul. I think these kinds of blended approaches (with a mix of quantitative and qualitative techniques) might be best to provide a complete (as far as possible) view of the outcome and impact of knowledge sharing. So, thank you Dennis, Keith and Paul.

I am also very interested in the suggestion of @Tom (thank you, Tom) regarding social network theory. Not only for this particular case, but also to analyse, visualize (and utilize) the existing (informal) networks.

@Daan, thank you for your suggestion. Especially the idea of measuring direct, indirect and secondary benefits is very compelling.

@Leif (and Stan), thanks for notifying me about your work (and sharing the hyperlinks). I have not read the book (yet), but based on the reviews it must be quite interesting.

Thank you all for taking the time to reply and for the inspiration!
 

Hi Tom, thanks for your reply. Do you have, by any chance, a good reference to a practical article/case where they describe how to conduct a social network analysis in a multinational organization?

Thanks again. Kind regards,
Jasper
 

Hi Paul, thank you for your reply. Can you share some more details (or provide some tips to other references) about the 'activity - insight - impact - outcome' evaluation framework that you have used?

I appreciate your help.
Kind regards,
Jasper
 

Wow! There are a lot of interesting things here to learn! Thank you all!

If I may say.. What about some practical examples that you can share from your experience? 

For example - to ask people if the knowledge they got from this meeting actually helped them? Did they now know something that they wouldn't know if this meeting wasn't exist? 
Ask them a true question - Would you like to have more meetings like that? or do you think it's waste of time?

I hope it was helpfull :)
Yehuda




‫בתאריך יום ה׳, 7 באוק׳ 2021 ב-17:20 מאת ‪Jasper Lavertu‬‏ <‪jlavertu@...‬‏>:‬

Hi All,

Wow, a lot of valuable ideas and suggestions so far.
@Maria, great suggestion to define goals and translate them to measurable KPIs. This is definitely a good starting point and something that we need to do, so thank you.

@Mohammad, thanks for sharing the link to the article (I have seen it before, in my search to solve my challenge, but still I appreciate it). And your 4-stage framework is very insightful and should be part of the solution. Part of the solution because from what I've learnt so far really tells me to also consider qualitative outcomes. 
Like the suggestions from @Dennis, @Keith and @Paul. I think these kinds of blended approaches (with a mix of quantitative and qualitative techniques) might be best to provide a complete (as far as possible) view of the outcome and impact of knowledge sharing. So, thank you Dennis, Keith and Paul.

I am also very interested in the suggestion of @Tom (thank you, Tom) regarding social network theory. Not only for this particular case, but also to analyse, visualize (and utilize) the existing (informal) networks.

@Daan, thank you for your suggestion. Especially the idea of measuring direct, indirect and secondary benefits is very compelling.

@Leif (and Stan), thanks for notifying me about your work (and sharing the hyperlinks). I have not read the book (yet), but based on the reviews it must be quite interesting.

Thank you all for taking the time to reply and for the inspiration!

 

On Thu, Oct 7, 2021 at 10:23 AM, Jasper Lavertu wrote:
Do you have, by any chance, a good reference to a practical article/case where they describe how to conduct a social network analysis in a multinational organization?
Jasper, look under Resources in Social Network Analysis.
 

Hi Jasper,

My apologies for not replying sooner.  I was offline for a while.

 

I will share a good example from my experience although I cannot give you the specific details of the organization.  At one organization I launched and moderated a CoP on the topic of Project Management (PM).   There were of course additional elements and factors involved in this specific case which I have not included.  I'm attaching a note that describes what some of the ‘evaluation’ framework looked like for that CoP.  It was a highly effective CoP and it continued to have a very useful impact in the organization long after I moved on. 

I hope this is helpful.  Feel free to ask other questions if desired.
Best
Paul

Know How Works
Ottawa

Evaluation Framework for the PMLF.pdf

Evaluation Framework for the PMLF.pdf

Valdis Krebs
 

Jasper,

We have done SNA/ONA projects in large/medium/small orgs, both for-profit, NGO and Govt since the late 1990s.  Visit our web pages for write-ups of various projects — not all clients want their projects shared.


This article, which showcases many clients,  seems to be the most popular amongst both practitioners and academics…


We have trained/mentored > 1000 internal/external analysts/consultants in the practice/application of SNA/ONA.

Enjoy!

Valdis


Valdis Krebs
Orgnet, LLC
http://orgnet.com/about.html
valdis@...
+1 440 331 1222

1 - 20 of 24