KMbeing

Knowledge Mobilization (KMb): Multiple Contributions & Multi-Production Of New Knowledge

Assessing Research Impact

For those of you who follow me on my KMbeing Twitter feed, you may have noticed a flurry of tweeting yesterday afternoon – June 17th, 2011.  My fingers were furiously flying on my laptop as I live-tweeted a presentation by Sandra Nutley, Professor of Public Policy and Management at the University of Edinburgh. She is also Director of the Research Unit for Research Utilisation (RURU) which investigates the use of social science research in public policy and service delivery settings.


Those of us in the Knowledge Mobilization field consider Sandra Nutley to be somewhat of a KMb celebrity. She along with Isabel Walter and Huw Davies wrote the highly-influential and important book Using Evidence: How research can inform public services. (She actually has KTE/KMb groupies who ask her to sign their copy of the book!  Thanks for your signature Sandra! )

Professor Nutley was addressing the Ontario KTE (Knowledge Transfer & Exchange) Community of Practice (CoP) (of which I am a member). I had the opportunity and privilege to mobilize some of her knowledge on Twitter as she presented on the topic Assessing impact of research & Knowledge Transfer & Exchange (KTE) activities. (KTE is another word used to describe the formal process of Knowledge Mobilization (KMb) –  I make the distinction between formal and informal or macro & micro KMb as well as the differences in use of terminology).

It was also a privilege to meet informally for dinner with Sandra after her talk, along with my husband and KMb partner David Phipps from ResearchImpact at York University (@researchimpact on Twitter), and Sarah Morton (@sasmort on Twitter),  Co-Director at the Centre for Research on Families and Relationships (CRFR) at The University of Edinburgh (@CRFRtweets on Twitter).  I also live-tweeted a presentation by Sandra on June 6th, 2011 titled Research Use in Different Contexts. You can expect an upcoming Mobilize This! blog from ResearchImpact on this event soon.

Sandra Nutley began by asking the important question Why assess research impact? Using a forward tracking and back tracking approach, Nutley pointed out the common reasons for the need to assess research impact such as addressing accountability, assuring value for money invested in research, setting priorities for research, assisting with learning and improving outcomes from research.

Within the scope of this forward/back tracking model are the various stakeholders that play a role in the research process: tracked back to ‘user’ communities – such as policymakers, practitioners, the media and other organizational use; while tracking forward with research stemming from either single studies, research programs or systematic reviews.

As part of an evaluation of research initiatives, Nutley reminded the group of the importance of their own KTE or KMb interventions (along with other centres) that play an important role in promoting research when assessing the impact of initiatives and increasing research use.

Nutley also pointed out two common methods used to assess the impact of research:

1: the “payback” model – which focuses more on the value gained from research

OR

2: the “mapping” model – which describes and maps networks and flows of knowledge and the effects of any interactions from research.

(Research Unit for Research Utilisation (RURU) at The University of Edinburgh)

Nutley went on to describe the work being done by RURU to assess research impact which focuses on KTE or KMb intervention types – specifically on the underlying mechanisms involved such as dissemination, interaction, social influence, facilitation, incentives & reinforcement. This created some valued discussion within the room with Peter Levesque, founder and managing Director of Knowledge Mobilization Works asking “how do we best distinguish between the complexity of the research environment & mechanisms used?”  Melanie Barwick, Health Systems Scientist from Sick Kids Hospital, suggested the use of the term mechanism doesn’t always take into consideration the audience or goal for sharing knowledge and is a limiting term.

The presentation continued with Professor Nutley pointing out some common challenges and methodological issues when assessing research impact such as the types and use of research being done, the timing of assessment, the importance of context, along with attribution and additionality to research, and the importance of getting away from linear models (which I have pointed out in a previous blog about the multi-directional flow of knowledge from context to context).

The floor was briefly turned over to Sarah Morton who described an emerging approach to address attribution in research using John Mayne’s work on contribution analysis.  Sarah has been actively involved in applying this approach to her own important and recognized research on families and relationships at CRFR. (I hope Sarah and Sandra will return to Canada to present further on this valuable “contribution” to knowledge mobilization).

Professor Nutley discussed the use of Erica Wimbush’s work on the Theory of Change to assess research impact, showing the direct and indirect control and influences on the process of research – from inputs, activities & outputs (direct control & influence) through the process of reach/engagement, reactions & capacity (direct influence) to ongoing practice, behavior change & end results (indirect influence) – all as external influences gradually increase along each stage of the process.  (Sandra Nutley’s slide presentation has been posted on the KTE CoP website).

Sandra Nutley’s interesting and engaging presentation concluded by emphasizing some generic features of effective KTE or KMb practices that RURU suggests applying to any research process.

Although these are valuable insights into effective knowledge mobilization, Nutley pointed out there is still much work to be done. She states we must move away from:

–Poor documentation and under-evaluated KTE (KMb) activities
–Studies that focus only on the instrumental use of research (see KMbeing blog on this)
–An assumption that research is used and applied mainly by individual practitioners
–Studies that result only in a now familiar listing of barriers and enablers, especially where these are the barriers/enablers experienced by individual practitioners
I always try to use my KMbeing blog to provoke and inspire deeper questions and thinking to break down some of  the barriers that Sandra Nutley has mentioned – barriers that also exist between academia & community.  It’s my hope that in some small way my KMbeing blog creates more inclusive, theoretical but simple and common approaches to our understanding of knowledge and knowledge mobilization (KMb) for the benefit of society. I also hope that by showcasing some of our great knowledge mobilizers (such as Sandra Nutley & Sarah Morton) – and the professional work they do – it will provide an opportunity to connect people who would not normally connect their knowledge in multi-directional ways across many sectors and communities.
(KMbeing Model of Knowledge Mobilization)
Many thanks to Sandra Nutley and Sarah Morton for a great presentations and for coming to Canada to speak to the KTE CoP on valuable approaches to research and KMb process.
Perhaps now it’s time to go out on Steve’s boat (Sandra’s husband) for a well-deserved rest back home in Scotland – if he gets it fixed up soon!

One response to “Assessing Research Impact

  1. Pingback: Whose Knowledge Is It Anyway? « KMbeing

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: