send us a message Diagrams Key areas of R&D effectiveness

diagram

Assessing R&D effectiveness

Working Group 62 “Assessing R&D Effectiveness” report Summary The Working Group presented its findings at a seminar Abstract of the seminar One participant described how he had tested and applied the methodology The winter 2004 edition of Innovation Quarterly included a special feature on the work

Project portfolio management

Working Group 59 “Project Portfolio Management” report Summary Working Group 52 “Project Management in R&D” report Summary Working Group 58 “Integrating Technology and Business Strategy” report Summary

Foresight and roadmapping

Working Group 52 “Technology Roadmapping” report Summary Working Group 55 “Technology monitoring” report Summary Working Group 50 “Long-term Planning in a Continually Changing World” report Summary 2009 Learning Group on Roadmapping

External awareness

“Responsible Partnering” initiative (guide to better practices for collaborative research and knowledge transfer between science and industry)More information “Reponsible Partnering” website Follow-up of “Reponsible Partnering” during the Special Conference in December 2007 2002 Brief on Industry-Institute Joint Programmes (“Changing Approaches to R&D Partnerships”) – see in particular Appendix 1 for the Underlying Benefits of Research Partnerships See also the 2002 Brief: Benefiting from External Advice and Review

Financial control in R&D

Round-Table meeting on financial control in R&D Summary of the Round Table meeting on financial control in R&D Special Interest Group III on “Knowledge Management” SIG-III meeting “Knowledge on the Move”SIG-III Meeting “Extension of KM Tools and Techniques beyond R&D” Working Group 54 “The Management of Corporate Knowledge” report Summary

The human dimension

January 2005 Representatives’ Round Table Meeting “New Paradigms for R&D” Learning Group on “Managing Creativity” Workshop 09 “Creativity and Innovativeness in R&D” report Summary Transformation Template TT© by Ken Taylor (January 2008), in particular Annex 2 for Typical Performance Indicators for People Issues. Article “A new template for high performance Science and Technology Parks in the European environment” by Prof Key Taylor, ETN & Silja Huhtiniemi, Kuopio Innovation Ltd. Paper presented at IASP 2006 annual conference in Helsinki, available at this site

Benchmarking

Working Group 51 “Benchmarking in R&D” report Summary First Special Conference on “Benchmarking R&D Processes” report Second Special Conference Report on “Benchmarking R&D Processes”

Effective innovation management

Effective innovation management

This article discusses techniques for assessing and managing effective innovation. Developing effective assessment strategies is complex: a group of EIRMA members spent hundreds of hours defining just one approach. Financial control systems can provide another form of assessment, although they don’t tell you how to get the most from your staff or access their creativity. Open innovation, collaboration and knowledge exchange can also increase the flow of innovation, while other techniques can help increase its effectiveness. For busy R&D managers, a well-designed assessment strategy can help benchmark overall performance, and provide early warning signals when things start to go wrong.

This article draws its insights from the wealth of material generated by EIRMA members during discussions at annual conferences, working groups, round tables and special interest group meetings.

eIQ Action Points – designing assessments of R&D effectiveness

One of the problems with running research and development organisations is demonstrating the value of what you do. Undertaking fundamental research means venturing into the unknown. Even advanced development work involves accepting a lot of uncertainty about outcomes and timescales. When recession hits, companies are bound to look closely at all their activities and ask them about their contribution.

R&D managers can usually point to a series of advances for which their teams have been responsible. The well prepared can also demonstrate the effectiveness with which they operate, to show that, even if the outcomes are uncertain, the path to those outcomes is well managed and carefully signposted.

Designing effective assessments

R&D is just one part of the corporate innovation story, which sees ideas developed into new products, processes or services. So how do you assess a company’s innovation performance and the effectiveness of supporting activities such as R&D? EIRMA members have dedicated hundreds of hours to this topic, and have jointly developed an assessment strategy that is summarised here and presented in full in an accompanying Management Brief, which is available on this page.

But assessing innovation performance is difficult. The meaning of ‘innovation’ is loosely defined, many factors contribute to its success, and a rapidly developing business environment makes things that appear innovative one year seem obvious the next (does anyone still think that ‘setting up a website’ is a business innovation?) Non-technical issues, such as supply-chain changes or new approaches to finance, are becoming more important to innovation, but their contribution is even more difficult to quantify.

It’s also important to consider innovation within the context of a company’s strategic direction, which may or may not have been well chosen, and which may or may not have been formulated to make the most of the innovations it creates. There’s not much point in counting patents if you’re not also checking how they came to be developed, and whether they’re being put to work in the service of company strategy.

These are difficult issues that help highlight the limitations of measurement strategies. But with those warnings in mind, how do you develop an assessment strategy? The first step is to establish how the sponsor defines successful innovation, the assessment criteria, what evidence will be considered credible, and the time-scale. It’s better to focus on assessing the extent to which managers have created the conditions for good innovation performance than on performance itself, because this will have greater credibility and provide more opportunities to involve people from across the organisation. It should also be possible to create assessments that can be done regularly, to track progress over time.

EIRMA’s assessment strategy

A working group of 30 senior R&D managers spent 500 person-days developing a process-based approach to assessing the conditions necessary for good innovation performance, based on a series of excellence indicators. The idea is to assess key aspects of each phase of the innovation process, to get a rounded picture of overall effectiveness. Because industries vary, the working group’s report includes recommendations on which areas are most important for a variety of industries.

Assessment needs to be carried out in the context of your business’s strategy. Does it want to be a product leader, relying on R&D results to achieve that; a fast follower, developing products based on well-understood markets and technology; or is it moving from one business sector to another, needing limited R&D to support the business it is leaving?

The assessment tool begins with excellence indicators, followed by a more detailed assessment of actual performance. Excellence indicators are meant to reflect the potential for high performance, rather than actual performance. The model uses a five-point scale for each indicator, and scores are determined using a matrix of objectives and responsibilities. The idea is that if the effectiveness of one aspect of a company’s innovation process is limited, its performance in that area will be too. Conversely, if effectiveness is high, there should be scope for high performance in that area as well.

Financial control in R&D

Financial control in R&D is more than just the control of finance in R&D. It provides another form of assessment, as well as acting in an enabling and advisory role, linking corporate R&D activities to corporate strategy, and business R&D to business unit strategy.

The financial control function has to understand the financial aspects of R&D in order to present balanced views of issues such as potential return on investment and profit margins in business cases, and demonstrate the connection between R&D and overall company performance. Financial control also involves connecting resources with costs; competencies with resources; and the R&D programme with the corporate and business centres. This kind of analysis can help decide the most appropriate funding model for each project in a portfolio.

A well-run financial control group should also contribute to decisions about the balance of innovation projects in a company’s portfolio, between short and long term; radical and incremental; new to the business or new to the world; and for mature or emerging markets. It may also help set financial goals for each research group or laboratory, ideally ensuring that these are SMART (specific, measurable, attainable, realistic and timely).

Financial control groups can make decisions that have large impacts. Even the choice of financial valuation model (for example, net present value) to assess the value of R&D projects can have an impact on what gets continued and what gets stopped. A host of other tools, such as influence diagrams, risk assessments, and sensitivity analyses, are also available to the financial control function, each with its own bias and impact.

For example, companies can fund R&D centrally or through the business units. The amount of money the company spends is the same, but centralised funding fits well with top-down management styles whereas funding through the business units give more power and responsibility to middle managers. Consider also R&D pricing strategies that charge internal customers at cost, but external customers at cost plus a profit margin. Such pricing is bound to affect the behaviour and focus of an R&D group, unless management steps in to ensure that internal customers get priority and external work is only taken on if it also fits with the company’s strategy. This was discussed at a recent Round Table meeting.

Getting the most from your staff

Innovation doesn’t happen just because you built a lab and filled it with people. It happens when those people apply human qualities such as creativity and personal entrepreneurship. So organisations need to actively support such qualities if they want to foster innovation. This is especially true in a world that is moving toward open innovation and greater interaction with the end user, approaches that demand different skillsets to traditional industrial R&D.

There have been many studies of human factors in innovation and each company tends to have its own set of guidelines for fostering what it sees as the right behaviour. Google, for example, says its managers should always try to hire the best - so there are no “second class citizens” in the organisation. They should encourage personal projects and entrepreneurship, and make the results of initiatives widely known, and quickly. Data should be used to provide a good basis for assessing their impact. And managers should constantly remind their staff that the users should come first, and that speed matters now more than ever.

Philips still uses the people management guidelines set out by the original founder of its research function: hire the brightest researchers; give them freedom; let them publish; promote cross-disciplinarity; and make sure leaders are responsible towards the company. Budget for overall research, not individual projects; promote the mobility of competent older researchers; and ensure that academic science is considered.

Are these approaches universally applicable? The answer is - probably not. They’ll work well in the right corporate context. For example, it is only worth encouraging personal projects if a company is also willing to take on the promising ones at some point. Similarly, putting users first can only work well if you develop close relationships with a broad set of customers, and reward schemes for those who make the effort to tap into their expertise. Assessment exercises can help expose this sort of discontinuity between the behaviours a company wants to encourage and the behaviours it actually supports.

Creativity

Your staff are creative people, but accessing that creativity means finding ways to help them generate ideas, giving them the time, tools and support to do it, and then having a systematic approach to assessing the resultant ideas. Embedding these first two steps in an organisational culture can be difficult: some companies create innovation centres, incubators, or ‘ideation labs’ to make the process easier by separating it from the company’s day-to-day concerns.

Creativity may mean individuals or groups finding themselves out of step with an organisation’s normal way of doing things. One way around this is to ensure that the company precisely defines the problem it wants to solve, so that any potential solutions that challenge the normal way of doing things at least have the benefit of being closely focused on the right problem.

Creativity needs to be nurtured, with support and recognition from top management. Ideas that contribute to a company’s success need to be rewarded; ideas that aren’t a good fit or fail during implementation should be seen as a positive part of the idea creation process rather than as mistakes for which people or teams should be taken to task.

Ken Taylor, a long-time EIRMA member and founder of the European Transformation Network (ETN), is convinced that most organisations can achieve a several-fold increase in innovation performance by improving the way people are managed, to create more effective organisations. He suggests a series of steps towards more effective organisations, such as reducing the complexity of innovation planning and exploitation and speeding up decision-making processes, while ensuring managers still get the critical information they need. Organisations should ensure they have techniques in place to merge critical information requirements across technology readiness levels, by establishing performance-readiness and risk-assessment metrics across these levels.

Managers need to check whether they’re using the right organisational system for their business, and whether they know how to assess people’s performance and get them to apply all their potential. They need to consider their ability to recognise useful innovation early, and to identify whether it will be disruptive and costly. And they need to assess the company’s knowledge of its most appropriate partners in a world of Open Innovation.

Taylor says enterprises can use this approach to create an environment free of bottlenecks, in which people’s full potential is converted to profitable performance. He has developed a technique to draw out all the skills and abilities of an organisation’s people and direct them to productive activities.

Open innovation, collaboration and knowledge exchange

Open Innovation, in which companies look for solutions outside their organisations as well as within, is becoming more widespread. But some companies have yet to integrate the use of Open Innovation into their corporate strategy, effectiveness assessment techniques and the processes they use to eliminate shortcomings.

Effective Open Innovation requires a change in culture and attitudes. Companies need to understand why they are adopting Open Innovation, in order to define suitable procedures for using it and to establish the required culture. To measure the effectiveness of their use of Open Innovation, companies need to have a deep understanding of their objectives and how well they are being addressed.

Industry has been practising a form of Open Innovation with academia for years. EIRMA members have helped develop Responsible Partnering, a voluntary programme and handbook designed to help public research organisations and companies work together more effectively. The report defines a set of core principles for successful industrial/academic collaboration and knowledge sharing, guidelines for putting these principles into practice, and some assessment criteria for companies implementing the guidelines.

Other approaches to effective innovation

Open Innovation is already showing us that companies can benefit from the ideas of other organisations - as one manager put it “we have lots of smart people but there are more of them outside the company than within it”. Ideas don’t have to be based on technology and if they are, they can be enriched by input from many other disciplines such as art, architecture, and anthropology.

Innovative organisations draw upon a range of other skills and activities, including portfolio and project management; business and technology roadmapping; strong team organisation; effective knowledge management; continuous benchmarking; effective knowledge exchange and technology transfer; and a deep-rooted understanding of their key relationships.

All of these need to be part of a corporate culture with coherent processes and proper support from senior management. For example, companies that successfully use scenario planning and technology roadmapping embed them into the corporate mentality, so that the whole organisation ends up with a shared view of future directions and risks.

Benchmarking and warning signals

By focusing on the conditions that support effectiveness and therefore performance, assessments serve as a kind of ‘corporate dashboard’: you check for emerging issues and get a red light when processes stop working well. A good assessment strategy should be able to warn if a business is competing with the wrong technology or in the wrong market, or show that it is not very good at turning research results into valuable innovations.

Assessments can provide a basis for benchmarking effectiveness and performance against other companies and organisations. If the chosen criteria suit a company’s organisational culture and type of activity, and reflect the influence of factors such as customers and other stakeholders, it should be possible to do in-depth assessments that fit with its strategy.

Managing R&D organisations and adapting them to corporate demands for innovation, rather than research results, is a complex job. It involves taking risks in a world that prefers certainty; balancing short-term demands with long timescales; asking staff to be free-thinking and creative as well as measured and precise; and rarely having easy answers to questions such as ‘When will it be ready?’ or ‘What am I paying you for?’

In this context, the best assessment systems should help you understand the way your work fits within your organisation and its strategy, the areas for improvement and any potential bottlenecks. Once these issues are sorted out your organisation should be well placed to achieve its best. Given the pressures on R&D managers, a good assessment strategy should ensure that you can focus on where you want to get to, keeping an eye out for any warning lights on your dashboard, without constantly having to stop to look at the engine.

eIQ Action Points

Focus the assessment by considering how well the activity is aligned with strategic goals and how well its results are used to support those goals
Characterise the business in terms of its capital intensity and profitability. Understand where the organisation wants to be in each of its markets: a leader, a fast follower or making a graceful exit
Start with a quick assessment based on excellence indicators. Establish the indicators that are appropriate for the profitability and capital intensity of the business being analysed
Score the organisation on these indicators. Use many assessors from many roles, to gain a more rounded view and wider ownership of the findings
Use the results of the quick assessment to establish more detailed assessments using performance indicators. Prioritise the areas that need addressing and develop an action plan
Feed the results back into the organisation at all levels, from informing board-level strategy to motivating individual performance

Bibliography

The EIRMA website contains a wealth of material about running effective high–performance R&D organisations. Most of it is held in the member area of the website, which requires the usual username and password.