Before you can assess your IP function, you need to know what measures to apply

Measuring the effectiveness of the intellectual property management function

This article discusses ways of developing tools with which to measure the performance of the IP management function. A benchmarking exercise with EIRMA members produced a number of insights about current IP management performance, and provided the basis for a workshop at which a number of metrics were discussed and refined. The workshop group found that there was no single ‘correct’ set of metrics, and advised that the choice of metrics would depend on a company’s strategy. It also found that developing a measure of return on IP investment was too difficult to be practical.

eIQ Action Points – Measuring the effectiveness of the IP management function

In the physical economy it’s easy to see that physical things, such as equipment and materials, are assets. To track these assets, companies make inventories of their equipment, count the materials in the warehouse and record the work they have in progress. Because managing these physical assets is so important, industry has also developed tools to measure how well it is being done, for example by checking how long it takes to turn incoming materials into finished goods.

As we move to a more knowledge-based economy it's becoming increasingly important to manage intangible assets such as a patents and trademarks, and also to check how well the people entrusted with doing this are performing. One way to achieve this is to develop and apply a set of key performance indicators (KPIs) for the IP management function that is meaningful, useful and practical.

Developing tools

This is not straightforward. Measuring the performance of an IP management function takes more than counting the number of patent applications it handles a year. A good system for making such assessments should enable efficient performance control, as well as being practically applicable. A company might decide it needs measures of the ‘quality’ of its IP portfolio, how well the IP department is protecting and exploiting that portfolio, and its alignment with the overall business strategy. Ideally, the measurement system should reflect the business’s expectations of the IP function, so its performance can be measured against those criteria and the resultant information used to provide feedback that helps improve its performance.

Prof Dr Alexander Wurzer of the Steinbeis-Transfer-Institute in Berlin has worked with members of EIRMA's special interest group on IP to develop a set of key performance indicators (KPIs) that would represent a step towards best practice in IP management.

Wurzer’s first step was to send a questionnaire to four participating companies. All four companies had revenues of tens of billions of euros, and employed thousands of R&D staff. Annual R&D spending was up to €1.3bn a year. Two of the companies sold their products to consumers, and two to other businesses.

Once the questionnaire results had been gathered and interpreted, key contributors attended a one-day workshop in Munich to hear the results, consider the IP metrics presented in the academic literature, discuss their common characteristics, and talk about how to choose metrics to fit with a given business strategy. The result of this effort was presented at the SIG-II meeting in Madrid in November 2009.

Insights

Some interesting results emerged from the questionnaire: for example, between 43% and 88% of the invention disclosures submitted to IP departments in this sample were converted into patent filings. Of those filings, between 33% and 83% were converted into a patent. So just one third of initial invention disclosures resulted in granted patents. The average R&D spending per granted patent was €2.3m.

Looking at the ‘freshness’ of patent portfolios, one company’s portfolio was growing at more than 20% a year, despite abandoning between 5% and 10% of its patents each year. In a second company, the rate of growth, at between 5% and 10%, matched the rate of abandonment, giving a stable portfolio size. For the remaining companies, the rate at which patents were being abandoned was greater than the rate at which new ones were being added.

Although IP is vital to making money in the knowledge economy only two of the companies tried to put a financial value on their patents and trademarks. Both used these valuations in deal-making and other transactions, while one of them also used it for financial reporting and the other as a basis for rewarding inventors. The monetary value of the IP wasn’t used as a performance indicator.

A question on how long it took to decide to apply for a patent showed a wide variety of answers. One company took just four to eight weeks from receiving an invention disclosure to deciding whether to patent it. If it decided to patent, it would take up to a further eight weeks, or a total of up to 16 weeks from invention disclosure, to apply. At the other end of the scale, another company took between eight and 16 weeks between receiving an invention disclosure and deciding to apply for a patent, and then up to a further 32 weeks to actually do so, in an overall process that could last up to 40 weeks. In all cases, it took three to five years for a patent application to be granted.

The relationship between the patent portfolio and the product portfolio also varied. In three companies, only 10% to 20% of the patents in the portfolio were relevant to current products, although this figure was 25% to 50% for a fourth company. Looking to the future, in one company less than 10% of the patent portfolio related to future products while in another more than 50% of it did so. Putting together results from the two questions, it appears that between 40% and 70% of these companies’ patent portfolios were unrelated to current or future products.

The patent mix also varied widely, with between 10% and 25% of portfolios relating to basic research and between 45% and 80% relating to products.

Choosing metrics

Having set some benchmarks, the workshop group moved on to categorise some IP metrics selected from the literature on the subject. It found 11 metrics to do with IP portfolios, seven to do with technology protection, six to do with IP exploitation and 10 measuring IP departments. In total 34 metrics were available to describe best-practice IP management approaches. Further practical metrics were discussed, including some to cover individual IP strategies or to focus on topics such as open innovation.

The group also decided that it was important that each of the metrics that was to be applied was analysed. This would enable each metric to be checked to ensure it was being described in ways that could be commonly understood, used common underlying information and data, had defined critical values and thresholds, and gave results that could be interpreted meaningfully so that realistic actions could follow.

For example, a measure of “the time from invention disclosure to the decision to file a patent” could be specified in terms of what it measures, in this case the time from invention submission to decision to file; the reason it is being measured, in this case to check whether the IP function is acting promptly on incoming invention disclosures; where to find the underlying information, for example from internal intellectual asset management software; the key values of the metric, set as a norm by management; and the actions that should follow if the value deviates from the norm. In this case, this might involve setting someone to find the bottleneck in the decision-making process if the time from invention disclosure to decision to file exceeded the norm.

Return on investment

One key measure that the workshop did not produce was an overall return on investment in IP. The participants found that it would be impossible to give a precise value or single metric for this without massive effort and expenditure. They suggested instead that the value of an IP portfolio should be expressed as a narrative. [One useful example of this approach relates to the work of Xerox’s Palo Alto Research Centre (PARC) in the 1970s and 1980s. PARC is widely credited with having developed the Ethernet networking standard, the laser printer, and the ‘windows, icons and mouse’ computer user interface. Many commentators believe that Xerox gave up a great deal of value when it failed to commercialise the windowing user interface, which was later taken up by Apple and Microsoft. Xerox, on the other hand, argues that its laser printer patents alone more than paid for PARC’s activities over the years.]

Faced with many possible metrics, the group suggested that there isn’t a ‘right’ way to measure the effectiveness of an IP management function: the key is to choose the set of measures that fits with the business strategy. It also concluded that there is a practical constraint to the total number of metrics that can be used, if they're to be applied consistently over the long term.

eIQ Action Points

IP asset management is increasingly important, even if the assets are intangible
If you want to know how well your IP management function works, you’re going to have to develop some measures of its effectiveness
Plenty of such measures are available in the literature – the key is to choose the set that best matches your company’s strategy
A good test is to ask whether the outputs of the measures you choose can be used to adapt the IP function’s work to fit better with corporate strategy
Don’t expect the monetary value of IP to be used as performance indicator: instead use it in financial reporting, deal making and to reward inventors
Apply some basic quality tests to the measures you choose: are they clearly expressed, based on clearly defined data sources and norms, and able to provide clear outputs?
You may have to accept that the best way to show the value of a company’s IP is to talk about it, not measure it