Best Practices & Benchmarks

Best Practices & Benchmarks

Regardless of what facet of organizational functioning and performance one considers, it is difficult to assess it without an appropriate frame of reference. Is 6% ROI good? Is 5% chance risk of incurring an adverse event high? Is 50% probability of repurchase high? The short answer is – it depends. If the industry average ROI is 4%, 6% is comparatively speaking quite strong, but if the industry average ROI is 10%, 6% would likely be seen as poor performance. The same applies to the said 5% chance of incurring a damaging event, and 50% repurchase probability: The qualitative ‘high-’ vs. ‘low’-risk exposure and repurchase probability conclusion would typically be a function of the company or customer, respectively, specific estimate compared to a pertinent benchmark, such as the industry average or the average across all customers. This illustration underscores the importance of benchmarks as the key component in arriving at decision-guiding conclusions.

This is an important point. Let us consider again the sample 5% chance of incurring an adverse event, and let us further assume that the average across all comparable companies (a peer group) is 7% for that particular exposure. From the risk management-related decision-making point of view, granular numeric estimates are ultimately interpreted in a more general context of conclusive categories such as ‘low’, ‘average’, or ‘high’ because risk response choices distinguish among categorically, not merely mathematically different values. In other words, the conclusion of high-risk exposure would likely compel materially different risk responses (in terms of specific risk mitigation, risk financing, and related decisions) than the conclusion of low-risk exposure while the estimate of 3% vs. 9% risk probability would not, by itself, trigger differentiated responses due to lack of proper contextualization (i.e., numeric estimates such as those don’t convey universally recognizable meaning in the way that ‘high’ vs. ‘low’ categories do). Hence, the importance of benchmark validity and reliability: To yield robust conclusive, decision-guiding categories, the underlying benchmark has to be sound (i.e., valid and reliable); only then will the mathematical estimates produce robust conclusions.

One of the key considerations entailed in defining an appropriate benchmark is the composition of the peer group. For some organizations, such as public utilities or insurance companies, it may be a relatively straightforward and objective task, given the well-defined industry boundaries and easily enumerable like organizations. For other organizations, however, such as diversified conglomerates, the peer selection process tends to be more subjective, and the resultant peer group noticeably more heterogeneous, in terms of composite companies’ profiles. Under most circumstances, the more heterogeneous the peer group composition the greater the within-peer-group variability of focal characteristics, such as exposure to specific risks, and outcomes, such as operational and other performance, which reduces the efficacy of benchmarking. A yet another frequently encountered challenge is that of limited sample size, as even well-defined industry segments, such as leisure and hospitality, are comprised of very small numbers of dominant firms,3 which effectively severely limits the number of true peers. Here, the same is true as in the case of high within-group variability mentioned earlier: The small sample size leads to greater benchmarking metrics’ variability, which, in turn, diminishes the efficacy of benchmarking.

The challenge associated with delineation and use of the so-called ‘best practices’ is considerably more vexing, primarily because of the far greater degree of subjectivity associated with the identification and interpretation meaningful and applicable praxis. Popularly characterized as professional or commercial procedures or courses of action prescribed as offering the most effective means of producing desired outcomes, best practices can be a product of experience or research, though in business management it tends to be overwhelmingly the former. The implied intent behind the pursuit and embrace of best practices is the desire to achieve superior results by replicating, or mimicking, processes or methods developed and used elsewhere.

Conceptually, it is a compelling idea, but practically it is fraught with problems as its premise is rooted in a number of questionable assumptions, the most fundamental of which is ‘that which worked there will also work here’. Even organizations that compete in the same industry producing functionally substitutable products and/or services are likely to exhibit material cultural, organizational, and structural differences that make some methods or practices more applicable in one organizational setting than in another. The obvious differences might include leadership style, which can be centralized in one organization and decentralized in another, risk appetite, with some companies being risk-averse, whereas others aggressive risk takers, or insourcing vs. outsourcing posture, with some organizations committed to reliance on their in-house operational infrastructure (insourcing) while others equally committed to using outside suppliers (outsourcing). A practice that has been shown to work well in an organization characterized by centralized leadership style, risk-averse decision-making and insourcing focused operations may be ill-fitted to a decentralized, aggressive risk taking, and outsourcing-minded organization.

The notion of best practices implies the degree of universality that may be found only in selected few areas of organizational management such as data management or business continuity planning. Those organizational domains are characterized by a considerable degree of cross-firm similarities due to a combination of structural- and oversight-related factors. Regarding the former, virtually all business enterprises use commercially procured (vs. in-house developed) data capture and management hardware/software systems which naturally lends itself to cross-organization standardization. In terms of the latter, laws and regulations addressing certain sensitive aspects of data capture and storage, such as privacy and security, apply uniformly to all companies that capture and maintain in storage certain types of data, such as financial or health care details. Consequently, identifying and adhering to industry best practices in the areas such as data security is highly desirable.

There is yet another reason why best practices could be less-than-ideal standards to emulate: At least some are rooted in tradition rather than good economic sense. For example, the widely accepted publishing industry’s best practice is that a bookseller can return any unsold books to a publisher, at that publisher’s expense, which is a practice that dates back to the time of the Great Depression. Not only did the demand for books declined precipitously during that time, but the vast majority of booksellers were small, independent shops that could not assume the financial risk of buying books that might not be able to sell to the public. One of the major publishers at that time, Simon & Schuster, came up with ingenious solution by offering to assume the financial risk of unsold inventory by allowing book retailers to return, expense-free, unsold books. The solution worked well for Simon & Schuster as it allowed the publisher to continue to sell books, so not surprisingly it was quickly copied by other publishers and it remains the industry best practice, in spite of fundamentally different demand characteristics and industry structure. One would be hard-pressed to defend the wisdom of embracing that particular practice today, or even to characterize it as ‘best practice’ as it is nothing more than an outdated norm that continues because of tradition, and in spite of lack of sound economic reasoning.

Still, perhaps the most significant downside to overreliance on industry best practices is that it can stifle innovation or strategic creativity. How many of today’s most admired companies – Amazon, Google, Apple, or Toyota – rose to prominence by emulating their industry peers? Setting aside a handful of structural and oversight areas briefly discussed earlier, industry benchmarking and best practices are rarely, if ever, thresholds of excellence – typically, they demark baselines of acceptable, that is, average, performance. It is more rational, in fact more statistically sound to think of industry benchmarks and best practices as guidelines to not being left behind (competitors), rather than a roadmap to attaining leading results. An investor with a highly diversified portfolio will most likely earn average returns, just as a business organization with peer benchmarking and industry best practices focused managerial mindset will also likely realize average results. For those who want to ‘play it safe’, that might indeed be an appropriate and a rational mindset – for those, however, who aspire to above average results, sound but independent decision-making might be a better bet.

Even worse, adoption of the industry best practices can ultimately (meaning, in the long run) adversely impact organizational performance. Business process reengineering, agile management, and lean/six sigma are among the many consultant-popularized frameworks that promised to streamline operations, increase productivity, enhance efficiency, or increase profitability. And to be sure, some of those frameworks deployed in some organizations indeed delivered short-term benefits, but there are few documented and empirically validated examples of those ‘best in class’ practices delivering sustained long-term benefits. Managers continue to embrace those ideas not only because of the almost universal desire to be on the cutting edge (which those approaches presumably represent), but also due to their auras of simplicity, making them easy to communicate, comprehend, and, when needed, defend.