Merge/Purge and DM Success

Have you ever wondered why so many rollouts are disappointing? Three prior DM News articles established that a significant part of the answer lies with statistical sampling theory ("How Big Should My Test Be?," October 1, 2001, "Identifying Millions in Lost Revenue," January 7, 2002, and "Why So Many Rollouts Disappoint," February 4, 2002). In addition, there are many non-statistical reasons, including changes in creative, improper test design, list brokerage and management mix-ups, suboptimal postal service performance, shifts in the economy, competitive developments, and inconsistencies and inaccuracies in the merge/purge process.

This month, we will focus on the merge/purge process. More precisely, we will revisit the topic by expanding on an article last year that outlined how merge/purge can affect reported list performance ("Merge/Purge Can Alter List Strategy," September 3, 2001).

Ignorance and Inattention

Since the late-1980's, merge/purge has been a perceived commodity not worthy of attention by our industry's movers and shakers. In such an environment, companies tend to compete on price. And, all too often, direct marketers get exactly what they are paying for in terms of quality.

In reality, merge/purge "at least for direct marketers who rent large quantities of rental lists "is a series of intricate and interconnected data processing steps. As such, every client company should designate at least one knowledgeable person as having overall responsibility for the process. This individual should determine overall strategies as well as the ongoing details of execution.

Most clients, however, delegate these decisions to a low level employee or to the service bureau itself. The disadvantages of the in-house option are self-evident. However, delegating to the service bureau can be equally problematic.

Often, service bureau account people operate as reactive order takers rather proactive strategic partners. Generally, these individuals have never been direct marketers, and view their assignments from a very narrow processing perspective. Also, the demands of the service bureau business tend to result in significant turnover, especially among those in the lower ranks. And, these are the same individuals who must ensure accuracy and consistency across jobs.

Steep Tax of Ignorance and Inattention

It is remarkable how many times our firm, when beginning work with a new client, discovers that formal, detailed merge/purge instructions do not exist. Often, the sole guidance the service bureau is given is to, "Run it just like last time." Frequently, when a careful analysis is conducted, one or more major problems are uncovered.

The most egregious example occurred with a direct marketing company whose merge/purge costs were approaching the outrageous level of $70 per thousand names mailed. When the situation was analyzed, the culprit turned out to be a suppression list that had grown wildly out of control.

Several years earlier, the company had notified its service bureau to suppress rental names that had been contacted during the previous drop. The results were so positive that the company decided to include such a suppression strategy in all subsequent merges. However, the company did not supply its service bureau with explicit instructions. Instead, the written guidance was to, "Run it just like last time."

What the direct marketer meant was to suppress only the rental names from the previous drop. What the service bureau did was create a running suppression list. As impossible as it might seem, this went on for many merge/purges until the suppression list had grown to encompass many tens of millions of records. Needless to say, a small fortune had been squandered in needless data processing, and the direct marketer's available prospecting universe had been severely and pointlessly constricted.

Unappreciated Complexities

The parameters that drive a merge/purge, and the circumstances that surround it, can affect the reported results of a mailing "and, therefore, the probability of rollout success. Because of this, the parameters should be thoughtfully considered when setting up a job, and the underlying circumstances factored in when conducting follow-up analysis.

As just one example, consider that the overall net rate out of the record matching portion of a merge/purge usually drops as the number of rental lists increases. This decreasing net rate, in turn, is driven by an increasing percentage of prospects who appear on multiple rental lists ("multis"). And, as a general rule, multis have a response rate that is between 50% and 100% higher than prospects who appear on just a single list ("uniques").

A hypothetical example will illustrate how all of this affects the reported results of a mailing, and therefore can contribute to a disappointing rollout. In this scenario, Prospect List A's full universe of 100,000 is rented and then promoted without any other outside lists. The resulting response rate is 1.01%, which we will refer to as Prospect List A's "true" response rate. Continuing with our example:

Subsequently, Prospect List A's full universe of 100,000 is included in a merge/purge that results in a net rate out of record matching of 80% for List A. For the sake of simplicity, we will assume that no multis appear on more than two lists, which means that 40% of the original 100,000 records are multis. (In other words, with 40,000 multis, 20,000 will be attributed to List A, and 20,000 will be attributed to other lists. The result will be a net rate of 80,000 records for List A, or 80% of the original 100,000.) We will also assume that multis do twice as well as uniques.

The chart illustrates how, as the total input volume to the record matching portion of the merge/purge increases, there is a direct impact on the percentage of net names and the response rate for List A. Specifically, as List A's response rate declines to 0.90%, compared with its "true" rate of 1.01%, as the net rate drops to 80%:


As a perceived commodity, merge/purge generally is not considered worthy of concentrated focus. Direct marketers pay for this with reduced circulation effectiveness. However, a knowledgeable client-side contact, coupled with a trusted service bureau partner, can transform the merge/purge into a powerful business tool.