The American Statistical Association has a set of ethical guidelines to be used by quantitative analysis professionals including statisticians, forecasters, market modelers and those helping brand managers to define the drives and barriers to product trial, adoption and long term use. Many members of MRA rely on external and internal statisticians to support quantitative research and rely on their ethical use of statistics.
In the Executive Summary of the Guidelines, it is noted that, “It is important that all statistical practitioners recognize their potential impact on the broader society and the attendant ethical obligations to perform their work responsibly. Furthermore, practitioners are encouraged to exercise ‘good professional citizenship’ in order to improve the public climate for, understanding of, and respect for the use of statistics throughout its range of applications.”
The last of eight is Responsibilities to Employers. In this area the following issues are raised, “Including Organizations, Individuals, Attorneys, or Other Clients Employing Statistical Practitioners encourages employers and clients to recognize the highly interdependent nature of statistical ethics and statistical validity. Employers and clients must not pressure practitioners to produce a particular ‘result,’ regardless of its statistical validity. They must avoid the potential social harm that can result from the dissemination of false or misleading statistical work.” I have seen egregious violations of this guideline where clients threaten retaliation if results do not support a preordained outcome.
As a “quant jock,” at a respected strategic research firm serving the pharmaceutical industry, I saw the in-house statistician routinely change the sign on predictors of product use so no questions would be asked. For example, one key driver pharmaceutical project involved performance on effectiveness, safety, price and side effects to predict product share. The client’s product had a price issue and the beta weight from a multivariate analysis had a negative sign suggesting that as price rose, share declined indicating the client’s product was losing to lower priced competition. A negative sign on that key driver was a problem resulting in a change in sign with no basis in the analysis itself.
In another such position at a “beltway” statistical firm supporting Federal Government agencies, I headed a team doing customer satisfaction with healthcare benefits provided by the government. On some items, the beneficiaries were highly dissatisfied with the service they were getting. This result was a problem for the government bureaucrat resulting in a request to not report those items.
In a third situation, my own firm, a forecast was to be developed for a new pharmaceutical product that would be advertised to consumers to gain early trial and adoption. The print ad campaign was designed to drive consumers of the doctor to request the new product. Extensive marketing research indicated that although the new product had a significant benefit advantage over current competition, the conversion ratios for consumers going to the doctor to ask for the product specifically as well as the ratio for the doctor to then prescribe the product were very low at less than five percent. What that meant was the current forecast highly overestimated trial and adoption as well as ROI on the ad campaign. The result was the forecast was left alone and the campaign failed.
In all three cases the client applied pressure to “massage” the results to put a bit of “lipstick on the pig.” Research ethics apply to both clients and their statisticians who work on their projects. Clients pressuring researchers is just as unethical as researchers who voluntarily change results.