.png)
The easiest person to fool is often yourself -- while this message rings true for many facets of life, it especially applies to developing a proprietary statistical methodology to solve a new applied problem! At Truthset, we know this all too well, as we talk to the ecosystem of data buyers and sellers about our data quality scoring solution, Truthscores™, and often hear every party believe that their data is the best data.
With this knowledge, we know that we too shouldn’t be blinded by optimism and confidence in our own Wisdom of the Crowds methodology for scoring data quality. In order to practice the principle of transparency that we so-often preach, we engaged Joel Rubinson, a luminary in the marketing science and market research community. Joel worked directly with Truthset’s data science team to deeply understand, scrutinize, and ultimately -- after months of review and collaboration -- validate our proprietary Truthscore™ methodology.
The result -- a white paper, “HOW USING DATA QUALITY METRICS CAN SIGNIFICANTLY IMPROVE THE PRECISION OF ADDRESSABLE MARKETING,” written by Rubinson for Truthset and available for download now.
In his Executive Summary, Rubinson highlights that addressable advertising promises “more than it delivers'' and that “as it stands now, marketers still need a reliable, independent assessment of the quality of targeting data.”
That’s where the value proposition of Truthset and Truthscores enters the picture: “Until now, no one has used such data assets and statistical approaches to tackle this use case,” writes Rubinson, and “without Truthset and Truthscores, a marketer has no way of independently evaluating the accuracy of individual demographic assertions from different segment providers, either collectively for the segment or individually for each targetable ID within the segment.”
Rubinson goes on to pose a specific challenge for Truthset and Truthscores: “If Truthset can identify which providers have a higher accuracy for a given targetable demographic segment and if they can sort out which specific IDs are most likely to be the ones that are accurate, they can provide enormous value to a marketer by improving on-target percentages for any addressable marketing use case.” To definitively prove that Truthscores drive a significant increase in on-target rates for a desired demographic among a group of IDs -- Rubinson devised a series of statistical hypothesis tests that “align to [this] primary marketer use case."
For every attribute (e.g., age) and attribute value (e.g., 18-24) under measure, the tests involve sorting and splitting IDs with assigned Truthscores into two groups: one group for IDs with Truthscores that fall above the incidence rate of the desired attribute value in the US general population, and one group for IDs with Truthscores that fall below that threshold. For Truthscores to be accurate, two observations must hold: first, there should be a statistically significant difference in the true, validated incidence of the desired attribute value between the two groups of IDs; and second, among the group of IDs above the Truthscore threshold, the validated incidence of the desired attribute value should be much higher than the respective incidence in the US population.
Let’s take an example, directly from Truthset’s data: the incidence of 18-24 year-olds in the US adult Internet population is approximately 14%. The true, validated incidence of individuals aged 18-24 among those HEMs above the chosen Truthscore threshold is 73.2% (almost a 5x increase!). This result is achieved by weeding out those HEMs that fall below the given Truthscore threshold – those records in that removed set had a true, validated incidence rate of individuals aged 18-24 of only 2.6%. Ouch! In this example, Truthset is able to weed out hundreds of thousands of IDs that are not likely to be 18-24, and thus would be unproductive to target and have created (avoidable) ad waste.
The results of these statistical hypothesis tests, “conclusively prove the validity of the Truthscore methodology,” writes Rubinson. “For every attribute value that was tested, the difference in the true incidence of the given attribute value between the two groups (i.e., those HEMs that fell above vs. below the given Truthscore threshold) was significant at the 99% level or higher,” found Rubinson. “In addition, the true incidence of the given attribute value among the group of HEMs that fell above the Truthscore threshold was always statistically significantly higher (at the 99% level) than the respective incidence in the US adult Internet population.”
Rubinson concludes, “for marketers, these results are really quite remarkable. Think of it this way: a provider offers a segment containing a list of IDs that are all purported to possess a desired attribute, demographic or otherwise. At the start, a marketer has no way of knowing what percent of those IDs truly possess that attribute, and more importantly which exact records accurately represent this attribute, and which do not. ...Truthset has created a statistical method for sorting these IDs into sub-segments based on attribute value Truthscores.”
Having Truthset’s methodology evaluated by an expert shows that we not only practice what we preach, but that our very own data science and engineering teams have gone toe-to-toe with a massive gap in the industry and created a solution that passes muster. “It’s enlightening to see that Truthset’s methodology stands strong against rigorous testing, and that, more importantly, our Truthscores translate into significant impacts for our clients,” said Kathryn Barnitt, Statistician at Truthset.
About Rubinson Partners, Inc., Joel Rubinson was retained to provide an independent assessment of the Truthset methodology. Joel is running a successful consulting business that has served 75 leading firms across AdTech and media (e.g., Oracle’s Moat, Viant, NBC, AOL (now Verizon), and many marketers (e.g., General Mills, Coca-Cola, Unilever, J&J, MetLife, Verizon, Estee Lauder). Joel was the former Chief Research Officer of the ARF. Among his consulting assignments, he has functioned since 2016 as the subject matter expert for Multi-Touch Attribution (MTA) approaches for the MMA, interacting with over 50 leading marketers, media companies, and AdTech firms on advanced analytics topics. Joel’s white paper, “The Persuadables”, tested in partnership with Viant and Nielsen Catalina Services, is viewed as a definitive study of the value of targeting. Joel was the CRO for the NPD Group, leading efforts on creating and refining their weighting and projection systems for their sales currency data on the industries they track. As a faculty member at NYU, Joel created and taught their first grad course on social media marketing. He started his career as the head of analytics for Unilever in the US, and holds an MBA with concentrations in economics and statistics from the University of Chicago.
For more information about Joel's professional credentials, please visit: https://www.linkedin.com/in/joel-rubinson-a3a0763/
About Truthset
Founded in 2019, Truthset is a data intelligence company focused exclusively on validating the accuracy of consumer data. The company helps brands build trust in data and improve the performance of any data-driven decision. Truthset does not sell data and is not a data broker; it compiles a likelihood of truth for any individual record that can be used to validate the accuracy of data and power more accurate consumer interactions. To learn more about Truthset, visit www.truthset.io.