---

What Is 'Analytical Honesty'?

Analytical Honesty means measuring and analyzing your data dispassionately.

The best way to maintain Analytical Honesty is to have your measurements evaluated by someone who isn't invested in your system.

Don't lie to yourself when it comes to your data: be honest and focus on improving the system instead.

Josh Kaufman Explains 'Analytical Honesty'

My last position at Procter & Gamble involved creating an online marketing measurement strategy: figuring out a way to measure the effectiveness of P&G's advertising on the Web.

P&G spent millions of dollars every year on banner ads, search engine advertising, and online video placements. My job was to figure out if the investment was worth it.

One of the things my team discovered while putting together our measurement recommendations was disturbing: the system we were using on most of our Web sites to measure unique visitors was wrong. Instead of simply counting human visitors, the system was treating visits from search engine programs as "visitors" as well, even though they weren't people.

Computer programs don't care about laundry detergent and shampoo, but they were being counted as potential customers. It's not uncommon for search engine spider programs to visit a Web site many times a day looking for new information, so the system was overstating how many people were visiting each Web site by a significant margin.

Put simply-it was a Garbage In, Garbage Out situation, and it made our measurements less useful.

Naturally, we recommended an upgrade to another tracking system that collected visitor data more accurately.

Almost universally, the teams declined to upgrade to the new system. They knew they were getting crappy data, but they didn't care.

Weird, right? Here's the rub: installing the new tracking system would make each Web site's unique visitors measurement-considered a Key Performance Indicator by most teams-go down dramatically.

Even though the new system was clearly more accurate, installing it would make the team look bad. Instead of correcting the issue, they chose to continue living a lie. As a result, they seriously compromised their ability to improve the Web site's actual performance.

Analytical Honesty means measuring and analyzing the data you have dispassionately.

Since humans are social creatures, we tend to care deeply about how others perceive us, which gives us an incentive to make things look better than they actually are. If your purpose is to make things better, this tendency can get in the way of collecting accurate data and conducting useful analysis.

The best way to maintain Analytical Honesty is to have your measurements evaluated by someone who isn't personally invested in your system. Incentive-Cased Bias and Confirmation Bias are all too easy to succumb to if your status or self-image are on the line.

Having an experienced but dispassionate third-party audit your measurement and analysis practices is a neat workaround for these tendencies: you might not like what you hear, but at least you'll be aware of potential issues.

Don't look at your data through rose colored glasses: always strive to be honest with yourself about what the data indicates you can improve.

Questions About 'Analytical Honesty'

• Are your measurement practices truly accurate?
• How can you prevent incentive-caused bias and confirmation bias from compromising your ability to analyze your system dispassionately?

---

"That which can be destroyed by the truth should be."

P. C. Hodgell, professor and author of The God Stalker Chronicles

---

---

Share this concept:

https://personalmba.com/analytical-honesty/

---

---

---

---