Gilligan on Data by Tim Wilson

In Part I, I took a run at assessing a couple of the most popular baseball statics to see how they measured up as well-formed performance metrics. The other thought that has been running through my mind as I’ve been scoring my son’s baseball games has to do with data management and data quality.

Scoring a baseball game requires a couple of things:

• Making judgment calls as to what actually happened
• Capturing the right information on screwy plays where a lot of stuff happens (this happens a lot more in 9-year-old baseball games than it does in college or professional games)

The first item is one of the reasons why college and professional games have an “official scorekeeper.” There are some plays that are clearly fielding errors…but there are some that require a subjective assessment. And, even if there is clearly an error, it’s sometimes subjective as to whether it was a bad throw or a bad catch.

And, things can get a little complicated. For instance, if you look at this picture closely, you’ll be able to tell that my son is churning his 9-year-old legs as fast as he can (admittedly in pants that would fit most 12-year-olds) as he runs towards first base. And, yet, the catcher is standing right at home plate with the baseball, looking like he’s about to make a throw. What’s going on is either totally obvious to you — meaning you played baseball or have followed it with a decent level of interest — or it seems very bizarre. My son had just struck out. The rule in baseball is that, if a player strikes out AND the catcher drops the ball AND EITHER first base is unoccupied OR there are already two outs in the inning, the catcher needs to retrieve the ball and either tag the batter or throw the ball down to first base so the first baseman can tag first base. This is what’s called a “strike him out, throw him out.” You don’t see it very often in the major leagues or college, because catchers don’t drop that many balls. You see it quite a bit when the players are nine and ten years old.

Either way, my son had an official at bat with a strikeout, even if he made it to first base safely (if, for instance, the catcher overthrew first base). If that had happened (in this case, it didn’t), I would have needed to record a strikeout as well as an error on the catcher.

Sound complicated? It is, and it isn’t. Baseball has other semi-obscure rules — if a baserunner passes another baserunner, he is out. I didn’t learn that rule until I saw it happen to Baylor in the College World Series several years ago. So, scoring a baseball game correctly requires:

• Paying close attention to every play throughout the game
• Knowing the rules well
• Knowing how to quickly and accurately record both “normal” plays and oddball plays
• Being able to make the subjective calls quickly and effectively

I’ve never actually tried to verify this, but I am fairly certain that, if you take three run-of-the-mill scorekeepers and have them score the same game and then compare their results, you will get three slightly different versions of what happened. Yet, we view baseball stats and box scores as being completely black-and-white.

I worked with a data management guru at National Instruments who had a Mark Twain quote in her e-mail signature that said something to the tune of: “A man with one watch always knows what time it is. A man with two watches is never sure.” (I’ve tried to look up the exact wording and confirm that this indeed originated with Mark Twain in the past, and I didn’t have much luck.) This is an excellent point, and it applies to both baseball and business.

If we see a number that appears to be precise — 73 pitches, 10,327 visits to a web site, 2,342 leads — we equate precision with accuracy. It doesn’t cross our mind that a scorekeeper might have inadvertently clicked his pitch counter when the pitcher actually made a throw over to first base to try to pick off a runner. We ignore the fact that all data capture methods when it comes to web analytics are inherently noisy. We forget that sometimes our lead management processes break down and load a duplicate lead or miss a lead. We assume that the data that gets entered into our systems by humans gets entered by a robot rather than by a human — no judgment calls, no mental lapses. And that is simply not reality.

None of this is to say that we should throw out the data. At the end of the day, the ERAs that I calculate for the pitchers on my son’s team are going to be pretty close to the ERAs that another scorekeeper would calculate. Close enough. But, it’s easy to get caught up first in assuming that precise numbers are perfectly accurate, and, then, when something happens where you see a discrepancy, focussing on trying to get the “right” number rather than asking, “Is the difference material?”

The moral? Well…baseball is a great sport!

Oh, wait. There’s more. Don’t rely too much on your data. Don’t expect it to be perfect. Don’t focus on making it perfect. Make sure it’s “good enough” and go from there.

Talk about milking a topic in multiple channels! The Marketing News Radio interview I did on multidimensional lead scoring is now available. And…we’ve now added a “video white paper” to the mix, which you can view on the Bulldog Solutions web site.

The story behind how this 8-minute video came to be is kinda interesting. At Bulldog Solutions, we’re constantly (too constantly, at times) looking for ways to put new Internet-based technologies to work for our clients. Generally, we like to put them to work for ourselves first. First, that lets us suffer through the hiccups and snags that come the first time you do anything new. And, second, it gives us something that we can use as an example of what it is our clients would get.

The truth is, this video white paper was driven more by the fact that we were starting to talk to prospects where the medium seemed like a fit than it was part of a glorious, fully-baked, multi-channel strategy for marketing multidimensional lead scoring.

In the case of the radio interview, our Field Marketing Manager pinged me months ahead of time about doing the show. She then wrote up an abstract for the interview. We had a timeline that we worked to, and the preparation included a pre-call with the interviewer the week before the show to get everyone on the same page and lay out the overall flow for the interview. And, the show itself was on my calendar for months.

Contrast that with the video white paper experience. I was in Austin for a week a couple of months ago. I flew in on Monday morning. On Tuesday, the same Field Marketing Manager said, “Tim, we’re going to be shooting a video of you talking about multidimensional lead scoring for 5-10 minutes on Thursday. It’s no big deal — just you talking through the concept. But, we’ve got a prospect who is interested in having us do some of these for them, and we need an example.” On Thursday morning, at the hotel, I realized that I’d brought all of one collared shirts on the trip (it’s Austin, we’re a startup — casual attire is the norm). The shirt was dark blue with some plaid-like stripes. I wore it to the office, and our video guru took one look and said, “We’re shooting you on a green screen — that’s a risky shirt to wear, as you may wind up with holes in your body on the final video.” At lunch, I headed to Stein Mart to pick up something that was solid (I clothes shop voluntarily approximately once every 3 years — this was not “voluntary”). I then spent an hour in one of our conference rooms with the A/C turned off (to eliminate background noise) doing three takes on the video.

The project then actually languished for a while (the urgent opportunity died or evolved to be something else — I’m not sure which). But, our video guru jumped back on it last week…and the end result turned out much better than I imagined it would, given the material he had to work with!

It’s a bit of a curse in the world of social media and personal branding to sport a John Smith-like name, and “Tim Wilson” certainly qualifies. From a data management perspective, I definitely fall in the “FirstName + LastName” is nowhere close to a unique identifier. I’ve had interesting “<groan>” moments over the years on that front:

• High School in Sour Lake, Texas — my sister’s name is Kim, and our 400-person high school had two Kim Wilsons and two Tim Wilsons…and a lousy intercom. My sister and I kept our noses pretty clean, which was not the case for our counterparts (who were not related to us or to each other); we both got used to ignoring any between-class announcements for “..im Wilson, please come to the office.” Generally, someone was in trouble, and, generally, it was not one of us.
• Registering for College Classes — as a small-town teenager in the big city of Cambridge, Massachusetts, the registration card that got handed to me was for Tim Wilson the graduate student, who was also registering for classes. That caused a bit of confusion that took a day or two to straighten out. Interestingly, that Tim actually cropped up on Jeopardy last month. I catch maybe 10 minutes of Jeopardy a year, but I happened to catch the end of Final Jeopardy a few weeks ago and saw Tim Wilson. He’s now a professor. In a quick Google search to track him down — using “Jeopardy” as part of the search — I turned up a post where the blogger calculated (with some help from the commenters…one of whom was Tim promoting his own appearance) the maximum theoretical one-day take on Jeopardy: $566,400.
• Buying a House while On the Lam — when Julie and I were working on the mortgage approval paperwork for our first house, we hit a snag, in that “I” had a 4-year-old outstanding arrest warrant in Austin. We’d only lived in Austin for a year, and I had had no brushes with the law of any sort during past visits to the city (see first bullet above).
• I’m a Funny Guy…but Not Professionally — in the early days of e-mail, I used to get occasional messages from random people telling me how funny I was and asking when I would next be in [random city]. These were followers of Tim Wilson the Deep South Comedian. I was tempted to respond by telling them I would be at [some club in the city they were asking about] on [a date in the near future] as an unpromoted surprise show, and that I would leave four tickets in their name at the door. I ultimately decided that would be too cruel. <sigh> See bullets above.

You get the idea. So, when I started this blog, I partially stole a page from Avinash Kaushik’s figurative book, in that his blog is Occam’s Razor by Avinash Kaushik. “Gilligan on Data by Tim Wilson” was born.

So, where did “Gilligan” come from? That’s a trail name I picked up when thru-hiking the Appalachian Trail from Georgia to Maine back in 1993 when I graduated from college. It’s a tradition on the Appalachian Trail to adopt a new moniker for the duration of your hike. There are practical reasons for this (”Buck,” who I met and hiked with quite a bit on the trail, was a female who was hiking alone, and she thought it wise not to advertise that fact when signing trail registers along the way). There were descriptive names — “Chowhound” really did eat like a horse, and “Bearanoid” did have a heightened fear of encountering a black bear.

At the time, the Appalachian Trail was 2,142.7 miles from the summit of Springer Mountain in Georgia to the summit of Mount Katahdin in Maine. The length of the trail varies from year to year as improvements are made and routes are updated, but that is what it was in 1993. I’m using my personal blog to enter my journal entries from that experience 15 years later. “Gilligan” — a name chosen largely in haste one evening with the assistance of the other hikers I was camping with that evening — remains fairly appropriate. I’m still clumsy. I still get myself into any number of improbable predicaments. And I’m skinny as…okay, so maybe that Bob Denver characteristic no longer particularly applies.

If “I had a common name” is one of the larger hurdles I face in my life, I’ll count myself fortunate, I suppose. But, sites like searchme.com just aren’t as much fun for me as they are for those of you who have more distinct names.

Awareness (the company) has a Measuring ROI Around Web 2.0 webinar this Thursday, May 22, at 2:00 PM EDT. That’s heavy on the buzzwords, but it sounds like it might have some interesting information. And, I found out about it thanks to a mention on Twitter from Connie Bensen, who will be leaving her new kayak behind and heading to London and Paris for some R&R, so will be missing the live event herself.

Unfortunately, it partially conflicts with Kalido’s What’s Behind Your BI? webinar, which starts at 2:30 PM EDT, and it conflicts with Fusing Field Marketing and Sales, which Hoover’s and Bulldog Solutions are putting on at 2:00 PM EDT on Thursday as well.

It looks like I’ll be doing some on-demand catch-up after the fact.

You keep hearing the buzz about this “multi-dimensional lead scoring” thing. You’ve seen some pictures in a blog entry. You might even have read the white paper. You might have even heard rumors that there is a 10-minute video version of the white paper in the works. Maybe you even read up on how Bulldog Solutions incorporated multi-dimensional lead scoring in their lead reengagment program.

But, what if none of these cover your preferred mode of learning? By my tally, we’ve missed three channels for getting out and communicating on the subject:

• Text messages
• Fax
• Radio

Well, you’re simply out of luck on the first two, but radio is coming! I’ll be spending an hour next week — Wednesday, May 21st from noon to 1:00 on the American Marketing Association’s Marketing News Radio. The experience promises to be…nothing at all like the picture depicted above. I’ll be sitting in my office on the phone. But, the AMA does a great job with the production of these shows.

The challenge for me is that the subject plays right into a nice, simple picture. And there won’t be any pictures. That’s had me thinking about some analogies and anecdotes that will help make the core concepts resonate a bit more. We’ll see how it goes.

Photo by yarra64

Tamara Gielen has a post titled Triggered Email Is Only As Good As Your Data over on the B2B E-mail Marketing blog. She describes a scenario where you want to send a satisfaction survey to customers 90 days before their contract expires with your company, and adds on some logic of resending the survey to customers who do not respond to the initial request. Her point is — that may be a lot trickier than it sounds if you’re actually living in the real world:

• The data needed to trigger and send these e-mails lives in different systems
• You don’t have a good way to determine whom at the account should receive the e-mail (you probably don’t want to send the survey to everyone you have in your systems for the acccount)
• You don’t have a mechanism for updating your data when someone leaves the account and is no longer an employee there
• The list goes on…

The real world would put you in a position of needing to make some unpleasant decisions:

• Do you cast the net broadly and risk sending a non-applicable e-mail to a bunch of people in your database, or do you cast the net very narrowly and only send to people you are absolutely sure are the right folk…but then limit the impact of doing the survey in the first place?
• Do you limit the amount of manual cleanup on the data, or do you engage your sales and account management groups to manually flag who should receive the invitation (or something in between)?
• Do you try to explain all of the caveats of the data to the person who initiated the project, or do you just make a series of judgment calls and be prepared to defend/explain them if asked later?

The truth is that, in most cases, this sort of initiative does make sense, but it also requires making a long list of trade-offs, assumptions, and judgment calls to balance the expected impact with the effort required.

The entry brought to mind some interesting data integrity snafus that I’ve come across in my personal life:

• For years, the phone company thought my name was “Jim” rather than “Tim” — initially, this only impacted caller ID, but, over time, that list got sold to various direct marketers, and “Jim” started getting junk mail
• Several months after we moved to Ohio, I went into REI in Austin, and when they looked up my account, they had my Ohio address with a phone number of my former employer — none of this was information that we had ever explicitly provided
• After moving to Ohio, we signed up for a Giant Eagle grocery store card, with our new address and phone number; somehow, a couple of months later, when I didn’t have my card with me and they had to look me up…they had the address as our address in Austin!
• I recently shifted my cell phone plan from my wife’s and my joint account with T-Mobile to my company’s account with AT&T; I found out last week that I show up as “Julie Wilson” on caller ID when I make calls with my cell phone
• For years, Microsoft was convinced that I was a high-level IT manager responsible for my company’s system administration and infrastructure — I’ve never been within a light-year of that sort of role

All of this is to say that the data is messy. It’s never going to be perfect. Spending time and energy on data integrity initiatives makes sense, but that has to be balanced with the practical reality of the world — short of an Orwellian society, the data is always going to have some level of inaccuracy. Understand the data. Understand what you’re trying to accomplish with it. And then make a judgment call.

Photo by Daquella manera

One of the areas that I’ve spent a lot of time focusing over the past six months is lead scoring. Specifically, multidimensional lead scoring. I’ve written about it here before, and my white paper is still available on the Bulldog Solutions web site. We shot a 10-minute video white paper a few weeks ago, which should be available soon. The slides I talk to in that are available below courtesy of slideshare.net.

| View | Upload your own

And, just yesterday, Laura Ramos posted a topic diving into lead scoring over on the Forrester Interactive Marketing blog.

All this is to say that the topic remains very much on the top of my mind. And, as is no great surprise, I’ve got additional thoughts that I’m still working out on the subject — mostly driven by our experiences putting multidimensional lead scoring into practice at Bulldog Solutions. I expect I’ll be working through some of these over on the Bulldog blog in the coming weeks. But, I thought I’d start here to at least get a little bit of a list going.

To start with, we haven’t uncovered anything that invalidates any of the concepts in the white paper. Quite the contrary, actually. It has been very well received by everyone to whom it has been presented and has sparked some interesting conversations. And, as we’ve put it into practice, it’s holding up to be as viable an approach as we expected.

The two areas that I’m most looking to flesh out my thinking are as follows:

• The Buying Cycle Dimension — Partly because the paper itself was getting to be darn long, and partly because we were not yet in the process of implementing a third dimension, the paper only briefly speaks to the “buying cycle position” dimension of lead scoring. The paper was initially titled “2-D” lead scoring rather than “Multidimensional” lead scoring, and our CMO pointed out that this was overly limiting — we were only implementing two dimensions at the time, but we knew there were others that we would be moving on to. Specifically, he was referring to the buying cycle position. This is a bit dicey to work out, but it’s a straightforward concept: a lead who is checking out detailed product specifications and pricing on your web site is likely much closer to making a purchase decisions than a lead who is simply reading white papers or analyst articles on a topic.
• Multiple Offering Types — this was flat out an oversight on my part, largely due to the fact that I’m at a startup that does not yet have a wildly extensive set of offerings. But, in my prior life, this issue very much was at the fore, so missing it was definitely a whiff. The issue is that one lead may be highly qualified for one type of offer but very much not qualified for another offer. In our case, we have offerings where the economic buyer is a marketing executive. We have other offerings where the economic buyer is a marketing manager. The marketing executive is actually not a qualified lead when it comes to some offerings. And vice versa. So, how does this get handled with the lead scoring? I think that this situation is an indication that you may have multiple profile scores — one for each broad offer type. I don’t think the engagement score changes from offer to offer — this is a measure of engagement with the company / the brand and is offering-independent. And, I don’t think the buying cycle position will differ, either. But, I’m still working this out. We’re testing it out manually with some upcoming activities.

One of the benefits of multidimensional lead scoring is that it does make it manageable to add additional nuances and complexity over time. And, while the lead scoring is primarily a lead qualification mechanism, it serves other purposes: it’s one factor in segmenting your database for different nurturing programs (or content within the nurturing programs) — based on what quadrant in the multidimensional matrix the lead falls in; and, it can be used for lead routing — again, based on what quadrant in the lead matrix the lead is in may dictate whether the lead is routed to an inside sales organization or directly to another organization within Sales.

As always, let me know if you disagree or can add color from your own experiences.

I had an experience over the past week that, in hindsight, I really should have been able to avoid. The situation was basically this: several different people had made comments in passing about how we were probably “overcommunicating” to our database. “Overcommunication” being the tactful way to say “spamming.” In this case, I can actually trace the perception back to at least two different highly anecdotal events, which then spawned comments that led to assumptions, and so on.

Now, I am all for diligent database management, especially when it comes to how often and with what content we communicate with our contacts. My general sense was that we could be doing better, but we were far from reaching a crisis point (I lived through a situation at another company where we did reach that crisis point, and there were plenty of telltale signs leading up to that). “I can pull some quick data on that to at least get some basic facts circulated,” I innocently thought. And, that’s what I did.

I knew going in that, while the data was one thing, the definition of “good” vs. “bad” was likely all over the map, so I wasn’t likely to change many people’s opinions as to the situation by simply sharing the data. So, I shot an e-mail out to a group of interested parties and told them I had the data, and I’d be happy to share it, if they shared with me their opinions as to what an acceptable maximum of communications per week and per month would be.

As I suspected, I got a wide range of responses, and most of the responses had some form of qualifier — well-founded qualifiers regarding the type of communication, actually. So far, so good.

I then shared the data that I had spent 15 minutes compiling in a way to make for easy analysis, still knowing that there was no clear good/bad definition, and there was no clear hypothesis being tested or action being planned that this analysis would influence. The data did show a few things unequivocably — really just highlighting that the concerns were somewhat well-founded and that discussions should continue amongst the people who already tacitly owned the situation. But, it also spawned requests for additional data that was more curiousity-driven than actionability-driven. Several people asked that the data be pulled with their particular qualifiers addressed. Most of these people were in no position to actually take any action based on the results. And, unfortunately, as reporting and analysis systems can sometimes be — applying the qualifiers would have turned the analysis into a highly manual, multiple man-hours exercise, whereas the high-level, basic pull was a 15-minute task.

On the one hand, I could ding our data storage system. By golly, Tenet No. 1 of good BI/DW design is to design for flexibility, right? In this case, the system limitations are actually a boon — they give me an out for simply saying, “No,” rather than the much more involved discussion that begins, “Why?”

It’s a punt, I realize. And not an out I would take if it was throwing anyone in IT under a bus.

My point is that “interesting” can be a Siren Song that dwarfs the pragmatism of “actionability.”

As I’m thinking about how to categorize this post, I’m realizing that, despite the fact that it includes scatter plots, it’s a bit off topic for the sort of things I generally put in this blog.

But, back in December, in an afternoon of frenzied, yet focussed, exposition, I dashed out a 15-page pager on the topic of multidimensional lead scoring. That’s right. 15 pages. But with lots of pictures! It laid out, as clearly as possible, the rationale and “how to” for something we’ve been working on in the R&D labs at Bulldog Solutions for quite some time. It’s something we actually use ourselves…but we hadn’t taken the time to sit down and try to explain it clearly. Of the 8 scatter plots, here’s one that comes mid-way through the explanation and sort of tells the whole story:

The main point of going to a lead score that has two dimensions is to recognize that there are multiple unique facets of your ideal lead. It’s not just whether he/she is the perfect “profile” — a director-level decision maker with budget authority at a company that is $100 million or greater in the medical device industry (for instance) — but it’s also important to determine if they have a clue who you are and have any interest in talking to you! That’s where the “engagement level” comes in.

Your best leads fall in the top right area of the scatter plot — you want to talk to them, and they seem interested in talking to you. That’s it in a nutshell. The paper goes into more step-by-step detail as to what that really means, how to implement it, and what to watch out for. There’s a longer introduction/overview (no registration required) on the Bulldog Solutions web site. To get the full paper, you have to register (or, shucks, shoot me an e-mail and I’ll send it to you directly). An unnamed source (no, not a relative or a current co-worker!) made the following comment about the paper:

Really like the way you structured the approach to the topic and broke it down into steps that are easily internalized as well as actionable and measurable. As always, I am in awe of your writing skills, which made for a enjoyable read. But most of all it provoked some new thoughts on a topic that had fallen of my radar…

That was pretty high praise from an experienced B2B marketer who is a notoriously straight shooter (painfully so, at times).

Now, as I was working on the paper, with Twitter running on my second screen…and jumping over to Facebook periodically…and checking out the various blogs I subscribe to…

I couldn’t help but think about how “B2B lead scoring” fits in with “social media.” Actually, this was more than idle distraction — I’m also working on a project that brings those two concepts together for our own internal operations. On the one hand, “lead scoring” still feels a bit old school. I mean, we’re focussed on watching what people are doing and, basically, pouncing on them with a rabid salesperson when our processes spit out that they’ll be easy prey. Right? Well, not really. By introducting the “engagement level” dimension, we’re actually saying, “We’re happy to keep feeding you useful information. We’re actually motivated to nurture you without a hard sell…until you start poking around on our site and showing that you think we’ve got some credibility.” And, ideally, we’ll also try to snoop out where in the buying process you are and not reach out to you personally until you’re in the middle to later stages.

I guess I see multidimensional lead scoring as a bridge between the past — Marketing gets the leads and tosses them over to Sales to call ‘em up and sell — and the future — hyper-interconnectivity and information sharing among peer groups, where the company’s only option is to have a great product and support their community of potential users with high quality information through whatever channel the users want to consume it and engage with it.

What do you think? Does this make sense, or am I fooling myself?

That’s gapingvoid as in “Hugh Macleod draws quick, insightful, witty things.”

WARNING: the “Who Owns My Data?” cartoon is highly irreverent (it’s gapingvoid, people!) and includes profanity.

‘nuf said — it’s a chuckler: http://www.gapingvoid.com/Moveable_Type/archives/004409.html

There’s really nothing I can add to this.

Except that I’m still chuckling. Or, maybe I’m cackling maniacally, as my day started at 3:45 EST this morning in Columbus, and it’s now 10:00 CST in Austin!