Gilligan on Data by Tim Wilson » Analysis

Integrated View of Visitors = Multiple Data Sources

Tim Wilson — Tue, 22 Jun 2010 18:30:25 +0000

I attended the Foresee Results user summit last month, and John Lovett of Web Analytics Demystified was the keynote speaker. It’s a credit to my general lack of organization that I wasn’t aware he was going to be speaking, much less keynoting!

John showed this diagram when discussing the importance of recognizing your capabilities:

The diagram starts to get at the never-ending quest to obtain a “360 degree customer view.” A persistent misperception among marketers when it comes to web analytics is that behavioral data alone can provide a comprehensive view of the customer. It really can’t — force your customers to behave in convoluted ways and then only focus on behavioral data, and you can draw some crazily erroneous conclusions (“Our customers appear to visit our web site and then call us multiple times to resolve a single issue. They must like to have a lot of interactions with us!”).

Combining multiple data sources — behavioral and attitudinal — is important. As it happened, Larry Freed, the Foresee Results CEO, had a diagram that came at the same idea:

This diagram was titled “Analytics Maturity.” It’s true — slapping Google Analytics on your web site (behavioral data) is cheap and easy. It takes more effort to actually capture voice-of-the-customer (attitudinal) data; even if it’s with a “free” tool like iPerceptions 4Q, there is still more effort required to ensure that the data being captured is valid and to analyze any of the powerful open-ended feedback that such surveys provide. Integrating behavioral and attitudinal data from two sources is tricky enough, not to mention integrating that data with your e-mail, CRM, marketing automation, and ERP systems and third-party data sources that provide demographic data!

It’s a fun and challenging world we live in as analysts, isn’t it?

(On the completely off-topic front: I did snag 45 minutes one afternoon to walk around the University of Michigan campus a bit, as the conference was hosted at the Ross School of Business; a handful of pictures from that moseying is posted over on Flickr.)Similar Posts:

Feed enhanced by Better Feed from Ozh

From Data to Action — The Many Flavors of Latency

Tim Wilson — Wed, 09 Jun 2010 16:30:14 +0000

I was flipping through the slides from a workshop that Teradata put on at The Ohio State University several months ago, and one of the diagrams jumped out and resonated with me. As I did some digging, it turns out this diagram has been floating around since at least 2004, if not for longer. It was created by Dr. Richard Hackathorn of Bolder Technology Inc. (BTI).

There are a slew of lousy recreations of the diagram (the original diagram wasn’t so hot, either). Rather than recreating it myself, I just snagged one of the cleaner ones, which came from a 4-year-old TDWI article:

The point of the diagram, as well as of most of the derivative works that reference it, is that the value of information has a direct relationship to the speed with which you can react to it. And, there are three distinct things that have to happen between the business event that triggers the information and ation actually being taken.

I don’t know if there is any real math or science behind the shape of the curve. As diagrammed, this says that you’ve already lost most of your value by the time you get to the “decision latency” point in the process. I don’t know that that is necessarily true in most cases. The diagram supports the assertions by all of the various BI/data tool vendors that data needs to be available in near real-time (and, of course, that’s something that all of the vendors claim they are better at than their competition).

But, is the data latency and analysis latency really the big value driver for marketers? In some cases, the data latency is a structural issue — conducting a campaign where the people exposed to it are likely to not convert for anywhere from 1 to 30 days…means you really need to wait for 30 days to see how the campaign played out. Analysis latency is real…but this really can be broken into two pieces: 1) the time to do the analysis and get it packaged for delivery, and 2) the time to schedule/coordinate the information delivery. And, then, certainly the decision latency is real.

In short, the “action time” components totally make sense, and it’s good to understand them. The shape of the curve, though, doesn’t necessarily stand up to scrutiny when looked at through a marketer’s lens.Similar Posts:

Feed enhanced by Better Feed from Ozh

Answering the “Why doesn’t the data match?” Question

Tim Wilson — Tue, 18 May 2010 17:05:51 +0000

Anyone who has been working with web analytics for more than a week or two has inevitably asked or been asked to explain why two different numbers that “should” match don’t:

Banner ad clickthroughs reported by the ad server don’t match the clickthroughs reported by the web analytics tool
Visits reported by one web analytics tool don’t match visits reported by another web analytics tool running in parallel
Site registrations reported by the web analytics tool don’t match the number or registrations reported in the CRM system
Ecommerce revenue reported by the web analytics tool doesn’t match that reported from the enterprise data warehouse

In most cases, the “don’t match” means +/- 10% (or maybe +/- 15%). And, seasoned analysts have been rattling off all the reasons the numbers don’t match for years. Industry guru Brian Clifton has written (and kept current) the most comprehensive of white papers on the subject. It’s 19 pages of goodness, and Clifton notes:

If you are an agency with clients asking the same accuracy questions, or an in-house marketer/analyst struggling to reconcile data sources, this accuracy whitepaper will help you move forward. Feel free to distribute to clients/stakeholders.

It can be frustrating and depressing, though, to watch the eyes of the person who insisted on the “match” explanation glaze over as we try to explain the various nuances of capturing data from the internet. After a lengthy and patient explanation, there is a pause, and then the question: “Uh-huh. But…which number is right?” I mentally flip a coin and then respond either, “Both of them” or “Neither of them” depending on how the coin lands in my head. Clifton’s paper should be required reading for any web analyst. It’s important to understand where the data is coming from and why it’s not simple and perfect. But, that level of detail is more than most marketers can (or want to) digest.

After trying to educate clients on the under-the-hood details…I almost wind up at a point where I’m asked the “Well, which number is right?” question. That leads to a two-point explanation:

The differences aren’t really material
What matters in many, many cases is more the trend and change over time of the measure — not its perfect accuracy (as Webtrends has said for years: “The trends are more important than the actual numbers. Heck, we put ‘trend’ in our company name!”

This discussion, too, can have frustrating results.

I’ve been trying a different tactic entirely of late in these situations. I can’t say it’s been a slam dunk, but it’s had some level of results. The approach is to list out a handful of familiar situations where we get discrepant measures and are not bothered by it at all, and then use those to map back to the data that is being focussed on.

Here’s my list of examples:

Compare your watch to your computer clock to the time on your cell phone. Do they match? The pertinent quote, most often attributed to Mark Twain, is as follows: “A man with one watch knows what time it is; a man with two watches is never quite sure.” Even going to the NIST Official U.S. Time Clock will yield results that differ from your satellite-synched cell phone. Two (or more) measures of the time that seldom match up, and with which we’re comfortable with a 5-10 minute discrepancy.

Photo courtesy of alexkerhead

Your bathroom scale. You know you can weigh yourself as you get out of the shower first thing in the morning, but, by the time you get dressed, get to the doctor’s office, and step on the scale there, you will have “gained” 5-10 lbs. Your clothes are now on, you’ve eaten breakfast, and it’s a totally different scale, so you accept the difference. You don’t worry about how much of the difference comes from each of the contributing factors you identify. As long as you haven’t had a 20-lb swing since your last visit to the doctor, it’s immaterial.

Photo courtesy of dno1967

For accountants…”revenue.” If the person with whom your speaking has a finance or accounting background, there’s a good chance they’ve been asked to provide a revenue number at some point and had to drill down into the details: bookings or billings? GAAP-recognized revenue? And, within revenue, there are scads of nuances that can alter the numbers slightly…but almost always in non-material ways.

Photo courtesy of alancleaver_2000

Voting (recounts). In close elections, it’s common to have a recount. If the recount re-affirms the winner from the original count, then the results is accepted and moved on from. There isn’t a grand hullabaloo about why the recount numbers differed slightly from the original account. In really close races, where several recounts occur, the numbers always come back differently. And, no one knows which one is “right.” But, once there is a convergence as to the results, that is what gets accepted.

Photo courtesy of joebeone

That’s my list. Do you have examples that you use to explain why there’s more value in picking either number and interpreting it rather than obsessing about reconciling disparate numbers. I’m always looking for other analogies, though. Do you have any?

Similar Posts:

Feed enhanced by Better Feed from Ozh

A Record-Setting Web Analytics Wednesday in Columbus with CRM Metrix

Tim Wilson — Tue, 02 Feb 2010 16:51:04 +0000

Last week’s Columbus set a new record for the meetup — we had exactly FIFTY attendees, which was a great showing. Part of the large draw was undoubtedly the event sponsor, CRM Metrix (@crm_metrix on Twitter).

Pre-Meal Networking (and a Friendly Wave from Jonghee!)

Hemen Patel, CRM Metrix CTO, facilitated a lively discussion about incorporating the voice of the customer in web site measurement and optimization.

Hemen Patel Presents

Hemen walked through a brief deck (below) that sparked some great back-and-forth with the crowd.

Hemen Patel CRM Metrix: Columbus WAW, Jan 27, 2010

View more presentations from CRM Metrix.

A Rapt Audience

Monish Datta Asks a Question

With a crowd of fifty people, not only did I not get to meet the first-time attendees, but I barely had a chance to say, “Hi” to some of the long-time regulars. I guess we’ll just have to have another one in February (I’m working on it!) so I’ll get that chance!

Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2010. | Permalink | One comment | Add to del.icio.us
Post tags: CRM Metrix, WAW, WAW Columbus

Feed enhanced by Better Feed from Ozh

The Fun of Facebook Measurement

Tim Wilson — Mon, 11 Jan 2010 14:06:27 +0000

If you are a marketer, Facebook is important — the number of active users of the site exceeds the population of the United States, and it’s growth is going to do nothing but increase. Check out the Facebook statistics page for a slew of numbers that are all…big. Because of the growth of Facebook as a critical marketing channel, a hot topic around the office right now is “Facebook measurement.” It’s a tricky topic that reminds me of the early days of web analytics: there’s some basic stuff that’s easy to measure, and it’s basically helpful, but there’s a lot more that can’t be measured or can’t be measured well, and that’s where the real value is.

There are (at least) five different aspects of Facebook that can be measured:

Facebook display ads — I’m not going to cover that at all; I haven’t spent a whole lot of time digging into it with our clients, so I’m not going to write about it. Check out What about advertising on Facebook? over on the Site Pro Specialties site for a quick overview of the ins-and-outs and their experience with Facebook display ads
Facebook applications — I’m not going to cover this, either, largely because my experience on the subject is pretty limited, but also because Facebook apps annoy the bejeezus out of me, and I don’t want to have to stifle a gag reflex by writing about measuring them
Facebook groups — Holy cow! Another topic I’m not going to cover! Since Facebook pages came on the scene, that’s where brands tend to be living more, and Facebook provides more measurement support for pages, so that’s where I’m going to focus the most
Facebook pages — I’ll focus on these quite a bit, as this is an area that brands are really starting to settle into as a formal presence on Facebook
General Facebook activity – this is an area where measurement is highly limited, but I’ll lay out what is there and what I hope comes sooner rather than later

We’re in a bit of an ugly period from an analyst’s perspective, in that Facebook hasn’t made supporting marketers a high priority beyond paid advertising. And, the company is being very cautious on the privacy front (which, from a consumer’s perspective, is a good thing!). The easiest way to reduce the risk of a PR blowup from misuse of Facebook data is to limit the availability of that data to marketers. I can’t blame them, but it doesn’t mean they’ve made my life easy on that front.

Ready? Let’s go! Here’s a quick set of links to use to jump down to specific topics:

Facebook Pages — Fan Count
Facebook Pages — Facebook Insights Data
Facebook Pages — Web Analytics Measurement, Part I (The Ugly Part)
Facebook Pages — Web Analytics Measurement, Part II (The Pretty…but Short…Part)
General Facebook Activity — Web Analytics Measurement
General Facebook Activity — LOTS is Missing

Facebook Pages — Fan Count

Facebook pages are a way for brands to establish a formal, managed presence on the site. They’re easy to set up, and they can range from the very simple and unused (see the Smuirfield Golf Club page) to the very elaborate and active (see the Victoria’s Secret PINK page). For any page, regardless of whether you are an admin for it or not, you can see the total number of fans at a point in time — the example below is from the Slate Political Gabfest page:

That can be useful for a couple of reasons:

Organically grown pages — it’s fairly common for major brands to have their fans set up pages and grow a decent following; being able to tell the reach of those pages can help identify when outreach or integration might be in order
Competitive research — it can be tedious, but assessing the size and growth of competitor fan pages over time can provide insight (albeit limited insight) into their overall social media strategy and their ability to execute

There is no way to measure the change in any page’s fan count over time other than periodically going and checking and recording it. And, what does total fans tell you? It tells you something…but not as much as you might like. More on that later.

Facebook Pages — Facebooks Insights Data

Now, if you have admin access to a Facebook page, you can get much richer data, and you can get a historical view of some of that data. On the page itself, above the Fans box, is the basic Facebook Insights box:

While this looks encouraging, it’s not particularly useful. “Post Quality” sounds like a good idea (pick any measure of activity volume, and you can say, “It’s not just about quantity — it’s about quality!” and sound smart), exactly how Facebook determines quality is a bit of a mystery. From the Facebook Help Center:

The Post Quality score measures how engaging your Posts have been to Facebook users over a rolling seven-day window.

Post Quality is an important indicator for how fans gauge your posts. This score is calculated with an algorithm that takes into account your number of posts, total fan interactions received, number of fans, as well as other factors.

It’s a measure that’s almost too vague to be useful. And, in practice, the historical trending of Post Quality shows that something about the way it is measured makes it pretty non-actionable — even for pages that have a high level of fan engagement consistently, a trendline of Post Quality goes all over the place.

So, now we dive into the real meat of Facebook Insights, which initially looks like a nice, juicy T-bone, but which turns out to be more like a pretty lean cut of venison. The See All link in the Insights box brings up the main Facebook Insights page (click on the image below to view a larger version):

This page has the second not-nearly-as-useful-as-you’d-like measure: Active Fans. Facebook is even more fuzzy about how this is calculated than it is about Post Quality. And, historical data is not available. In my experience, Active Fans is a pretty big crap shoot — it varies widely from day to day and, since it’s not easy to get historical data, it’s a mess to try to analyze what is going on and how it is really changing over time with any granularity. Conceptually, active fans are high-quality fans. In my experience, the number of active fans in any given period is a tiny fraction of the overall fans. So, the million-dollar question — “What is the value of a Facebook fan?” — should probably include a separate calculation for an “active fan.” But, “active fan” is such a messy measure with such limited availability, that it’s barely worth pursuing until it’s more accessible and explainable.

Most of the other measures, though, have historical data available via the graphs shown on the page. Some underlying data can be exported as a CSV or Excel file with granularity at the individual day level. Two wrinkles with that data, though:

The timing of the data updates is inconsistent, and it doesn’t seem like “if data is there, it’s good data” — a note in the bottom of the Insights window states: “Please allow 48 hours for data to be available for a daily report;” it’s common to see some data for a given day populated while other data for the same day isn’t; while I don’t feel like “real-time” data is generally warranted, the 48-hour lag can put a real crimp in effectively weekly reports, as well as in getting a good, timely view into the results of a new Facebook campaign
The data doesn’t appear to be kept forever; it used to seem like data dropped off once it was ~3 months old, but the actual range of available data seems to vary, and Facebook doesn’t provide information on the subject; we’re in the practice of exporting all available data monthly so that we’ve got it retained offline for our clients

The main export option is the Fans and Interactions export. The other two exports that are available are Demographics and Country. The demographics export simply shows, by day, the number of fans of a given age range/gender. The demographics of active fans over time is not available, unfortunately. The Country export simply shows the number of fans from each country over time.

Now, Fans and Interactions is where the most useful information is. You can get a great look into how fan growth has been growing over time — new fans, total fans, unsubscribes, etc. This provides a way to do a classic “leaky bucket” report — how many fans you are losing compared to how many new fans you are acquiring. Unsubscribes are interesting, because that means fans have explicitly removed themselves as fans rather than simply choosing to remove the page’s updates from their feeds. Which…alludes to the Big Wrinkle when it comes to fans — just because someone is a fan of your page doesn’t mean they’re seeing anything that happens on the page — it’s very easy for users to hide all updates from a page from their feeds. And Facebook doesn’t provide data as to how many people have done that!

Fans and Interactions also provides data on the number of “interactions” which is the sum of all of the likes, posts, and comments that occur each day. In my mind, a “like” is a pretty light interaction, while a post or a comment is a more significant interaction, because a fan actually had to put together words to express an idea. Facebook Insights provides details for each type of interaction, too, though, so you can measure the different types of interactions. This export provides four types of interactions: Likes, Comments, Wall Posts, and Discussion Posts. It can get a little confusing as to which type of user activity occurs where, so be prepared to click back and forth between your page and the data for a while to get the hang of it (I’d write it out here, but this post is already getting pretty long and unwieldy!). The data also includes “Posts” — these are your posts rather than fan posts.

Finally, Fans and Interactions provides basic web analytics data. VERY basic. Page views, unique page views, audio plays, video plays, and photo views. At a very high level, this is useful information, as it’s a measure of whether the page is sufficiently engaging to drive people to visit (note that someone by no means has to be a fan to visit the page, view content, and comment on it — if a page has a lot of page views but a small number of fans, then it may be an indication that users would like to engage with the brand in Facebook, but the actual content/activity occurring on the page is not strong enough to get them to become a fan once they actually visit). Data that is not provided includes: which tabs of the fan page were visited, which videos were played (and how much of the video was viewed), and which photos were viewed. Supposedly, this sort of capability is in the works at Facebook, but no one I’ve talked to is committing to any dates for them to roll out.

Facebook Insights also doesn’t provide data on:

Suggest to Friends usage
Subscribe via SMS usage
Add to My Favorites usage
The ability to export wall posts, discussion posts, and comments (more on this in the last section of this post)
Page visit frequency

The lack of Suggest to Friends data is particularly painful — this would be a powerful measure of how engaging the content on the page is, and there is zip when it comes to any visibility into that.

I expect that Facebook Insights will evolve over time to provide more content-level detail, as well as usage of other “page” features. It’s less likely that Insights will evolve to include user-level detail due to privacy concerns, although it’s not inconceivable — this would be the equivalent of having access to detailed behavioral data for users who have registered with your web site and are making subsequent visits.

Facebook Pages — Web Analytics Measurement, Part I (The Ugly Part)

Depending on how you squint when you look at it, a Facebook fan page for your brand is just an off-site extension of your web site — just like any content you host on a third-party site (job postings that are hosted by a recruiting site, events that get managed through a third-party event management site, etc.). For third-party sites whose bread and butter is extending the content offerings from web sites, it’s common to deploy the main site’s web analytics page tag on the third-party content pages. There are myriad ways to set up the reporting for that in any web analytics tool — Google Analytics, SiteCatalyst, Webtrends, Coremetrics, etc. In theory, Facebook pages would be the same way — just as you can embed all sorts of rich content on custom tabs, it seems like you would be able to insert your web analytics page tag on the pages where you have heavy control over content.

But, Facebook currently has an industrial-sized monkey wrench inserted into that approach by not allowing Javascript to execute on its pages. Presumably, this gets back to privacy — concern that opening up the site to allow scripts to execute would open up the potential for some page admins to figure out a way to capture too much personal information from visitors/fans of their pages.

So, what options are there? There are several, but they’re all clunky.

[UPDATE: The next little section is continuing to evolve, as I've been doing a lot of digging and experimentation in this area, finding both new roadblocks as well as trying out workarounds]
Generally speaking:

Use an iFrame for the content and put your usual page tag in it — the wrinkle here is that you can’t put an iFrame on a custom tab; it has to be a standalone application canvas. Now, you can include within the frame a dummied-up re-rendering of the tabs on your fan page, but that’s really not ideal. There is a mildly helpful thread on the Google Analytis forum on the subject, as well as a thread on the Facebook developers forum with some useful tips
Either use the
Build a custom solution that makes an image (or some other asset) call to a reporting server you manage — you would need a unique call for each activity you want to track — and then sift through the server log file to construct what’s happened; you’re going to run into challenges with caching of images, though, so this will be incomplete data at best

All of these only work on pages where you have a decent level of control over the content, which leaves out the Info, Photos, Videos, and Discussion tabs…and it’s a little dicey as to what’s doable on the Wall. But, presumably, it’s the custom tabs where you’re investing the most resources to develop content, so that’s a pretty good place to get some more granular web analytics data.

We’ve actually managed to get some tracking of interactions occurring on a user’s wall within a Flash-based status update using Google Analytics (using the third approach above), and we’re close to rolling out some pages that will use the second item above with Webtrends (which will track both interactions within a Flash app as well, we expect, as traffic to individual custom tabs).

[End of section that is still evolving]

In short, though, this is pretty messy.

Facebook Pages — Web Analytics Measurement, Part II (The Pretty…but Short…Part)

If you link back to your main site from your Facebook page (which, presumably, you do in multiple places), then standard parameter-based campaign tracking works. Use it. ‘nuf said.

General Facebook Activity — Web Analytics Measurement

In addition to tracking links that you control on Facebook with campaign tracking (the previous section), you can and should look at Facebook as a broader source of traffic to your site. If you are posting content on your site that is share-worthy, then Facebook users can pick it up and share it through Facebook, which will drive referrals to your site. If you’ve actually enabled content-sharing capabilities on your site, and those capabilities include Facebook, then you can add campaign tracking parameters to content as it gets shared, which will give you better visibility into what specific content is most compelling and passed along. Beyond just the traffic to the site, the bounce rate and conversions from that traffic are useful — is the sharing of your content bringing visitors to your site who are finding value and doing valuable things?

The caution here is to not get overly obsessed with Facebook as a source of traffic to your site. It certainly can (and probably should) be a source of traffic, but your site isn’t necessarily the best destination point for all of your customers. Just because this is easy data to get to doesn’t mean that it is the best data to use to measure the performance of your site.

General Facebook Activity — LOTS is Missing

Overall, Facebook measurement — measurement of what really matters — is still very immature. We’re largely stuck with measuring basic counts of things that are easy to measure: total fans, unique pageviews, etc. But, when it comes to both measuring the impact of a Facebook investment as well as being able to analyze what is and is not working, we’re missing a lot:

Impressions – how many people are actually being presented with content related to your brand? Besides Facebook display ads, this is total guesswork; just because a page posts a status update doesn’t mean it ever shows up on the screen of a fan (the update may slip well into the “More” area before the fan logs on again, the fan may have those updates hidden); “impressions” is far from being an end-all/be-all measure, but it’s a pretty good indicator of reach, and it’s really not available in the Facebook world
[UPDATE: Since I originally wrote this post, I've found out that Facebook has something in the works for this -- the one referenceable source is Facebook Presentation Reveals "Post Analytics" And Real-Time Ad Targeting. It's a total crapshoot as to when this functionality will be available and to whom it will be available.][UPDATE No 2: This capability was formally rolled out on January 21, 2010. I posted my take on what that provides.]
Social Graph and Impact — all Facebook users (and, thus, all Facebook page fans) are not equal; all of the major online listening platforms attempt to measure the influence of the “speaker,” and, conceptually, this construct applies in the Facebook world, driven by various aspects of the user: how many friends they have, how often they update their status, and, most importantly, how often the content they share gets liked/commented on/re-shared; it is currently not possible to get any visibility into and segment users who are interacting with your brand on Facebook based on their influence in the medium
Sentiment – Facebook has the “Like” feature, but no comparable “Dislike” option; this is grade school manners enforcement: “If you can’t give it a thumbs-up, don’t give it any thumb at all…” From a brand perspective, though, it would be nice to be able to track what sorts of posts raise users’ ire (especially for user-generated content) without having to sift through individual posts and comments by hand, which leads me to…
Sentiment…continued — sentiment is a tough nut to crack, but it’s something that everyone who deals with social media recognizes as being important; while I don’t necessarily expect Facebook to develop sentiment measurement tools inherently, if Facebook Insights was enhanced to enable the export of all user interactions for a fan page, then third-party tools could be used to conduct a sentiment analysis, and that would be useful
[UPDATE: While it's not necessarily a business/analyst-friendly option, the Facebook API does allow the retrieval of comments and posts. If you have the chops to tackle it, you can read about the options at http://wiki.developers.facebook.com/index.php/API#Data_Retrieval_Methods. One company that is using the API for that purpose (among others) is Vitrue -- comments and posts get pulled into their Vitrue SRM product in a pretty slick way.]
Online Listening…to Facebook — Google announced late last year that they were going to start crawling publicly available content in Facebook, and, presumably, online listening platforms will not be far behind (maybe some of them already do?). But, this listening is inherently limited to public content in Facebook (fan pages are public, so they would be included, presumably, which is a good thing). There would be a major backlash if Facebook enabled third-party tools to crawl and index “private” content. Does that mean that Facebook should enable it’s own intra-Facebook online listening capability? Marketers would certainly love to have the information, even if it is only available in a way that maintains users’ anonymity, but any move in this direction would be a dicey proposition for Facebook (even if they hid user information, it would be conceivable that users would provide enough information in what they post that a company would still be able to identify a specific individual — even if that was only going to be possible 1 time in 100,000, privacy advocates would jump all over Facebook for allowing the theoretical possibility)

It will be interesting to see where Facebook goes over the next 1-2 years when it comes to empowering marketers to measure and analyze their Facebook-based tactics. It should be a fun ride.

What am I missing here?Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2010. | Permalink | 12 comments | Add to del.icio.us
Post tags: Facebook, sentiment, Web Analytics

Feed enhanced by Better Feed from Ozh

The Spectrum of Data Sources for Marketers Is Wide (and Overwhelming)

Tim Wilson — Mon, 14 Dec 2009 14:00:01 +0000

I’ve been using an anecdote of late that Malcolm Gladwell supposedly related at a SAS user conference earlier this year: over the last 30 years, the challenge we face when it comes to using data to drive actions has fundamentally shifted from a challenge of “getting the right data” to “looking at an overwhelming array of data in the right way.” To illustrate, he compared Watergate to Enron — in the former case, the challenge for Woodward and Bernstein was uncovering a relatively small bit of information that, once revealed, led to immediate insight and swift action. In the latter case, the data to show that Enron had built a house of cards was publicly available, but there was so much data that actually figuring out how to extract the underlying chicanery without knowing exactly where to look for it was next to impossible.

With that in mind, I started thinking about all of the sources of data that marketers now have available to them to drive their decisions. The challenge is that almost all of the data sources out there are good tools — while they all claim competitive advantage and differentiation from other options…I believe in the free markets to the extent that truly bad tools don’t survive (do a Google search for “SPSS Netgenesis” and the first link returned is a 404 page — the prosecution rests!). To avoid getting caught up in the shiny baubles of any given tool, it seems worth organizing the range of available data some way — put every source into a discrete bucket. It turns out that that’s a pretty tricky thing to do, but one approach would be to put each data source available to us somewhere on a broad spectrum. At one end of the spectrum is data from secondary research — data that someone else has gone out and gathered about an industry, a set of consumers, a trend, or something else. At the other end of the spectrum is the data we collect on our customers in the course of conducting some sort of transaction with them — when someone buys a widget from our web site, we know their name, how they paid, what they bought, and when they bought it!

For poops and giggles, why not try to fill in that spectrum? Starting from the secondary research end, here we go…!

Secondary Research (and Journalism…even Journalism 2.0)

This category has an unlistable number of examples. From analyst firms like Forrester Research and Gartner Group, to trade associations like the AMA or The ARF, to straight-up journalists and trade publications, and even to bloggers. Specialty news aggregators like alltop.com fall into this category as well (even if, technically, they would fit better into a “tertiary research” category, I’m going to just leave them here!).

I stumbled across iconoculture last week as one interesting company that falls in this category…although things immediately start to get a little messy, because they’ve got some level of primary research as well as some tracking/listening aspects of their offer.

Listening/Collecting

Moving along our spectrum of data sources, we get to an area that is positively exploding. These are tools that are almost always built on top of a robust database, because what they do is try to gather and organize what people — consumers — are doing/saying online. As a data source, these are still inherently “secondary” — they’re “what’s happening” and “what’s out there.” But, as our world becomes increasingly digital, this is a powerful source of information.

One group of tools here are sites like compete.com, Alexa, and even Google’s various “insights” tools: Google Trends, Google Trends for Websites, and Google Insights for Search. These tools tend to not be so much consumer-focussed as site-focussed, but they’re getting their data by collecting what consumers are doing. And they are darn handy.

“Online listening platforms” are a newer beast, and there seems to be a new player in the space every day. The Forrester Wave report by Suresh Vittal in Q1 2009 seems like it is at least five years old. An incomplete list of companies/tools offering such platforms includes (in no particular order…except Nielsen is first because they’re the source of the registration-free PDF of the Forrester Wave report I just mentioned):

And the list goes on and on and on… (see Marshall Sponder’s post: 26 Tools for Social Media Monitoring). Each of these tools differentiates itself from their competition in some way, but none of them have truly emerged as a sustained frontrunner.

Web Analytics

I put web analytics next on the spectrum, but recognize that these tools have an internal spectrum all their own. From the “listening/collecting” side of the spectrum, web analytics tools simply “watch” activity on your web site — how many people went where and what they did when they got there. Moving towards the “1:1 transactions” end of the spectrum, web analytics tools collect data on specifically identifiable visitors to your site and provide that user-level specificity for analysis and action.

Google Analytics pretty much resides at the “watching” end of this list, as does Yahoo! Web Analytics (formerly IndexTools). But, then again, they’re free, and there’s a lot of power in effectively watching activity on your site, so that’s not a knock against them. The other major players — Omniture Sitecatalyst, Webtrends, Coremetrics, and the like — have more robust capabilities and can cover the full range of this mini-spectrum. They all are becoming increasingly open and more able to be integrated with other systems, be that with back-end CRM or marketing automation systems, or be that with the listening/collecting tools described in the prior section.

The list above covered “traditional web analytics,” but that field is expanding. A/B and multivariate testing tools fall into this category, as they “watch” with a very specific set of options for optimizing a specific aspect of the site. Optimost, Omniture Test&Target, and Google Website Optimizer all fall into this subcategory.

And, entire companies have popped up to fill specific niches with which traditional web analytics tools have struggled. My favorite example there is Clearsaleing, which uses technology very similar to all of the web analytics tools to capture data, but whose tools are built specifically to provide a meaningful view into campaign performance across multiple touchpoints and multiple channels. The niche their tool fills is improved “attribution management” — there’s even been a Forrester Wave devoted entirely to tools that try to do that (registration required to download the report from Clearsaleing’s site).

Primary Research

At this point on the spectrum, we’re talking about tools and techniques for collecting very specific data from consumers — going in with a set of questions that you are trying to get answered. Focus groups, phone surveys, and usability testing all fall in this area, as well as a plethora of online survey tools. Specifically, there are online survey tools designed to work with your web site — Foresee Results and iPerceptions 4Q are two that are solid for different reasons, but the list of tools in that space outnumbers even the list of online listening platforms.

The challenge with primary research is that you have to make the user aware that you are collecting information for the purpose of research and analysis. That drops a fly in the data ointment, because it is very easy to bias that data by not constructing the questions and the environment correctly. Even with a poorly designed survey, you will collect some powerful data — the problem is that the data may be misleading!

Transaction Data

Beyond even primary research is the terminus of the spectrum — it’s customer data that you collect every day as a byproduct of running your business and interacting with customers. Whenever a customer interacts with your call center or makes a purchase on your web site, they are generating data as an artifact. When you send an e-mail to your database, you’ve generated data as to whom you sent the message…and many e-mail tools also track who opened and clicked through on the e-mail. This data can be very useful, but, to be useful, it needs to be captured, cleansed, and stored in a way that sets it up for useful analysis. There’s an entire industry built around customer data management, and most of what the tools and processes in that industry focus on is transaction data.

What’s Missing?

As much as I would like to wrap up this post by congratulating myself on providing an all-encompassing framework…I can’t. While there are a lot of specific tools/niches that I haven’t listed here that I could fit somewhere on the spectrum of tools as I’ve described it, there are also sources of valuable data that don’t fit in this framework. One type that jumps out to me is marketing mix-type data and tools (think Analytic Partners, ThinkVine, or MarketShare Partners). I’m sure there are many other types. Nevertheless, it seems like a worthwhile framework to have when it comes to building up a portfolio of data sources. Are you getting data from across the entire spectrum (there are free or near-free tools at every point on the spectrum)? Are you getting redundant data?

What do you think? Is it possible to organize “all data sources for marketers” in a meaningful way? Is there value in doing so?Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2009. | Permalink | One comment | Add to del.icio.us
Post tags: data sources, Malcolm Gladwell

Feed enhanced by Better Feed from Ozh

The Most Meaningful Insights Will Not Come from Web Analytics Alone

Tim Wilson — Mon, 14 Sep 2009 14:20:28 +0000

Judah Phillips wrote a post last week laying out why the answer to the question, “Is web analytics hard or easy?” is a resounding “it depends.” It depends, he wrote, on what tools are being used, on how the site being analyzed is built, on the company’s requirements/expectations for analytics, on the skillset of the team doing the analytics, and, finally, on the robustness of the data management processes in place.

One of the comments on the blog came from John Grono of GAP Research, who, while agreeing with the post, pointed out:

You refer to this as “web analytics”. I also know that this is what the common parlance is, but truth be known it is actually “website analytics”. “web” is a truncation of “world wide web” which is the aggregation of billions of websites. These tools do not analyse the “web”, but merely individual nominated “websites” that collectively make up the “web”. I know this is semantics … but we as an industry should get it right.

It’s a valid point. Traditionally, “web analytics” has referred to the analysis of activity that occurs on a company’s web site, rather than on the web as a whole. Increasingly, though, companies are realizing that this is an unduly narrow view:

Search engine marketers (SEO and SEM) have, for years, used various keyword research tools to try to determine what words their target customers are using explicitly off-site in a search engine (although the goal of this research has been to use that information to bring these potential customers onto the company’s site)
Integration with a company’s CRM and/or marketing automation system — to combine information about a customer’s on-site activity with information about their offline interactions with the company — has been kicked around as a must-do for several years; the major web analytics vendors have made substantial headway in this area over the past few years
Of late, analysts and vendors have started looking into the impact of social media and how actions that customers and prospects take online, but not on the company’s web site, play a role in the buying process and generate analyzable data in the process

The “traditional” web analytics vendors (Omniture, Webtrends, and the like) were, I think, a little late realizing that social media monitoring and measurement was going to turn into a big deal. To their credit, they were just getting to the point where their platforms were opening up enough that CRM and data warehouse integration was practical. I don’t have inside information, but my speculation is that they viewed social media monitoring more as an extension of traditional marketing and media research companies that as an adjacency to their core business that they should consider exploring themselves. In some sense, they were right, as Nielsen, J.D. Power and Associates (through acquisition), Dow Jones, and TNS Media Group all rolled out social media monitoring platforms or services fairly early on. But, the door was also opened for a number of upstarts: Biz360, Radian6, Alterian/Techrigy/SM2, Crimson Hexagon, and others whom I’m sure I’ve left off this quick list. The traditional web analytics vendors have since come to the party through partnerships — leveraging the same integration APIs and capabilities that they developed to integrate with their customers’ internal systems to integrate with these so-called listening platforms.

Somewhat fortuitously, a minor hashtag snafu hit Twitter in late July when #wa, which had settled in as the hashtag of choice for web analytics tweets was overrun by a spate of tweets about Washington state. Eric Peterson started a thread to kick around alternatives, and the community settled on #measure, which Eric documented on his blog. I like the change for two reasons (notwithstanding those five precious characters that were lost in the process):

As Eric pointed out, measurement is the foundation of analysis — I agree!
“Web analytics,” which really means “website analytics,” is too narrow for what analysts need to be doing

I had a brief chat with a co-worker on the subject last week, and he told me that he has increasingly been thinking of his work as “digital analytics” rather than “web analytics,” which I liked as well.

It occurred to me that we’re really now facing two fundamental dimensions when it comes to where our customers (and potential customers) are interacting with our brand:

Online or offline — our website, our competitors’ websites, Facebook, blogs, and Twitter are all examples of where relevant digital (online) activities occur, while phone calls, tradeshows, user conferences, and peer discussions are all examples of analog (offline) activities
On-site or off-site — this is a bit of a misnomer, but I haven’t figured out the right words yet. But, it really means that customers can interact with the company directly, or, they can have interactions with the company’s brand through non-company channels

Pictorially, it looks something like this:

I’ve filled in the boxes with broad descriptions of what sort of tools/systems actually collect the data from interactions that happen in each space. My claim is that any analyst who is expecting to deliver meaningful insight for his company needs to understand all four of these quadrants and know how to detect relevant signals that are occuring in them.

What do you think?Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2009. | Permalink | 4 comments | Add to del.icio.us
Post tags: Crimson Hexagon, CRM, Cymfony, digital analytics, GAP Research, J.D. Power, John Grono, Nielsen, Omniture, SEM, SEO, SM2, Techrigy, WebTrends

Feed enhanced by Better Feed from Ozh

Where BI Is Heading (Must Head) to Stay Relevant

Tim Wilson — Tue, 07 Jul 2009 13:00:40 +0000

I stumbled across a post by Don Campbell (CTO of BI and Performance Management at IBM — he was at Cognos when they got acquired) today that really got my gears turning. His 10 Red Hot BI Trends provide a lot of food for thought for a single post (for one thing, the post only lists eight trends…huh?). It’s worth clicking over to the post for a read, as I’m not going to repeat the content here.

BUT…I can’t help but add in my own ~~drool~~ thoughts on some of his ideas:

Green Computing — not much to add here; this is more about next generation mainframes that run on a less power than the processors of yesteryear
Social Networking — it stands to reason that Web 2.0 has a place in BI, and Campbell starts to explain the wherefore and the why. One gap I’ve never seen a BI tool fill effectively is the ability to embed ad hoc comments and explanations within a report. That’s one of the reasons that Excel sticks around — because an Excel based report has to be “produced” in some fashion, there is an opportunity to review, analyze, and provide an assessment within the report. Enterprise BI tools have a much harder time enabling this — when it’s come up with BI tool vendors, it tends to get treated more as a data problem than a tool problem. In other words, “Sure, if you’ve got data about the reports stored somewhere, you can use our tool to display it.” What Campbell starts to touch on in his post is the potential for incorporating social bookmarking (“this view of this data is interesting and here is why”) and commenting/collaboration to truly start blending BI with knowledge management. The challenge is going to be that reports are becoming increasingly dynamic, and users are getting greater control over what they see and how. With roles-based data access, the data that users see on the same report varies from user to user. That’s going to make it challenging to manage “social” collaboration. Challenging…but something that I hope the enterprise BI vendors are trying to overcome.
Data Visualization — I wouldn’t have a category on this blog dedicated to data visualization if I didn’t think this was important. I can’t help but wonder if Campbell is realizing that Cognos was as guilty as the other major BI players of confusing “demo-y neat” with “effective” when it comes to past BI tool feature development. From his post: “The best visualizations do not necessarily involve the most complex graphics or charts, but rather the best representation of the data.” Amen, brother!!! Effective data visualization is finally starting to get some traction — or, at least, a growing list of vocal advocates (side note: Jon Peltier has started up a Chart Busters category on his blog — worth checking out). What I would like to see: BI vendors taking more responsibility for helping their users present data effectively. Maybe a wizard in report builders that ask questions about the type of data being presented? Maybe a blinking red popup warning (preferably with loud sirens) whenever someone selects the 3D effect for a chart? The challenge with data visualization is that soooooo many analysts: 1) are not inherently wired for effective visualization, and 2) wildly underestimate how important it is.
Mobile — I attended a session on mobile BI almost five years ago at a TDWI conference…and I still don’t see this as being a particularly hot topic. Even Campbell, with his mention of RFIDs, seems to think this is as much about new data sources as it is about reporting and analysis in a handheld environment.
Predictive Analytics — this has been the Holy Grail of BI for years. I don’t have enough exposure to enough companies who have successfully operationalized predictive analytics to speak with too much authority here. But, I’d bet good money that every company that is successful in this area has long since mastered the fundamentals of performance measurement. In other words, predictive analytics is the future, but too many businesses are thinking they can run (predictive analytics) before they crawl (performance measurement / KPIs / effective scorecards).
Composite Applications — this seems like a fancy way to say “user-controlled portals.” This really ties into the social networking (or at least Web 2.0), I think, in that a user’s ability to build a custom home page with “widgets” from different data sources that focus on what he/she truly views as important. Taking this a step farther — measuring the usage of those widgets — which ones are turned on, as well as which ones are drilled into — seems like a good way to assess whether what the corporate party line says is important is what line management is really using. There are some intriguing possibilities there as an extension of the “reports on the usage of reports” that gets bandied about any time a company starts coming to terms with report explosion in their BI (or web analytics) environment.
Cloud Computing – I actually had to go and look up the definition of cloud computing a couple of weeks ago after asking a co-worker who used the term if cloud computing and SaaS were the same thing (answer: SaaS is a subset of cloud computing…but probably the most dominant form). This is a must-have for the future of BI — as our lives become increasingly computerized, the days of a locally installed BI client are numbered. I regularly float between three different computers and two Blackberries…and lose patience when what I need to do is tied to only one machine.
Multitouch — think of the zoom in / zoom out capabilities of an iPhone. This, like mobile computing, doesn’t seem so much “hot” to me as somewhat futuristic. The best example of multitouch data exploration that I can think of is John King’s widely-mocked electoral maps on CNN (never did I miss Tim Russert and his handheld whiteboard more than when watching King on election night!). I get the theoretical possibilities…but we’ve got a long ways to go before there is truly a practical application of multitouch.

As I started with, there are a lot of exciting possibilities to consider here. I hope all of these topics are considered “hot” by BI vendors and BI practicitioners — making headway on just a few of them would get us off the plateau we’ve been on for the past few years.Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2009. | Permalink | One comment | Add to del.icio.us
Post tags: CNN, Cognos, Don Campbell, IBM, trends

Feed enhanced by Better Feed from Ozh

What is “Analysis?”

Tim Wilson — Tue, 05 May 2009 15:00:53 +0000

Stephen Few had a recent post, Can Computers Analyze Data?, that started: “Since ‘business analytics’ has come into vogue, like all newly popular technologies, everyone is talking about it but few are defining what it is.” Few’s post was largely a riff off of an article by Merv Adrian on the BeyeNETWORK: Today’s ‘Analytic Applications’ — Misnamed and Mistargeted. Few takes issue (rightly so), with Adrian’s implied definition of the terms “analysis” and “analytics.” Adrian outlines some fair criticisms of BI tool vendors, but Few’s beef regarding his definitions are justified.

Few defines data analysis as “what we do to make sense of data.” I actually think that is a bit too broad, but I agree with him that analysis, by definition, requires human beings.

With data “coming into vogue,” it’s hard to walk through a Marketing department without hearing references to “data mining” and “analytics.” Given the marketing departments I tend to walk through, and given what I know of their overall data maturity, this is often analogous to someone filling the ice cube trays in their freezer with water and speaking about it in terms of the third law of thermodynamics.

I’ve got a 3-year-old daughter, and it’s through her that I’ve discovered the Fancy Nancy series of books, in which the main character likes to be elegant and sophisticated well beyond her single-digit age. She regularly uses a word and then qualifies it as “that’s a fancy way to say…” a simpler word. For instance, she notes that “perplexed” is a fancy word for “mixed up.”

“Analytics” is a Fancy Nancy word. “Web analytics” is a wild misnomer. Most web analysts will tell you there’s a lot of work to do with just basic web site measurement. And, that work is seldom what I would consider “analytics.” As cliché as it is, you can think about data usage as a pyramid, with metrics forming the foundation and analysis (and analytics) being built on top of them.

There are two main types of data usage:

Metrics / Reporting – this is the foundation of using data effectively; it’s the way you assess whether you are meeting your objectives and achieving meaningful outcomes. Key Performance Indicators (KPIs) live squarely in the world of metrics (KPIs are a fancy way to say “meaningful metrics”). Avinash Kaushik defines KPIs brilliantly: “Measures that help you understand how you are doing against your objectives.” Metrics are backward-looking. They answer the question: “Did I achieve what I set out to do?” They are assessed against targets that were set long before the latest report was pulled. Without metrics, analysis is meaningless.
Analysis — analysis is all about hypothesis testing. The key with analysis is that you must have a clear objective, you must have clearly articulated hypotheses, and, unless you are simply looking to throw time and money away, you must validate that the analysis will lead to different future actions based on different possible outcomes. Analysis tends to be backward looking as well — asking questions, “Why did that happen?”…but with the expectation that, once you understand why something happened, you will take different future actions using the knowledge.

So, what about “analytics?” I asked that question of the manager of a very successful business intelligence department some years back. Her take has always resonated with me: “analytics” are forward-looking and are explicitly intended to be predictive. So, in my pyramid view, analytics is at the top of the structure — it’s “advanced analysis,” in many ways. While analysis may be performed by anyone with a spreadsheet, and hypotheses can be tested using basic charts and graphs, analytics gets into a more rigorous statistical world: more complex analysis that requires more sophisticated techniques, often using larger data sets and looking for results that are much more subtle. AND, using those results, in many cases, to build a predictive model that is truly forward-looking.

The key is that the foundation of your business (whether it’s the entire company, or just your department, or even just your own individual role) is your vision. From your vision comes your strategy. From your strategy come your objectives and your tactics. If you’re looking to use data, the best place to start is with those objectives — how can you measure whether you are meeting them, and, with the measures you settle on, what is the threshold whereby you would consider that you achieved your objective? Attempting to do any analysis (much less analytics!) before really nailing down a solid foundation of objectives-oriented metrics is like trying to build a pyramid from the top down. It won’t work.Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2009. | Permalink | One comment | Add to del.icio.us
Post tags: Analysis, analytics, Avinash Kaushik, Fancy Nancy, Metrics, objectives, predictive modeling, pyramid

Feed enhanced by Better Feed from Ozh

The Best Little Book on Data

Tim Wilson — Fri, 06 Mar 2009 02:58:30 +0000

How’s that for a book title? Would it pique your interest? Would you download it and read it? Do you have friends or co-workers who would be interested in it?

Why am I asking?

Because it doesn’t exist. Yet. Call it a working title for a project I’ve been kicking around in my head for a couple of years. In a lot of ways, this blog has been and continues to be a way for me to jot down and try out ideas to include in the book. This is my first stab at trying to capture a real structure, though.

The Best Little Book on Data

In my mind, the book will be a quick, easy read — as entertaining as a greased pig loose at a black-tie political fundraiser — but will really hammer home some key concepts around how to use data effectively. If I’m lucky, I’ll talk a cartoonist into some pen-and-ink, one-panel chucklers to sprinkle throughout it. I’ll come up with some sort of theme that will tie the chapter titles together — “myths” would be good…except that means every title is basically a negative of the subject; “Commandments” could work…but I’m too inherently politically correct to really be comfortable with biblical overtones; an “…In which our hero…” style (the “hero” being the reader, I guess?). Obviously, I need to work that out.

First cut at the structure:

Introduction — who this book is for; in a nutshell, it’s targeted at anyone in business who knows they have a lot of data, who knows they need to be using that data…but who wants some practical tips and concepts as to how to actually go about doing just that.
Chapter 1: Start with the Data…If You Want to Guarantee Failure – it’s tempting to think that, to use data effectively, the first thing you should do is go out and query/pull the data that you’re interested in. That’s a great way to get lost in spreadsheets and emerge hours (or days!) later with some charts that are, at best, interesting but not actionable, and, at worst, not even interesting.
Chapter 2: Metrics vs. Analysis — providing some real clarity regarding the fundamentally different ways to “use data.” Metrics are for performance measurement and monitoring — they are all about the “what” and are tied to objectives and targets. Analysis is all about the “why” — it’s exploratory and needs to be hypothesis driven. Operational data is a third way, but not really covered in the book, so probably described here just to complete the framework.
Chapter 3: Objective Clarity — a deeper dive into setting up metrics/performance measurement, and how to start with being clear as to the objectives for what’s being measured, going from there to identifying metrics (direct measures combined with proxy measures), establishing targets for the metrics (and why, “I can’t set one until I’ve tracked it for a while” is a total copout), and validating the framework
Chapter 4: When “The Metric Went Up” Doesn’t Mean a Gosh Darn Thing – another chapter on metrics/performance measuremen. A discussion of the temptation to over-interpret time-based performance metrics. If a key metric is higher this month than last month…it doesn’t necessarily mean things are improving. This includes a high-level discussion of “signal vs. noise,” an illustration of how easy it is to get lulled into believing something is “good” or “bad” when it’s really “inconclusive,” and some techniques for avoiding this pitfall (such as using simple, rudimentary control limits to frame trend data).
Chapter 5: Remember the Scientific Method? — a deeper dive on analysis and how it needs to be hypothesis-driven…but with the twist that you should validate that the results will be actionable just by assessing the hypothesis before actually pulling data and conducting the analysis
Chapter 6: Data Visualization Matters – largely, a summary/highlights of the stellar work that Stephen Few has done (and, since he built on Tufte’s work, I’m sure there would be some level of homage to him as well). This will include a discussion of how graphic designers tend to not be wired to think about data and analysis, while highly data-oriented people tend to fall short when it comes to visual talent. Yet…to really deliver useful information, these have to come together. And, of course, illustrative before/after examples.
Chapter 7: Microsoft Excel…and Why BI Vendors Hate It – the BI industry has tried to equate MS Excel with “spreadmarts” and, by extension, deride any company that is relying heavily on Excel for reporting and/or analysis as being wildly early on the maturity curve when it comes to using data. This chapter will blow some holes in that…while also providing guidance on when/where/how BI tools are needed (I don’t know where data warehousing will fit in — this chapter, a new chapter, or not at all). This chapter would also reference some freely downloadable spreadsheets with examples, macros, and instructions for customizing an Excel implementation to do some of the data visualization work that Excel can do…but doesn’t default to. Hmmm… JT? Miriam? I’m seeing myself snooping for some help from the experts on these!
Chapter 8: Your Data is Dirty. Get Over It. – CRM data, ERP data, web analytics data, it doesn’t matter what kind of data. It’s always dirtier than the people who haven’t really drilled down into it assume. It’s really easy to get hung up on this when you start digging into it…and that’s a good way to waste a lot of effort. Which isn’t to say that some understanding of data gaps and shortcomings isn’t important.
Chapter 9: Web Analytics – I’m not sure exactly where this fits, but it feels like it would be a mistake to not provide at least a basic overview of web analytics, pitfalls (which really go to not applying the core concepts already covered, but web analytics tools make it easy to forget them), and maybe even providing some thoughts on social media measurement.
Chapter 10: A Collection of Data Cliches and Myths — This may actually be more of an appendix, but it’s worth sharing the cliches that are wrong and myths that are worth filing away, I think: “the myth of the step function” (unrealistic expectations), “the myth that people are cows” (might put this in the web analytics section), “if you can’t measure it, don’t do it” (and why that’s just plain silliness)
Chapter 11: Bringing It All Together — I assume there will be such a chapter, but I’m going to have to rely on nailing the theme and the overall structure before I know how it will shake out.

What do you think? What’s missing? Which of these remind you of anecdotes in your own experience (haven’t you always dreamed of being included in the Acknowledgments section of a book? Even if it’s a free eBook?)? What topic(s) are you most interested in? Back to the questions I opened this post with — would you be interested in reading this book, and do you have friends or co-workers who would be interested? Or, am I just imagining that this would fill a gap that many businesses are struggling with?Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2009. | Permalink | 7 comments | Add to del.icio.us
Post tags: book, cliches, Edmund Tufte, myths, Stephen Few

Feed enhanced by Better Feed from Ozh