WSJ: How Allstate and Dstillery Avoid Fraudulent Ads on Exchanges

Mike Shields wrote a nice piece in the Wall Street Journal about how Dstillery helps protect Allstate from buying fraudulent ad impressions in the open exchanges.

I spoke to Mike about this article and was quoted a couple of times including:

“The thing about fraud, it’s all still the same,” he said. “What we identified a few years ago, it’s still out there. The best you can do is stay a half step behind the people who perpetrate it. You need to be constantly vigilant on it.”

I’m really proud of the work Dstillery does to protect clients from ad fraud and happy to see it recognized.

Product Managers and Data Scientists

Over the past couple of decades, product management has emerged as an important function within any organization that is building technology products. Data science has arrived more recently but is also proving its value in helping to solve problems for customers. In data-centric organizations, there is an important symbiotic relationship between the two functions, but it’s not always clear where data science should fit in the product development cycle. Should data scientists be treated like developers? Analysts? Or something entirely new?

While both groups want to solve problems for customers, it is important to understand the differences in their fundamental mindsets. Data science is about exploring data, understanding its predictive possibilities and creating tools that use data to optimize defined performance metrics. Product management is about deciding if the things that are technically possible are actually worth doing from a business standpoint and if so, how those new capabilities should be delivered to customers as products or features.

Collaboration between the two groups is vital, but works best when each team has a clearly defined role and set of duties. Here are some approaches on how to successfully integrate data science into the product development process:

Data Science is the R in R&D – Data scientists need the latitude to explore problems that they think are interesting and come up with innovative approaches. Celebrate this. Not everything they do will address the problems you are currently trying to solve, but it is important to allow them the freedom to explore possibilities. Often, they will surprise you with a solution to another problem that will help shape your business in a positive way.

Data Scientists Build Great Prototypes – Projects work best when the data science team develops working prototypes using whatever tools they want. These prototypes demonstrate the requirements of a new system to software developers and others. Only once there is a working prototype that has been green lit by all the necessary parties should you move forward with building out the new capability in your production systems.

Lean Data Science is Possible – Once data science has developed a working prototype, the team should figure out the smallest increment that can be put into production to test it in the real world; a Minimally Viable Algorithm. This could be as simple as developing a workable process that provides alerts when certain conditions occur. This approach helps identify errors in assumptions and bugs in implementation before a full release that builds on the ultimate algorithm.

Data Scientists Do Great QA – I’m not referring to regression testing of code. If you are building a data product or an ETL process, a qualified expert needs to validate the data at some point. Data scientists are obviously the best candidates to conduct that oversight, so make their signoff part of the formal criteria for prototype acceptance. And after the development teams have built production versions of the prototype, don’t let the data scientists wander off to get buried in the next great paper-worthy research project. Get them involved in validating that the production features are indeed working as intended.

While a lot has been written about product management, software development and how product development processes should be designed, processes that incorporate the data science function are less well understood. Data science is often essential to the development of products that can compete in a data-intensive market. Developers and product managers are best served by finding ways to integrate data science into their projects from start to finish; from design to QA and through to market feedback. Data science is now an integral part of almost every technology ecosystem, and product managers everywhere should embrace their full involvement.

As published by iMedia Connection on 4/4/2014.

PRODUCT Manager v. PROJECT Manager

I was recently asked to highlight the key differences between Product Management and Project management. While the two functions are clearly seen as different by the practitioners, the lines of responsibility can get blurred for other groups.

In reading what others had to say about the issue, I came across a post by Jeff Lash that almost perfectly captures my view on the issue:

Project managers are responsible for the successful delivery of a project — a one-time endeavor with a goal, scope, deadline, budget, and other constraints. A project manager will work to align resources, manage issues and risks, and basically coordinate all of the various elements necessary to complete the project. As they relate to products, projects can be undertaken to build a product, to add new features to a product, or create new versions or extensions of a product. When the project is complete, the project manager will usually move move to a new project, which may be related to a different product.

Product managers are responsible for the overall and ongoing success of a product. Once the project to build the product is complete and the project manager has moved on, the product manager remains to manage the product through the entire lifecycle. Other projects related to the product may be initiated, with the product manager being the one constant stream throughout, defining the project goals and guiding the team to accomplish the business objectives that have been defined.

I only thing I would change here is to amplify the responsibility of the Product Manager as the voice of the customer in the development process. In owing the product lifecycle, the Product Manager must make sure the projects that are prioritized are the ones that best solve the customer’s biggest problems. This is the most important part of the Product Manager’s job.

Roles and Responsibilities in Pricing

In two jobs prior to Dstillery, among other things, I had responsibility for managing product pricing. In that role, I’ve priced hardware, software, SaaS services, professional services, support services, and labor. At some point, I had jotted down a bunch of notes on how the pricing process should work and I thought I would share them here.

It has been said that pricing is not an event, it is a process. And it is not just a process that should be owned by finance, sales or marketing but one that needs to be truly cross-functional.

This diagram outlines the key activities that go into setting, managing, and measuring prices. The various activities can be, and often are, owned by different functions depending how each organization is structured, but there needs to be someone looking across all of these activities in a holistic manner. The key role of the “pricing owner” is to coordinate across all the interested functions and drive the process to establish the best pricing model for the company.


Continue reading

Quote: Belief vs. Observation

“Do not believe in anything simply because you have heard it…or because it is spoken and rumored by many…or because it is found written in your religious books. But after observation and analysis, when you find that anything agrees with reason and is conducive to the good and benefit of one and all, then accept it and live up to it.”

– The Buddha

Three Simple Rules for Great Companies

The amount of printed material available about managing and running great companies is pretty staggering.  And, like fad-diets, the philosophies espoused in this literature seem to be either overly simplistic or incredibly weird.  That’s why I found this article in the most recent issue of The Harvard Business Review pretty refreshing.

Two researchers looked at the historical performance of over 25k public companies in an attempt to identify a common set of strategies that separate great performers from mediocre ones.  They looked at the impact of innovation levels, risk taking behaviors, M&A activity, hiring practices, and so on.  In the end, none of those factors were determinant.  In fact, the researchers did not find any silver bullets or magic strategies.  What they did find, were three simple rules that are practices by all these great companies:

  1. Better before cheaper—in other words, compete on differentiators other than price.
  2. Revenue before cost—that is, prioritize increasing revenue over reducing costs.
  3. There are no other rules—so change anything you must to follow Rules 1 and 2.

It is obvious if you think about it-if you have a better product or service, more people will want to buy it.  As long as your price is in line with the value it creates, unit sales and revenues will be strong.  This point is often lost by the short sighted equity markets and financial press which seem to love all cost-cutting announcements.  It has said many times, “you can’t cut your way to greatness” and the research here seems to prove this point.

The authors call this discovery “liberating” because it gives managers the flexibility to follow any tactics or strategies that support the first two rules and offer advice:

Here’s how to put the rules into operation: The next time you find yourself having to allocate scarce resources among competing priorities, think about which initiatives will contribute most to enhancing the nonprice elements of your position and which will allow you to charge higher prices or to sell in greater volume. Then give those the nod.

This is, now quantifiably, a recipe for success.

Its Not Rocket Science

Back in January, Duncan Watts of Microsoft Research spoke at m6d’s ADSCON event at NYU.  Duncan has a unique view of the world having begun his career in the hard sciences (math and physics) and migrated to social sciences.  He quoted a passage from his book on the paradox of thinking rocket science requires brilliance but social science less so:

Typically people in these positions [public policy makers, marketers, economists] do not expect to get everything right all the time.  But they also feel that the problems they are contemplating are mostly within their ability to solve – that “it’s not rocket science,” as it were.  Well, I’m no rocket scientist, and I have immense respect for people who can land a machine the size of a small car on another planet.  But the sad fact is that we’re actually better at planning the flight path of an interplanetary rocket than we are at managing the economy, merging two corporations, or even predicting how many copies of a book will sell.  So why is that rocket science seems hard[?]

This is really interesting thought.  Rocket science is, clearly, very hard.  Rockets are complex systems but the physical laws governing the behavior of these systems are predictable and consistent.  People, and social interactions between them, are also made up of very complex systems but human behavior is anything but predictable and consistent.  Given this unpredictable nature, maybe it makes sense that studying human behavior in a scientific way is much harder than it is given credit for.

Product Management and Start Up Maturity Cycle

A few weeks back, I had the opportunity to attend Practical Product Management training from Pragmatic Marketing.  The instructor, John Gatrell, said something about the maturity cycle in early stage organizations as it relates to product management that I had not considered before.

Jon said that in the beginning most startups are engineering led (this part I knew).  Since the founder usually leads the product development team, this is likely the one time when a company is most market-driven; where they are most focused on building the products to solve real market problems.  As the organization grows and functional specialization begins, a professional sales team is hired and the development organization become more removed from the market and loses some of this focus.

With the new sales team in place, the company focuses on top-line growth and the organization tends to become more sales driven.  One down side of a sales driven organization is that the sales team can get overly aggressive and will sell features that the product doesn’t necessarily support yet.  More and more the product roadmap gets determined by the commitments sales people are making to clients in order to close deals.  This approach helps win business but is not a strategic or thoughtful approach on how to evolve a product for long term market success.

Eventually, the sales team asks for help telling the company’s story and the organization will strengthen its marketing capability.  The marketing team will naturally focus on outbound activities like PR, collateral, and lead generation that help build the sales funnel.  At this point the company is talking to the market but probably not listening as effectively as it could be.

When the company reaches this point, it needs to evolve back towards a market-driven, listening organization that is centered around solving real market problems.  The Pragmatic approach emphasizes, and I agree, that the key role of the Product Manager is force this market-centric discipline on the development organization.  The Product Manager needs to synthesize all the information he or she receives to make data-driven decisions about how to evolve the product and not simply respond to the latest “must-have” feature from a sales rep.

As a side note, I’m glad that Pragmatic has certified me after 7+ years of operating without a license to practice product management.  It is quite a relief to be legit.

From the m6d Blog: The Constant Gardener

Earlier this week I had the opportunity to post on the m6d blog about some exciting technology we’re developing to ensure the quality exchange inventory we buy.  I re-posted it below, but you can (and should) read the original here.


Ad exchanges are like a garden bed. Just as soil doesn’t know the difference between a flower and a weed, ad exchanges do not always distinguish between good and bad inventory. It takes a vigilant gardener to weed out any undesirable elements. Lately, Media6Degrees has been doing a lot of gardening.

We are, and intend to remain, the most aggressive company in the exchange ecosystem in rooting out the bad stuff so that we buy only the best impressions for our marketers. We’ve been told repeatedly by our supply partners that nobody takes inventory quality more seriously than we do.

Our approach manifests itself in four areas:

  1. Brand Safety
  2. Ad Collision (also known as Unintended Roadblocks) Prevention
  3. Fraud Protection
  4. Viewability

Continue reading