Animated sequence of a race horse galloping. P...

Image via Wikipedia

I have been thinking about this article from Eric Ries that was published on TechCrunch. He makes a really strong case for pushing scientific methods into product management  and in particular the use of A/B testing as a means to zoom in on whether a new feature being introduced is actually making the product better or worse according to the metrics of the business. Whilst I am really in line with bringing more robust and rational methods into the product management practice, I am less crazy about A/B testing.

I think there is enormous value in looking at a product with a scientific mind: formulating hypotheses about what may make the product better, making predictions about the product metrics based on these hypotheses, designing experiments to verify these hypotheses, evaluating the results and improving on the hypotheses. Product development becomes part of your experiments. This method can be applied to a range of hypothesis from what can make your existing customers even more satisfied with your product experience, to what new problems could the product solve for your customer, to what changes could be made to improve the product profitability… However the method of the experiment doesn’t have to be A/B or split testing. You have to pick the right format for the experiment to best verify your hypothesis. Some testing method will be better suited to specific types of hypotheses and predictions you may be making.

It would be interesting to continue the thought process and to design a scientific framework for Product Management including maybe a classification of different types of hypotheses and  best practice in designing and setting up experiments to verify them.

Qu’est-ce que tu en penses?

Enhanced by Zemanta