Experiments, theories, testing and measurement.
Getting a product built is a huge milestone, one any organization should be proud of. However, until a product is put in front of a customer, we're really just guessing about whether it will be successful. Don't get me wrong, "guessing" is an important part of product design; we call them "hypotheses" and they are highly important to crafting an experimentation plan. But, what are product experiments? Why should you be experimenting, testing & measuring? How do you turn hypotheses into actionable theories?
Turning hypotheses into theories.
Hypotheses and theories are often related; when a hypothesis is sufficiently validated it can often be stated as a theory. Theories are generally accepted explanations that can be used to inform and rationalize change. In product design, generating theories to explain why and how customers use products is an ongoing objective. Theories always begin with a hypothesis, or a guess:
-
Educated guessing: Everyone and their goat has opinions. Guessing is a skillset, like any other, and can be learned and honed. The best guessing requires a complete understanding of the problem; with the best guessers able to determine root-causes vs mere symptoms. Hostile Sheep has helped clients uncover (and frame) the most important problems in order to develop hypotheses. We've facilitated a "guessing game" workshops with internal stakeholders and customers to completely understand problems and generate educated guesses. Hypotheses are just a first step toward generating theories that can be acted upon with confidence.
-
Experiment design: product experimentation, or feature experimentation, within a jobs-to-be-done framework examine which products or features best help customers get the job done. The best experiments will surface where customers are delighted and where they're frustrated. Experiments should be designed to validate/invalidate hypotheses, so, unless there are hypotheses there is no need for an experiment. Hostile Sheep has helped clients clearly articulate how and when to experiment. We've walked clients through experimentation design, execution and analysis.
-
Measurement and analysis: is such an important aspect of experiment design that we've pulled it out as being its own discipline that we actively invest in. We believe a measurement plan is absolutely critical to running an effective experiment. Not only will it establish what needs to be measured, it will also establish what won't be measured; ensuring analysis can be done in near-real-time. With a Hostile Sheep measurement plan, our analysis of the findings is always clear, succinct, and actionable.
-
Theory development: Hostile Sheep has been developing and utilizing theories to guide product development for over a decade. Theories are sets of principles or concepts that explain why something occurs. In product design, we're primarily concerned with theories that explain why and how customers use products. While it may seem like a single, general theory would be capable of explaining why and how customers use a product, more specific (and helpful) theories tend to fall out of a general theory. Theories are key components for establishing an effective product management practice.
Using theories to guide product evolution.
Hostile Sheep believes using pattern design and design systems is the best way for current, modern products to evolve. The best design systems include a section that defines some set of design principles that govern the system. Hostile Sheep prefers to frame these design principles as product design theories. As such, much of the design system is tied to product design theories; which require special attention and management.
- Establishing a design system: Long before the rise of custom design systems, Hostile Sheep was an advocate of pattern design. In fact, Hostile Sheep was a pioneer in pattern design, establishing an initiative called "Understanding Patterns". Since then, Hostile Sheep has built foundational design systems for several organizations; most are still in use after years and multiple iterations. One of the best parts of using design systems is the ease of running experiments on a single component, without needing to touch the rest of the product. Thus, establishing a design system can supercharge product optimization through experimentation and testing.
- Finding WIND: Product design theories are living entities that will evolve as the organization learns about the product, market, competition, and other aspects that inform product requirements. Hostile Sheep developed the WIND framework (Widen, Improve, New, Deepen) that provides a level of taxonomy to product design theory experimentation. We want to design experiments that will challenge established product design theories in specific ways: (1) Can the theory be widened?, (2) Can the theory be improved?, (3) Can a new theory be defined?, (4) Can the theory be deepened? Hostile Sheep can help you find the WIND that will power the evolution of your products.
- Theory-feature mapping: Hostile Sheep has been exposed to so many different ways of creating and managing product roadmaps, we should really write a book. One method we've adopted is Concept Managed Design (CMD) which involves examining concepts proposed by each product design theory and exploring solutions that will produce the best return on investment. These solutions can be broken into features/feature-sets and added to a product roadmap through a standard prioritization exercise. Allowing features to be tied back to theories enables PM teams to quickly determine the impact of constantly-evolving product design theories. (i.e., now that we know more about a concept, what existing features need to be re-validated?)
Not exactly what you're looking for?
We move our clients through discovery and definition phases of product design projects. Hostile Sheep has tackled almost every aspect of product design from conducting primary research to uncover needs, to co-creating alpha/design rapid-prototypes that can be tested with real customers and can inform downstream development teams. Read more on our home page.
Here for the Moment.
Hostile Sheep has helped dozens of organizations develop fantastic digital products that their customers absolutely love. However, we're only around for a moment before we move on. Even the best products need to be supported and managed by a product-centric organization to stay great.
Hostile Sheep has developed a friendship with the renown innovation design firm The Moment. Together we've helped organizations understand and implement the conditions conducive of supercharging their product-led growth engine.