{"id":16415,"date":"2019-05-07T13:26:42","date_gmt":"2019-05-07T13:26:42","guid":{"rendered":"https:\/\/productcollective.com\/?p=16415"},"modified":"2019-06-17T02:51:31","modified_gmt":"2019-06-17T02:51:31","slug":"experiment-driven-product-development","status":"publish","type":"post","link":"https:\/\/productcollective.com\/experiment-driven-product-development\/","title":{"rendered":"Experiment-Driven Product Development"},"content":{"rendered":"

The following insights were gleaned from a presentation at INDUSTRY given by Jon Noronha of Optimizely @thatsjonsense.<\/a> Download Product Management TACTICS eBook<\/a> for more.<\/strong><\/p>\n

When it comes to product development, experimenting badly can be worse than not experimenting at all. As one of the most powerful tools available to product teams, when done right, experimentation allows teams to iterate gradually and validate ideas with data.<\/p>\n

We\u2019re all familiar with the process of building and launching products. The problem is that until we know how it went, we\u2019re essentially blind. There\u2019s a better approach:  Experiment and iterate from start to finish, repeatedly.<\/p>\n

Product experimentation involves learning, building, measuring, and repeating. Once this happens, we can start to quantify the impact that product changes has. Failure will happen, and that\u2019s OK — we should embrace it.  In fact, failing can help us find bigger and better successes as long as we\u2019re working to learn from those failures. But failure often happens from simple mistakes that can be avoided.  Some of those mistakes include:<\/p>\n

    \n
  1. Optimizing the wrong metrics. Most product people understand the importance of setting goals in place — but what happens if we\u2019re setting the wrong goal? Put yourself in the user\u2019s shoes. Ask, \u201cWhat if this went up and nothing else?\u201d Constantly re-evaluate and don\u2019t underestimate the value of trusting your gut at times.<\/li>\n
  2. Getting tricked by the stats. Don\u2019t peek at A\/B results too soon. There\u2019s a risk of false positives \u2014 more than you\u2019d normally see. If you\u2019re in doubt, get a data scientist or statistician, or use a platform that can help you dissect the results at the right time.<\/li>\n
  3. Thinking too small. Experimentation is often misunderstood as simply A\/B testing\u2026 with a literal \u201cA\u201d and a \u201cB.\u201d. A better system would be A-B-C-D-E-F testing because at that level, creativity increases. Testing five or more variations can improve your win rate by 75%.<\/li>\n<\/ol>\n

    And like products, your experimentation process should always be open to iteration and improvement.<\/p>\n

    To view Jon\u2019s full presentation, visit:<\/p>\n

    https:\/\/productcollective.com\/experiment-driven-product-development-weve-learned-1-million-experiments\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    The following insights were gleaned from a presentation at INDUSTRY given by Jon Noronha of Optimizely @thatsjonsense. Download Product Management TACTICS eBook for more. When it comes to product development, experimenting badly can be worse than not experimenting at all. As one of the most powerful tools available to product teams, when done right, experimentation allows teams […]<\/p>\n","protected":false},"author":2,"featured_media":16460,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[],"yoast_head":"\nExperiment-Driven Product Development - Product Collective | Organizers of INDUSTRY: The Product Conference<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/productcollective.com\/experiment-driven-product-development\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Experiment-Driven Product Development - Product Collective | Organizers of INDUSTRY: The Product Conference\" \/>\n<meta property=\"og:description\" content=\"The following insights were gleaned from a presentation at INDUSTRY given by Jon Noronha of Optimizely @thatsjonsense. Download Product Management TACTICS eBook for more. When it comes to product development, experimenting badly can be worse than not experimenting at all. As one of the most powerful tools available to product teams, when done right, experimentation allows teams […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/productcollective.com\/experiment-driven-product-development\/\" \/>\n<meta property=\"og:site_name\" content=\"Product Collective | Organizers of INDUSTRY: The Product Conference\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/productcollective\" \/>\n<meta property=\"article:published_time\" content=\"2019-05-07T13:26:42+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-06-17T02:51:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/productcollective.com\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-05-07-at-9.25.16-AM.png\" \/>\n\t<meta property=\"og:image:width\" content=\"966\" \/>\n\t<meta property=\"og:image:height\" content=\"534\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Paul McAvinchey\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Paul McAvinchey\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/productcollective.com\/experiment-driven-product-development\/\",\"url\":\"https:\/\/productcollective.com\/experiment-driven-product-development\/\",\"name\":\"Experiment-Driven Product Development - Product Collective | Organizers of INDUSTRY: The Product Conference\",\"isPartOf\":{\"@id\":\"https:\/\/productcollective.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/productcollective.com\/experiment-driven-product-development\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/productcollective.com\/experiment-driven-product-development\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/productcollective.com\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-05-07-at-9.25.16-AM.png\",\"datePublished\":\"2019-05-07T13:26:42+00:00\",\"dateModified\":\"2019-06-17T02:51:31+00:00\",\"author\":{\"@id\":\"https:\/\/productcollective.com\/#\/schema\/person\/f34ceda9eb6237e864633ca033061c0b\"},\"breadcrumb\":{\"@id\":\"https:\/\/productcollective.com\/experiment-driven-product-development\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/productcollective.com\/experiment-driven-product-development\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/productcollective.com\/experiment-driven-product-development\/#primaryimage\",\"url\":\"https:\/\/productcollective.com\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-05-07-at-9.25.16-AM.png\",\"contentUrl\":\"https:\/\/productcollective.com\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-05-07-at-9.25.16-AM.png\",\"width\":966,\"height\":534},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/productcollective.com\/experiment-driven-product-development\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/productcollective.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Experiment-Driven Product Development\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/productcollective.com\/#website\",\"url\":\"https:\/\/productcollective.com\/\",\"name\":\"Product Collective | Organizers of INDUSTRY: The Product Conference\",\"description\":\"For people who build, launch and scale world-class software products.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/productcollective.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/productcollective.com\/#\/schema\/person\/f34ceda9eb6237e864633ca033061c0b\",\"name\":\"Paul McAvinchey\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/productcollective.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/productcollective.com\/wp-content\/uploads\/2020\/12\/PLnHr3Hu_400x400-150x150.jpg\",\"contentUrl\":\"https:\/\/productcollective.com\/wp-content\/uploads\/2020\/12\/PLnHr3Hu_400x400-150x150.jpg\",\"caption\":\"Paul McAvinchey\"},\"description\":\"For over 20 years, Paul has been building and collaborating on digital products with fast-growing startups and global brands, including AOL and WMS Gaming. Currently, he's a co-founder of Product Collective, a worldwide community of product people. Members collaborate on in the exclusive Member Hub, meet at INDUSTRY: The Product Conference, listen to Rocketship.fm, learn at Product Interviews and get a weekly newsletters that includes best practices in product management. In recent years he led business development at DXY, a leading product design firm in the Midwest, and product innovation at MedCity Media, a publishing startup acquired by Breaking Media in 2015.\",\"sameAs\":[\"https:\/\/twitter.com\/https:\/\/twitter.com\/yaboya\"],\"url\":\"https:\/\/productcollective.com\/author\/newindsum\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Experiment-Driven Product Development - Product Collective | Organizers of INDUSTRY: The Product Conference","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/productcollective.com\/experiment-driven-product-development\/","og_locale":"en_US","og_type":"article","og_title":"Experiment-Driven Product Development - Product Collective | Organizers of INDUSTRY: The Product Conference","og_description":"The following insights were gleaned from a presentation at INDUSTRY given by Jon Noronha of Optimizely @thatsjonsense. Download Product Management TACTICS eBook for more. When it comes to product development, experimenting badly can be worse than not experimenting at all. As one of the most powerful tools available to product teams, when done right, experimentation allows teams […]","og_url":"https:\/\/productcollective.com\/experiment-driven-product-development\/","og_site_name":"Product Collective | Organizers of INDUSTRY: The Product Conference","article_publisher":"https:\/\/www.facebook.com\/productcollective","article_published_time":"2019-05-07T13:26:42+00:00","article_modified_time":"2019-06-17T02:51:31+00:00","og_image":[{"width":966,"height":534,"url":"https:\/\/productcollective.com\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-05-07-at-9.25.16-AM.png","type":"image\/png"}],"author":"Paul McAvinchey","twitter_misc":{"Written by":"Paul McAvinchey","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/productcollective.com\/experiment-driven-product-development\/","url":"https:\/\/productcollective.com\/experiment-driven-product-development\/","name":"Experiment-Driven Product Development - Product Collective | Organizers of INDUSTRY: The Product Conference","isPartOf":{"@id":"https:\/\/productcollective.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/productcollective.com\/experiment-driven-product-development\/#primaryimage"},"image":{"@id":"https:\/\/productcollective.com\/experiment-driven-product-development\/#primaryimage"},"thumbnailUrl":"https:\/\/productcollective.com\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-05-07-at-9.25.16-AM.png","datePublished":"2019-05-07T13:26:42+00:00","dateModified":"2019-06-17T02:51:31+00:00","author":{"@id":"https:\/\/productcollective.com\/#\/schema\/person\/f34ceda9eb6237e864633ca033061c0b"},"breadcrumb":{"@id":"https:\/\/productcollective.com\/experiment-driven-product-development\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/productcollective.com\/experiment-driven-product-development\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/productcollective.com\/experiment-driven-product-development\/#primaryimage","url":"https:\/\/productcollective.com\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-05-07-at-9.25.16-AM.png","contentUrl":"https:\/\/productcollective.com\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-05-07-at-9.25.16-AM.png","width":966,"height":534},{"@type":"BreadcrumbList","@id":"https:\/\/productcollective.com\/experiment-driven-product-development\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/productcollective.com\/"},{"@type":"ListItem","position":2,"name":"Experiment-Driven Product Development"}]},{"@type":"WebSite","@id":"https:\/\/productcollective.com\/#website","url":"https:\/\/productcollective.com\/","name":"Product Collective | Organizers of INDUSTRY: The Product Conference","description":"For people who build, launch and scale world-class software products.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/productcollective.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/productcollective.com\/#\/schema\/person\/f34ceda9eb6237e864633ca033061c0b","name":"Paul McAvinchey","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/productcollective.com\/#\/schema\/person\/image\/","url":"https:\/\/productcollective.com\/wp-content\/uploads\/2020\/12\/PLnHr3Hu_400x400-150x150.jpg","contentUrl":"https:\/\/productcollective.com\/wp-content\/uploads\/2020\/12\/PLnHr3Hu_400x400-150x150.jpg","caption":"Paul McAvinchey"},"description":"For over 20 years, Paul has been building and collaborating on digital products with fast-growing startups and global brands, including AOL and WMS Gaming. Currently, he's a co-founder of Product Collective, a worldwide community of product people. Members collaborate on in the exclusive Member Hub, meet at INDUSTRY: The Product Conference, listen to Rocketship.fm, learn at Product Interviews and get a weekly newsletters that includes best practices in product management. In recent years he led business development at DXY, a leading product design firm in the Midwest, and product innovation at MedCity Media, a publishing startup acquired by Breaking Media in 2015.","sameAs":["https:\/\/twitter.com\/https:\/\/twitter.com\/yaboya"],"url":"https:\/\/productcollective.com\/author\/newindsum\/"}]}},"_links":{"self":[{"href":"https:\/\/productcollective.com\/wp-json\/wp\/v2\/posts\/16415"}],"collection":[{"href":"https:\/\/productcollective.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/productcollective.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/productcollective.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/productcollective.com\/wp-json\/wp\/v2\/comments?post=16415"}],"version-history":[{"count":0,"href":"https:\/\/productcollective.com\/wp-json\/wp\/v2\/posts\/16415\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/productcollective.com\/wp-json\/wp\/v2\/media\/16460"}],"wp:attachment":[{"href":"https:\/\/productcollective.com\/wp-json\/wp\/v2\/media?parent=16415"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/productcollective.com\/wp-json\/wp\/v2\/categories?post=16415"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/productcollective.com\/wp-json\/wp\/v2\/tags?post=16415"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}