© 2014 ConceptSpring
Primary Market Research
Methodologies Overview
Primary Market Research
Methodologies Overview
Elaine Chen
© 2014 ConceptSpring
Why do research?
© 2014 ConceptSpring
You are not your customer
© 2014 ConceptSpring
© 2014 ConceptSpring
A few tenets
© 2014 ConceptSpring
Not a sales pitch
© 2014 ConceptSpring
eep your eyes and ears open
© 2014 ConceptSpring
!e open minded
© 2014 ConceptSpring
Primary Market
Research:
Sausage making
instructions
© 2014 ConceptSpring
" classes o# research
•
Problem Research
•
To understand who the buyers and users are
•
To understand the problem statement
•
To understand the context in which the product will
be used
•
To understand use cases
•
Solution Research
•
To understand the usability and utility o the product
•
To help prioriti!e the eature set
•
To understand pricing elasticity
•
To understand customer satisaction
© 2014 ConceptSpring
$he anatomy o# a research program
•
Clearly articulate a hypothesis to be tested
"a#a research goals and ob$ecti%es&
•
'e(ne the mar#et segment to do research in
•
'e%elop sub$ect recruitment criteria according
to the de(nition o the chosen segment
•
Pic# a methodology
•
Run the research program
•
Crunch the data
•
Ta#e steps based on (ndings
© 2014 ConceptSpring
E%ample recruitment &uestionnaire
'(egment) people with sleep issues*
• )hat is your age* gender and proession+ "terminate once
age,gender -uota has been (lled or the matching segment&
• )hat is your household income+ "terminate i ./xx#&
• 0re you interested in learning more about your sleep+ "terminate i
not&
• 0re you currently encountering sleep problems+ "terminate i not&
• 0re you currently under the care o a doctor or your sleep issues+
"terminate i true&
• 1ow many nights in an a%erage wee# are you encountering sleep
issues+ "select between 223&
• Please describe the sleep issues you are currently encountering
"chec# all that apply&4 "cannot all asleep5 multiple awa#enings5
cannot go bac# to sleep ater awa#ening5 snoring spouse5 etc&
• 'o you currently share your bed+ "partner* child* pet& 6 so7 how
many nights in an a%erage wee# is your sleep a8ected by your bed
partner"s&+
© 2014 ConceptSpring
Qualitative versus
quantitative
Qualitative
• 9pen ended7 you are
ha%ing a con%ersation
• 0necdotal7 you deri%e
meaning rom each
con%ersation
• Re-uires excellent listening
and obser%ation s#ills
• :est done ace to ace "or
at least by %ideo S#ype or
by %oice call&
• :est used or exploratory
research where you are
trying to get to #now a
problem or a persona
Quantitative (>1 sam!les"
• Close ended7 you are
administering a
-uestionnaire
• Statistical7 you deri%e
meaning rom loo#ing at
aggregate results
• Re-uires systematic
analysis s#ills
• :est done %ia on2line
sur%ey "or* in the olden
days* %ia phone sur%ey&
• :est used or con(rmation
research where you are
trying to %eriy , -uantiy
what you thin# is important
© 2014 ConceptSpring
+ualitative, then &uantitative
The -ualitati%e pass helps you de%elop your -uantitati%e pass and interpret
the result with context and meaning
© 2014 ConceptSpring
#ommon metho$ologies
Qualitative (%&
sam!les"
• Contextural inter%iew
• 9bser%ation , shadowing
• 6mmersion
• ;ongitudinal diary study
• Photo essay
• etc
Quantitative (>1 sam!les"
• Sur%eys
• 9n line sur%eys
• Con$oint analysis
• Pricing studies
• ?onadic
• ?ultiple monadic
• @an )estendorp
• > etc
• Customer satisaction7
APS* P,? (t
• )eb testing
• 0,: split* ?ulti%ariate
• )eb analytics
• > etc
© 2014 ConceptSpring
Some qualitative
techniques
© 2014 ConceptSpring
#onte'tural interview: interview
su()ect where !ro$uct will (e use$
© 2014 ConceptSpring
(tructure the interview #or consistency,
ask open ended &-s, let the su./ect lead
Set up
gear
Warm
up chat
“Tell us the story of sleep
in your house”
House
tour
Discuss
sleep
problems
Probe
expect-
ations
Show prouct
iea if an only
if there!s if time
left o"er an
they as#
© 2014 ConceptSpring
O.servation) learning things people cannot
articulate
6n this pro$ect* we were trying to understand peopleBs showering habits4 )e
contracted the research to an ethnographic research company* who too#
shower ootage o %olunteer sub$ects in a nudist colony4
)e were able to measure things li#e C time people stand acing the
showerhead %ersus the other way* etc4 that no one can tell us with words4
© 2014 ConceptSpring
O.servation) learning things people will
not articulate
6n this pro$ect* we were trying to understand peopleBs inotainment needs and
multitas#ing beha%ior in the car4 6n this example* the sub$ect "a real estate
agent& was tal#ing on the phone and scheduling appointments in her boo#
while dri%ing at D0mph ")ith our cowor#ers in the bac#* who were shadowing
her or Eh that day &
© 2014 ConceptSpring
0mmersion) .ecome the user
6n this pro$ect* we were trying to understand the usability o competiti%e C0'
programs4 )e had our in house C0' experts try out F or 4 applications4 They
spent F days on each application* designing the same set o parts and
documenting their experience learning and mastering each application4
© 2014 ConceptSpring
1ongitudinal diary studies) learn things over
days2weeks, not over 34"h
© 2014 ConceptSpring
Photo essays) use imagery to capture the
conte%t
© 2014 ConceptSpring
5sa.ility .enchmark) 6ive users a task, watch
them struggle and learn why
© 2014 ConceptSpring
7ocus groups) not a #an8 9ynamics hard to
manage, e%pensive to do right
6mage credit7 msresearchinc4com
© 2014 ConceptSpring
Some quantitative
techniques
© 2014 ConceptSpring
7irst things :rst) the product concept
6n an on line sur%ey* you ha%e 2 pages to ma#e your point about your
productBs %alue proposition and its rele%ance to your sur%ey respondents4 =or
all practical purposes* you are creating an ad4 The e8ecti%eness o your
sur%ey depends on this ad4 6n%est in the creati%e content to ma#e your point4
Get a mar#eter to help do it right4
© 2014 ConceptSpring
On line surveys ; gauging interest
Construct your sur%ey careully or easy tabulation o results4
:est practice is to use 3 point scales* except or speci(c methodologies that
re-uire 10 point scales "e4g4 Aet promoter score H see slide in customer
satisaction section&
There are techni-ues to IcodeJ open ended responses H read more here4
http7,,www44uwm4edu,cuir,resources,upload,Planning2Council2-ualitati%e2
analysis2handout4pd
© 2014 ConceptSpring
Con/oint analysis
•
Con$oint analysis measures the %alue the mar#et
places on each eature o your product and
predicts the %alue o any combination o eatures4
•
1ow it wor#s7
• 0s# -uestions that orce respondents to ma#e trade2o8s
among eatures
• 'etermine the %alue they place on each eature based
on the trade2o8s they ma#e
• Simulate how the mar#et reacts to %arious eature trade2
o8s you are considering
•
Read more at7
http7,,www4pragmaticmar#eting4com,resources,co
n$oint2analysis2101Ksthash4d-LG%TpS4dpu
© 2014 ConceptSpring
Pricing research) Monadic ; a8k8a8 an Westendorp 'the
indle team did this and came up with ?@@ *
• Show the product concept* then as# these 4 -uestions with
a ree response7
• 0t what price would you consider the product to be so
expensi%e that you would not consider buying it+ "Too
expensi%e&
• 0t what price would you consider the product to be priced so
low that you would eel the -uality couldnBt be %ery good+ "Too
cheap&
• 0t what price would you consider the product starting to get
expensi%e* so that it is not out o the -uestion* but you would
ha%e to gi%e some thought to buying it+ "Mxpensi%e,1igh Side&
• 0t what price would you consider the product to be a bargainN
a great buy or the money+ "Cheap,Good @alue&
• Chart cumulati%e re-uencies4 6ntersection O ideal price4
• Read more here4
http7,,en4wi#ipedia4org,wi#i,@anP)estendorpQsPPricePSensiti
%ityP?eter
© 2014 ConceptSpring
Customer satis#action measure) Net
Promoter (core
• Show the product concept* then as# this -uestion on a 10
point scale "1 O not at all li#ely* 10 O %ery li#ely&7
• I1ow li#ely is it that you would recommend Ryour productS to a
riend Ror amily* or colleague etcS+J
• Categori!e responses7
• Promoters O T* 10 H your loyal enthusiasts
• Passi%es O U* E H satis(ed but unenthusiastic
• 'etractors O 02D H unhappy* can damage brand by word o mouth
• Calculate the Aet Promoter Score as ollows7 "APS&
• APS O C Promoters 2 C'etractors
• Calibrating yoursel
• 201F scores7 0pple laptop "best computer company& H UDC5
Vaiser "best health insurance company& H FDC 2 (nd your industry
or comparables
• Read more here7
http7,,www4netpromoter4com,why2net2promoter,#now,
© 2014 ConceptSpring
Customer satis#action measure) Product 2
Market 7it
•
Show the product concept* then as# this -uestion
on a F point scale "1 O not at all disappointed* 2
O somewhat disappointed* F O %ery
disappointed&
• I1ow would you eel i they could no longer use Rthe
productS+J
•
6 o%er 40C o responses O F "%ery disappointed&
you ha%e achie%ed Product ?ar#et =it
•
Read more here7
http7,,www4startup2mar#eting4com,using2sur%ey2io
,
© 2014 ConceptSpring
A word o# caution) know your statistics and
e%perimental design
•
The mean o any metric is meaningless
without the %ariance4 ;earn how to do
analysis o %ariance "0A9@0& properly to
interpret your results correctly4 Start with
the student2t test and build up rom there
•
The best 0A9@0 e8ort with the best
experimental design alls down with
insuWcient sample si!e4
•
300 is the absolute minimum
•
3000 is the best practice number
•
1000 is a good compromise
© 2014 ConceptSpring
A2! split testing
6mage credit7 http7,,www4smashingmaga!ine4com
© 2014 ConceptSpring
Multivariate testing
6mage credit7 http7,,%isualwebsiteoptimi!er4com
© 2014 ConceptSpring
We. analytics
6mage credits7 emaginewebmar#eting4com* http7,,www4openwebanalytics4com,
© 2014 ConceptSpring
$he possi.ilities are endless
© 2014 ConceptSpring
*chenelai
ne
(log+conce!ts!ring
+com
$hank you
doc_658967562.pptx
Primary Market Research
Methodologies Overview
Primary Market Research
Methodologies Overview
Elaine Chen
© 2014 ConceptSpring
Why do research?
© 2014 ConceptSpring
You are not your customer
© 2014 ConceptSpring
© 2014 ConceptSpring
A few tenets
© 2014 ConceptSpring
Not a sales pitch
© 2014 ConceptSpring
eep your eyes and ears open
© 2014 ConceptSpring
!e open minded
© 2014 ConceptSpring
Primary Market
Research:
Sausage making
instructions
© 2014 ConceptSpring
" classes o# research
•
Problem Research
•
To understand who the buyers and users are
•
To understand the problem statement
•
To understand the context in which the product will
be used
•
To understand use cases
•
Solution Research
•
To understand the usability and utility o the product
•
To help prioriti!e the eature set
•
To understand pricing elasticity
•
To understand customer satisaction
© 2014 ConceptSpring
$he anatomy o# a research program
•
Clearly articulate a hypothesis to be tested
"a#a research goals and ob$ecti%es&
•
'e(ne the mar#et segment to do research in
•
'e%elop sub$ect recruitment criteria according
to the de(nition o the chosen segment
•
Pic# a methodology
•
Run the research program
•
Crunch the data
•
Ta#e steps based on (ndings
© 2014 ConceptSpring
E%ample recruitment &uestionnaire
'(egment) people with sleep issues*
• )hat is your age* gender and proession+ "terminate once
age,gender -uota has been (lled or the matching segment&
• )hat is your household income+ "terminate i ./xx#&
• 0re you interested in learning more about your sleep+ "terminate i
not&
• 0re you currently encountering sleep problems+ "terminate i not&
• 0re you currently under the care o a doctor or your sleep issues+
"terminate i true&
• 1ow many nights in an a%erage wee# are you encountering sleep
issues+ "select between 223&
• Please describe the sleep issues you are currently encountering
"chec# all that apply&4 "cannot all asleep5 multiple awa#enings5
cannot go bac# to sleep ater awa#ening5 snoring spouse5 etc&
• 'o you currently share your bed+ "partner* child* pet& 6 so7 how
many nights in an a%erage wee# is your sleep a8ected by your bed
partner"s&+
© 2014 ConceptSpring
Qualitative versus
quantitative
Qualitative
• 9pen ended7 you are
ha%ing a con%ersation
• 0necdotal7 you deri%e
meaning rom each
con%ersation
• Re-uires excellent listening
and obser%ation s#ills
• :est done ace to ace "or
at least by %ideo S#ype or
by %oice call&
• :est used or exploratory
research where you are
trying to get to #now a
problem or a persona
Quantitative (>1 sam!les"
• Close ended7 you are
administering a
-uestionnaire
• Statistical7 you deri%e
meaning rom loo#ing at
aggregate results
• Re-uires systematic
analysis s#ills
• :est done %ia on2line
sur%ey "or* in the olden
days* %ia phone sur%ey&
• :est used or con(rmation
research where you are
trying to %eriy , -uantiy
what you thin# is important
© 2014 ConceptSpring
+ualitative, then &uantitative
The -ualitati%e pass helps you de%elop your -uantitati%e pass and interpret
the result with context and meaning
© 2014 ConceptSpring
#ommon metho$ologies
Qualitative (%&
sam!les"
• Contextural inter%iew
• 9bser%ation , shadowing
• 6mmersion
• ;ongitudinal diary study
• Photo essay
• etc
Quantitative (>1 sam!les"
• Sur%eys
• 9n line sur%eys
• Con$oint analysis
• Pricing studies
• ?onadic
• ?ultiple monadic
• @an )estendorp
• > etc
• Customer satisaction7
APS* P,? (t
• )eb testing
• 0,: split* ?ulti%ariate
• )eb analytics
• > etc
© 2014 ConceptSpring
Some qualitative
techniques
© 2014 ConceptSpring
#onte'tural interview: interview
su()ect where !ro$uct will (e use$
© 2014 ConceptSpring
(tructure the interview #or consistency,
ask open ended &-s, let the su./ect lead
Set up
gear
Warm
up chat
“Tell us the story of sleep
in your house”
House
tour
Discuss
sleep
problems
Probe
expect-
ations
Show prouct
iea if an only
if there!s if time
left o"er an
they as#
© 2014 ConceptSpring
O.servation) learning things people cannot
articulate
6n this pro$ect* we were trying to understand peopleBs showering habits4 )e
contracted the research to an ethnographic research company* who too#
shower ootage o %olunteer sub$ects in a nudist colony4
)e were able to measure things li#e C time people stand acing the
showerhead %ersus the other way* etc4 that no one can tell us with words4
© 2014 ConceptSpring
O.servation) learning things people will
not articulate
6n this pro$ect* we were trying to understand peopleBs inotainment needs and
multitas#ing beha%ior in the car4 6n this example* the sub$ect "a real estate
agent& was tal#ing on the phone and scheduling appointments in her boo#
while dri%ing at D0mph ")ith our cowor#ers in the bac#* who were shadowing
her or Eh that day &
© 2014 ConceptSpring
0mmersion) .ecome the user
6n this pro$ect* we were trying to understand the usability o competiti%e C0'
programs4 )e had our in house C0' experts try out F or 4 applications4 They
spent F days on each application* designing the same set o parts and
documenting their experience learning and mastering each application4
© 2014 ConceptSpring
1ongitudinal diary studies) learn things over
days2weeks, not over 34"h
© 2014 ConceptSpring
Photo essays) use imagery to capture the
conte%t
© 2014 ConceptSpring
5sa.ility .enchmark) 6ive users a task, watch
them struggle and learn why
© 2014 ConceptSpring
7ocus groups) not a #an8 9ynamics hard to
manage, e%pensive to do right
6mage credit7 msresearchinc4com
© 2014 ConceptSpring
Some quantitative
techniques
© 2014 ConceptSpring
7irst things :rst) the product concept
6n an on line sur%ey* you ha%e 2 pages to ma#e your point about your
productBs %alue proposition and its rele%ance to your sur%ey respondents4 =or
all practical purposes* you are creating an ad4 The e8ecti%eness o your
sur%ey depends on this ad4 6n%est in the creati%e content to ma#e your point4
Get a mar#eter to help do it right4
© 2014 ConceptSpring
On line surveys ; gauging interest
Construct your sur%ey careully or easy tabulation o results4
:est practice is to use 3 point scales* except or speci(c methodologies that
re-uire 10 point scales "e4g4 Aet promoter score H see slide in customer
satisaction section&
There are techni-ues to IcodeJ open ended responses H read more here4
http7,,www44uwm4edu,cuir,resources,upload,Planning2Council2-ualitati%e2
analysis2handout4pd
© 2014 ConceptSpring
Con/oint analysis
•
Con$oint analysis measures the %alue the mar#et
places on each eature o your product and
predicts the %alue o any combination o eatures4
•
1ow it wor#s7
• 0s# -uestions that orce respondents to ma#e trade2o8s
among eatures
• 'etermine the %alue they place on each eature based
on the trade2o8s they ma#e
• Simulate how the mar#et reacts to %arious eature trade2
o8s you are considering
•
Read more at7
http7,,www4pragmaticmar#eting4com,resources,co
n$oint2analysis2101Ksthash4d-LG%TpS4dpu
© 2014 ConceptSpring
Pricing research) Monadic ; a8k8a8 an Westendorp 'the
indle team did this and came up with ?@@ *
• Show the product concept* then as# these 4 -uestions with
a ree response7
• 0t what price would you consider the product to be so
expensi%e that you would not consider buying it+ "Too
expensi%e&
• 0t what price would you consider the product to be priced so
low that you would eel the -uality couldnBt be %ery good+ "Too
cheap&
• 0t what price would you consider the product starting to get
expensi%e* so that it is not out o the -uestion* but you would
ha%e to gi%e some thought to buying it+ "Mxpensi%e,1igh Side&
• 0t what price would you consider the product to be a bargainN
a great buy or the money+ "Cheap,Good @alue&
• Chart cumulati%e re-uencies4 6ntersection O ideal price4
• Read more here4
http7,,en4wi#ipedia4org,wi#i,@anP)estendorpQsPPricePSensiti
%ityP?eter
© 2014 ConceptSpring
Customer satis#action measure) Net
Promoter (core
• Show the product concept* then as# this -uestion on a 10
point scale "1 O not at all li#ely* 10 O %ery li#ely&7
• I1ow li#ely is it that you would recommend Ryour productS to a
riend Ror amily* or colleague etcS+J
• Categori!e responses7
• Promoters O T* 10 H your loyal enthusiasts
• Passi%es O U* E H satis(ed but unenthusiastic
• 'etractors O 02D H unhappy* can damage brand by word o mouth
• Calculate the Aet Promoter Score as ollows7 "APS&
• APS O C Promoters 2 C'etractors
• Calibrating yoursel
• 201F scores7 0pple laptop "best computer company& H UDC5
Vaiser "best health insurance company& H FDC 2 (nd your industry
or comparables
• Read more here7
http7,,www4netpromoter4com,why2net2promoter,#now,
© 2014 ConceptSpring
Customer satis#action measure) Product 2
Market 7it
•
Show the product concept* then as# this -uestion
on a F point scale "1 O not at all disappointed* 2
O somewhat disappointed* F O %ery
disappointed&
• I1ow would you eel i they could no longer use Rthe
productS+J
•
6 o%er 40C o responses O F "%ery disappointed&
you ha%e achie%ed Product ?ar#et =it
•
Read more here7
http7,,www4startup2mar#eting4com,using2sur%ey2io
,
© 2014 ConceptSpring
A word o# caution) know your statistics and
e%perimental design
•
The mean o any metric is meaningless
without the %ariance4 ;earn how to do
analysis o %ariance "0A9@0& properly to
interpret your results correctly4 Start with
the student2t test and build up rom there
•
The best 0A9@0 e8ort with the best
experimental design alls down with
insuWcient sample si!e4
•
300 is the absolute minimum
•
3000 is the best practice number
•
1000 is a good compromise
© 2014 ConceptSpring
A2! split testing
6mage credit7 http7,,www4smashingmaga!ine4com
© 2014 ConceptSpring
Multivariate testing
6mage credit7 http7,,%isualwebsiteoptimi!er4com
© 2014 ConceptSpring
We. analytics
6mage credits7 emaginewebmar#eting4com* http7,,www4openwebanalytics4com,
© 2014 ConceptSpring
$he possi.ilities are endless
© 2014 ConceptSpring
*chenelai
ne
(log+conce!ts!ring
+com
$hank you
doc_658967562.pptx