data gathering
why gather data?
goals
information about users
user behaviors
user reactions to technology
important to set goals at start
hypotheses or assumptions
data
types of data
quantitative
qualitative
quantitative
numbers
measurements
time
logs
qualitative
words
quotes
photos
videos
sketches
triangulation
investigate from multiple perspectives
different sources
different investigators
different theories
different data gathering techniques
make up for data limitations
data recording
needed for analysis
self-documenting
notes
photographs
record video / audio
notes
least technical
most flexible
handwritten or typed
transcription
photographs
artifacts, events, and environment
supplement notes and sketches
audio
alternative to note-taking
less intrusive than video
investigator attention
transcription
video
short clips with phone
longer videos with camera
participants
identifying participants
who to gather data from?
population
chosen
recruitment
sampling
saturation sampling
probability sampling
nonprobability sampling
convenience sampling
snowball sampling
incentives
consent
informed
voluntary
protect interest of all parties
interviews
unstructured
open-ended
no expectation about format or content of answers
still needs plan
rich data
time consuming to analyze
structured
ask predetermined questions
short and clearly worded
useful when goals understood
semi-structured
combine features of structured and unstructured
follow script for guidance
probe for more information
neutral probes
follow emergent lines of questioning
focus groups
interview groups of people
3-10 participants
developing interview questions
open-ended or close-ended
exploratory vs focused
keep it simple
straightforward explanation
neutral
examples
How amazing is the product?
Would you rather use the old version or this improved version of the application?
I saw you were having difficulty with the navigation. What happened?
running the interview
listen, not talk
introduction and why
warm-up questions
key probing questions
cooling-off questions
conclusion
medium
face-to-face
online
phone
props and artifacts
enrich interview
provide context
provoke memories
critical incident technique
human memory
recall most recent or impactful
describe experience
asks questions based upon description
questionnaires
structure
basic demographic information
key questions
divided into categories
order matters
length matters
question and response format
check boxes and ranges
predictable responses or limits
ranges for less precise responses
rating scales
responses for comparison among participants
good for asking judgments
likert scales
measuring opinions, attitudes, and beliefs
agreement with a statement
semantic differential scales
range of bipolar attitudes
pair of adjectives
administering
representative sample
response rate
sampling technique for large samples
online or paper
pilot questionnaire
observations
direct observation in the field
difficult for participants to explain what they do
perceptions vs actual behavior
help fill in the details
balance between directed at goals and openness
structuring observation
frameworks
help focus observations
who, where, what?
attention to context of activity
space
actors
activities
objects
acts
events
time
goals
feelings
participant observer
become member of group studied
difficult to separate roles
better context
better understanding of perceptions
influence on others
planning and conducting
choosing a framework
how to record data
build rapport
how to handle sensitive topics
different perspectives
debrief
ethnography
no framework to start
everything strange
participants perspective
increasingly popular within interaction design
nuanced understanding
participant observer
many forms of data
direct observation in controlled environments
usability lab
more formal
ease some aspects of data collection
think-aloud
field observation limits questions
can be more intrusive in controlled settings
participants verbally describe thoughts and actions
understand user perceptions
indirect observation
direct not possible
too intrusive
activities tracked indirectly
diaries
participants asked to write a diary
useful when unreachable or activity is private
take less of researcher's time to collect
require minimal equipment
suitable for long studies
participant reliability
collect other data beyond writings
interaction logs
record users activity through software
unobtrusive
ethical concerns
data scraping
collect data from online venues, such as forums
examine trail of activities
automation and visualization
choosing and combining techniques
interviews
exploring issues
mostly qualitative data
guided
artificial environment
focus groups
collecting multiple view points
mostly qualitative data
areas of consensus
dominant characters
questionnaires
answer specific questions
quantitative and qualitative data
reach many people
design is key; response rates
direct field observations
understanding context of activity
mostly qualitative data
unique insights
time-consuming; lots of data
direct controlled observation
capturing detail of individual actions
quantitative and qualitative data
focus on task details without interruption
results may have limited implications
indirect observation
less disruptive; easier data collection
quantitative and qualitative data
doesn't distract participants
need tools for analysis; participant exaggeration
questions?
reading for next class
Chapter 11: Discovering Requirements
Interaction Design: Beyond Human-Computer Interaction
