Drawing in Evaluation: Deep Dive

Did I illustrate this blog post with crayons and scrap paper to prove a point? Yes. Yes I did. Enjoy!

Drawing is the most contentious, nail-biting, quickly scan the room for an exit sign method that I've worked with to date.

Oh, I should mention that I mostly work with adults. That explains it, right?

When I worked with kids, drawing was no problem. A box of crayons was an invitation to create - if nothing else - a mess. Kids could draw to their heart's content and then mush the wax nubs together to create a candle. When in doubt, set things on fire - that's our motto at AND.

Just kidding! Jennica put the extinguisher down.

Adults often see that box of crayons as a threat. Scared of being "bad" at drawing and looking foolish, many people refuse to draw - even if it is the right tool for the job. Sure, embarrassment is a significant motivator for behaviour. But, this hesitation towards arts and silliness goes beyond individual behaviour. You know, because it's never really about individual behaviour.

At a societal - rather than individual - level in Canada, I have seen arts deprioritized. Whether through public education, funding for arts programs, and/or the expectations for artists to work for less than their worth, arts are seen as a nice-to-have but not a need-to-have. Young people get the artist trained out of them when they are encouraged to pursue "serious" money-making careers. Because arts aren't always a lucrative path, this can be complicated for folks who support families and/or feel responsible for lifting themselves and their communities out of poverty. When investing in arts is not normalized in society, it is tricky to get people to professionally invest in artistic practices. So, when we suggest to professionals that drawing might be the exact right tool for their evaluation, we see those eyes dart towards the exit.

Yes, introducing drawing into evaluation can be a challenge. And also...it is an AMAZING tool!

What is drawing as an evaluation technique?

As a technique, the process of drawing or doodling is used to elicit reflections or data. It is similar to photo elicitation (read more) with one key difference. When people take a picture, they capture an image of something that already exists. For example, if someone takes a picture of a tree, that tree exists. They add their own interpretation when deciding to take a picture of a tree and which elements are in frame (and which are not). When a person draws, they invent from their imagination. Their only starting point is a blank page. So, if they want to represent a tree they must conjure one based on (a) their recollection of what a tree is (or isn't) and (b) their ability to translate the image in their head onto the page.

But, drawing is not an existential crisis. It is precise technique that is helpful because:

1. The symbols, figures, and colours can help people to describe or make sense of things that are otherwise hard to articulate (e.g., emotions, gut reactions, instincts, nostalgia, abstract thinking).

2. It can potentially help to address power imbalances in groups.

   1. We have found that when we ask folks in positions of relative power to be vulnerable, we can get to interesting and meaningful places in a group exercise. For example, when we have used drawings with doctors and patient partners, we found that doctors deferred to patient partners to speak up.

   2. In most groups, there tends to be one loud voice who sucks all the oxygen out of the room. Language-based activities privilege loud voices. Drawing, however, invites introspection, space, and pause. This creates an opportunity to redistribute the oxygen and hear from folks who may not otherwise have the chance to share.

   3. When the goal of a group session is to honour divergence (rather than seeking consensus), drawing can help people share differences more authentically and constructively. Because no two drawings will be the same, the method lends itself well to understanding nuances in how folks differently arrive at their current perspectives without judgement or an expectation to conform.

3. Drawing helps to facilitate honest discussion. It is easy to tip-toe around hard concepts with words - we obscure what we mean with buzzwords that don't mean anything or gloss over the things we don't want to name. When we draw - that thing is on the page (or there is a noticeable amount of negative space). This make it easier to name that thing we aren't saying. Yes, that does mean that drawing can be vulnerable.

While there are many ways to structure a project, these are the core elements of our drawing projects:

Nichols, J. & Lefkowich, M. (2022)

• An evaluation question that guides the whole process

• A group or individual drawing activity

• A qualitative debrief (e.g., interview or focus group)

• Data analysis

• Data use

The technique works when drawing sparks a different way of thinking and a qualitative debrief captures those insights. Drawing excels when:

• The perspectives, reflections, or insights you need to satisfy your guiding question are hard to talk about directly (e.g., a health experience, memories of a specific instance or event, aspirations, motivations, intuition, gut wisdom, emotions)

• There is an aspect of speculation or imagination embedded in the global evaluation question (e.g., what would be helpful, what could work, what change do people envision, what success could be in a specific context)

• Words aren't the right tool (e.g., the topic can cause people to get tongue-tied, quiet, or unsure)

• Folks who will be engaged as participants are not into surveys or traditional interviews (e.g., if folks have already been exhausted with requests for interviews and want something else to engage their expertise)

• The project taps into underlying assumptions, values, beliefs, or biases (e.g., social norms, personal motivations, how people perceive or feel about...)

The Drawing Activity

In line with our guiding question and careful considerations about the context and group we're working with, we typically structure a drawing activity as follows.

	Individual	Group
In Person	​Ahead of or during an interview or qualitative debrief, the facilitator gives a participant a prompt/question, drawing supplies, and instructions. The participant then draws a visual response. The facilitator and participant discuss the elements of the drawing and their meaning relative to the global evaluation questions (e.g., lines, shapes, colours, symbols, figures)	The facilitator gives a group a prompt/question, drawing supplies, and instructions. The group draws individual or collective responses. The facilitator then leads the group through a collective unpacking of the different images, similar and different meanings/interpretations, and group wisdom related to the global evaluation question.
Pros	​Does not require a lot of time to be effective Can help individuals to think and express themselves differently Can stimulate imagination and tactile wisdom Can be cathartic and give the person an outlet to fidget or let their mind wander	In addition to individual benefits: Can help groups connect over a shared experience of potential vulnerability (i.e., showing each other their drawings) Effective at building consensus or honouring and holding space for divergence
Cons	​May privilege visual and sight-based wisdom and fine motor skills (the activity can be adapted for accessibility) Requires the evaluator to have a more advanced interview skillset Requires the facilitator to bring appropriate supplies based on their global question	In addition to the points to the left: ​Requires additional thoughtfulness around vulnerability and privacy as folks will likely be asked to share their drawings Requires people to be together in the same space, which may not be accessible or comfortable for all folks in light of COVID-19 and remote work options
	Individual	Group
Online	​​Ahead of or during an interview or qualitative debrief, the facilitator gives a participant a prompt/question and instructions. The participant then sources their own supplies or uses an online program to draw a visual response. The participant sends a copy of their finished drawing to the facilitators (e.g. If they draw on paper, the participant typically sends a photo). The facilitator and participant then discuss the different elements of the drawing and their meaning relative to the global evaluation question(s).	​Ahead of or during a focus group or group planning session, the facilitator gives a group of participants a prompt/question and instructions. The group then draw individual or collective responses using their own materials or an online program that allows for collaboration. Participants can describe their picture to the group or send it to the facilitator to be displayed collectively. The facilitator then leads the group through a collective unpacking of the different images, similar and different meanings/interpretations, and group wisdom related to the global evaluation question(s).
Pros	​Eliminates barriers to participation related to geography or potential health concerns The experience is close-enough to the in-person version that the online modification does not feel clunky or disruptive to an interview flow	​Accommodates teams that work remotely Spices up group planning sessions that tend to drag on (no, this is not a meeting that should have been an email)
Cons	​In addition to the cons listed for in-person, the online version: Requires participants to source their own supplies (which could be as simple as the back of an envelope and a pen or as complex as a full stationary set depending on the evaluation goal) Requires participants to have access to a phone or online platform for the interview May require participants to have access to technology, wifi, and some knowledge of online programs for digital drawing or send a picture of the drawing to the facilitator	​​In addition to the cons listed on the left: Requires participants to have access to an online platform for a group video conference Requires more in-depth knowledge of video conferencing and collaborative platforms.
	Individual	Group
​Asynchronous	The facilitator gives a participant a prompt/question and drawing instructions. The participant then sources their own supplies or uses an online program to draw a visual response. Then, they unpack the elements and meaning of their drawing either: (a) through a written response, (b) voice memo, or (c) online submission platform (e.g., a survey tool).	​We would not recommend an asynchronous group activity. There are too many potential "people factor" issues.
Pros	​Creates more flexible participation options for folks who prefer to go at their own pace, or could benefit from a care partner helping to facilitate the activity Potentially increases participation rates
Cons	​Does not allow the facilitator to prompt further or clarify Creates additional potential for confusion or misinterpretation (therefore, requires a facilitator with more expertise) Requires the submission to have both drawings and a comprehensive explanation to ensure that the data are useful. Requires more careful planning (e.g., detailed instructions, frequent reminders, ethical considerations regarding data use)

But, I can't draw...

Good thing that the goal is to collect data, not find the greatest artist of our time. A person's artistic skill is irrelevant. Besides, terrible drawings make for fabulous discussions! Very simple elements like lines, abstract shapes, stick figure people, negative space, and splotches of colour tell complex stories about relationships, memories, and experiences.

The key is not how well a participant can draw - it is how well an evaluator can facilitate. We do not expect anyone to be able to draw a horse from memory.

That being said, I do enjoy when people attempt to draw a horse or moose with no reference photo. Please make my day and send me your "no reference photo" horse or moose drawing to publications@andimplementation.ca. I'll even go first and show you mine (because I never ask people do things I wouldn't do).

What shape is a moose though?

Eliciting Data

The drawing activity is paired with a qualitative technique (e.g., an interview, focus group, or asynchronous prompt) to elicit:

• Insights, perceptions, or personal reflections

• Experiences or memories

• Shared or different meanings and assumptions (when a group of people are involved)

PART 1: Facilitated Drawing The drawing activity taps into wisdom that is most relevant to the project (leaving room for surprises). A facilitator will use a question or prompt to direct the drawing. For example, the facilitator may ask participants to imagine themselves at a certain point in time. This could be at the end of a program the person is participating in, before participating in an intervention of some kind, when they were their happiest etc. Or, the facilitator could ask participants to draw a key concept entangled in the evaluation question to understand underlying meanings, assumptions, and values (e.g., what does "community" mean to you?).

PART 2: Qualitative Debrief The facilitator conducts an interview, focus group, group planning session, or asynchronous activity to distill:

Descriptions of what is or is not in the drawing

• What different aspects of the drawing mean, symbolize, or represent

• How the wisdom gained relates to the global evaluation question

• (if in a group setting) shared or divergent perspectives

Analyzing Data

The dataset gained through this process is the qualitative reflections on the drawings and/or drawing process. The evaluator can then choose the data analysis technique that best corresponds to their question and the type of reflections gained in their interview (e.g., discourse analysis, descriptive analysis, thematic analysis, grounded theory). We often recommend that participants (alone or with the facilitator) create a short caption to accompany their drawing so that the essence of their wisdom is not lost in the final deliverable.

But aren't the drawings data also?

In evaluations, we typically do not recommend treating drawings as data because it is challenging and potentially unhelpful. Since drawings come from people's imaginations, it is impossible to claim with certainty:

• What figures, colours, or symbols mean

• What is not in the drawing

• If what is in the drawing represents what the person was thinking at the time OR if it is what they were able to draw

• If the drawing is really how the person feels OR if it is what they felt comfortable sharing

Another reason why we do not recommend using the drawings along as data is because underlying assumptions or biases often come to the surface during the qualitative. For example, did someone assign gender to particular people in their image? It may be startling for folks to be confronted with implicit biases, assumptions, or beliefs that they did not know they had. For this reason, we need extra sensitivity to avoid holding people hostage in one particular thought they had at one point in time. The ultimate goal of our evaluation practice at AND is to support learning. Therefore, we do not use arts to trap them in a (un)learning moment or turn around and say, "gotcha!"

Understanding that the qualitative responses will ultimately become the dataset allows a skilled facilitator to both capture what came up for the person when reflecting on their drawing AND what they think now that they are aware of whatever came up. The interview or focus group therefore invites movement, uncertainty, and growth. These data are far more useful and accurate because they account for the authentic ways in which people live with their own contradictions, change their minds, and evolve.

Umm...ethics?

Ethics are important - whether or not you have a formal ethics review process as part of your evaluation practice. While this is not an exhaustive list (or a way to bypass formal review if you have that mandate) here are some questions we find helpful when planning a drawing project:

• How will consent work for participation?

• How will you be thoughtful about potential risks? (e.g., What happens if the process of drawing or reflecting on a drawing evokes uncomfortable feelings, memories of a difficult experience, or past pain?)

• How will you be thoughtful about group dynamics (if using a group option)? (e.g., How will you account for vulnerability with sharing photos, potential power dynamics, and divergent perspectives between colleagues? How will you be mindful about the potential for implicit assumptions or icky reflections to surface in a group? Will you promise a "safe space" - if so, how?)

• How will consent work for using drawings in your project outputs? (e.g., How can someone consent to you using their drawing before they make it or know what it means?)

• Who owns the drawings and resulting qualitative data? (e.g., Will there be limits on how drawings can be used after/by whom? Are there existing data ownership rules in place for this group or community to be mindful of? What does data ownership mean in the context of this project and its outputs?)

• How will you avoid holding people hostage in drawings? (e.g., Will you invite people to add to or change their drawings after their debrief?)

• What happens if people change their mind about participation? (e.g., Because you cannot "unsee" a drawing, is there a limit on when people can withdraw?)

Limitations

Because drawing can be a vulnerable exercise, there are more limitations than other techniques that require less imagination and bravery. These are some limitations that we have found in our practice at AND:

1. Drawing isn't always appropriate This exercise does not work well with folks who do not connect with visuals or where image-based communication is not accessible.

2. Pairing a vulnerable or potentially risky question with drawing is not a good idea Because drawing can be vulnerable already, adding more vulnerability to the mix with a sensitive, personal, or contentious question/prompt is not a good idea. When people are overwhelmed with the amount of bravery required to participate, they won't. So, we try to balance how much we ask of people to to avoid evoking a flight, fight, freeze, or fawn response.

3. Drawing could require more effort to get buy-in It can be challenging to understand if people aren't into drawing because it genuinely isn't right for them, or because they internalized some myths about arts-based methods. When drawing is the best technique for the evaluation question, we have to invest more time and effort into demonstrating why folks should give drawing a chance. Not everyone (AND included) has time, resources, or capacity to educate clients, organizations, participants, and communities about the benefits of drawing.

4. The evaluator needs to be a highly-skilled facilitator to do drawing justice There are a lot of moving pieces in this technique. While drawing may seem simple, the facilitator must be able to ask the right question to evoke a meaningful drawing, give clear instructions for the activity, set meaningful boundaries for individual or group drawings, attend to descriptive and interpretive aspects of drawings in the debrief, respectfully prompt for underlying or implicit assumptions, capture movement and contradiction, and distill the essence of what the drawings mean to satisfy the global evaluation question. All while upholding ethics and attending to people's vulnerability. That's a lot!

Words of Caution

Not all drawing projects are done well.

At AND, we are on a mission to support meaningful arts-based practice. The goal isn't to gate-keep. The goal is to hold ourselves and our colleagues accountable because creative practice carries just as many risks as any other technique. But, too many people sidestep their responsibilities as evaluators and researchers by claiming that there is no way to evaluate creative techniques. While it is true that too often the wrong metrics of success are applied to arts projects, it is simply not true to claim that arts-based methods are above or exempt from critique.

Here are things I watch out for in drawing projects:

• Claims that drawing offers health benefits (Drawing can be cathartic, absolutely! But, unless the goal of the project is explicitly to support wellness, healing, repair, or health AND there are appropriate people involved to support this, evaluators cannot claim health as a benefit - and if they do, they need to back it up. Conflating arts therapy with arts-based methods for data collection is a no no for me.)

• Suggestions that drawing is "giving voice to the voiceless" (I've said it before, and I'll say it again. There is no such thing as a voiceless person. While drawing may be a good outlet for self-expression for some people, it does not bestow the gift of voice. Rather than continuing to imagine some people as voiceless, let's instead ask questions about what we are going to do with the wisdom people do offer us. Are we ready to listen and act? Are we ready to make space for folks in decision-making? Are we ready to take a little less credit and practice that humility thing? If people aren't sharing their thoughts with us, why might that be? What can we do to make sure they can engage on their terms?)

• Limiting drawing to kids or suggesting it is inherently a good fit for all kids (Drawing isn't inherently better or worse for any group of people. It depends on the evaluation question, context, and skills of the evaluator. We all need to stay open-minded about when drawing is the right choice - the answer could be surprising.)

• Using drawing interchangeably with other arts-based techniques (Not all arts do the same things, work the same way, or elicit the same kinds of data.)

• Claiming that any visual activity is drawing (While drawing is indeed a wide spectrum of visual representation activities - it is not a catch-all for anything that isn't a word or number. There are nuances in the craft of visually representing and eliciting ideas that deserve respect. I am sceptical when: the actual act of visually representing something would have worked equally well with words or numbers, data are elicited another way and then the evaluator or an artist illustrates what others have said afterwards (i.e., data visualization or knowledge translation), or visuals serve as a short-hand for words or numbers and do not elicit further reflection (e.g., colour-coding themes).)

• Assuming that drawings will speak for themselves (Nope! Not possible. A drawing cannot speak for itself and it should not be left up to interpretation from readers or audiences. Because the drawing came from the imagination of the artist, no one else can ever truly know what it means. Period.)

Learn More

There are WAY more drawing resources available for researchers rather than evaluators. That's why we developed our beginner arts-based course and organizational workshops for evaluators to translate techniques across and ensure that context, theoretical foundations, and practical applications make sense.

1. We offer organizational workshops teaching this (and other arts-based techniques). Ranging from 90 minutes to a full-day deep dive, we teach hands-on skills for this method with folks who have any amount of experience. Book now!

2. Drawing is one of the 4 techniques covered in our beginner arts-based methods course for evaluators. No photography experience is required. Learn more.

But, you don't have to work with us or take our course to learn more!

Here are some resources to help you get started. Quick heads-up... Many of these require a license, purchase, or subscription. These are English-language resources (that is my limitation that I will absolutely own - there are way more valuable resources in other languages). We are not endorsing these resources or profiting in any way. Please do your own assessment to see if they are a good fit for you. None of these resources have explicit applications for program evaluation - they come from research and strategic planning. So, if your goal is to use these in your evaluation practice, some finesse will be needed.

• Guillemin M. (2004). Understanding illness: using drawings as a research method. Qualitative Health Research, 14(2), 272-89. [Link]

• Leavy, P. (2020). Method Meets Art: Arts-Based Research Practice (3rd Ed.). Guildford Press. [Link]

• Lyon, P. (2020). Using drawing in visual research: materializing the invisible. In L. Pauwels, & D. Mannay The sage handbook of visual research methods (pp. 297-308). SAGE Publications, Inc. [Link]

• Miller, D.I., Nolla, K.M., Eagly, A.H. and Uttal, D.H. (2018), The Development of Children's Gender-Science Stereotypes: A Meta-analysis of 5 Decades of U.S. Draw-A-Scientist Studies. Child Dev, 89, 1943-1955. [Link]

• Theron, L., Mitchell, C., Smith, A., & Stuart, J. (2011). Picturing research: Drawing as visual methodology. Rotterdam: SensePublishers. [Link]

• TRACEY Drawing and Visualisation Research Journal. [Link]

• Warr, D. Guillemin, M., Cox, S. & Waycott, J. (Eds.) (2016). Ethics and Visual Research Methods: Theory, Methodology, and Practice. Palgrave Macmillan. [Link]

• Wujec, T. (2022). Draw Toast.

If you know of more creative evaluation work, please email us with more information! We're learning alongside you and happy to expand our library. You can email your ideas for resources to publications@andimplementation.ca

Subscribe to our blog to have new content delivered right to you!

#evaluation #programevaluation #methodology #creativity #drawing #artsbased #datacollection #creativepractice #creativefacilitation #research #researchmethod #artsbasedmethod #learning #nonprofit #sharingfindings