How to write better discussions for your HCI study

10 tips for CHI authors

A bad discussion section is my most common point of criticism across the papers I have reviewed as a CHI AC in recent years. Here is some practical advice on how to fix that and write better discussions.

Often, my impression is that people spend their time and energy on a great study but then write a discussion section that is too short and shallow — and mostly a summary anyway. This makes papers weak and rejection likely: If your submission is a “data report” without meaning, it can be difficult for the community to see the value of your hard work. So how can we do better?

Here are 10 ideas, plus concrete prompts at the end:

1. Writing a good discussion takes… discussions
2. Discussions need time and iterations, too
3. Structure your discussion with subsections
4. A discussion is not a summary
5. What does it mean?
6. Return to your related work
7. Relate on a higher and broader level
8. Consider multiple points of view
9. Dare to be critical
10. Be candid about limitations and decisions
11. Concrete prompts for your next discussion section

1. Writing a good discussion takes… discussions

Actually discuss your results before writing the discussion section. Like, verbally. At least with your co-authors but ideally also with people that have not been involved in the project so far: Other colleagues, students, members of your lab and other groups, and if possible people from other disciplines.

Summarise your work and results in your own words. Maybe as a short meeting over coffee. Note what kinds of immediate questions you get, both critical and curious ones. If you think it’s tricky to get people to spend time on that, especially if they’re busy writing papers themselves: It’s a great idea to offer to return the favour for their discussion.

As authors of a study, we might focus on the most obvious interpretation of the results with respect to our initial research question, and miss further interesting, insightful, or surprising aspects and perspectives.

2. Discussions need time and iterations, too

Don’t write the discussion that same night you finally completed your data analysis, and look at that, why is it September already, I wanted to start earlier this year, oh, I need another coffee.

Ideally, you have time to draft a discussion, get feedback from others, and revise and iterate it. The same is useful for any part of a paper, of course.

Related, great discussions take space: Half a page likely won’t do it. In a typical ten pager for CHI (two columns), as a rule of thumb, I suggest to consider 1.5–2 pages.

3. Structure your discussion with subsections

Don’t miss out on the subsection level of the template in your discussion. Use one subsection per discussion aspect. It’s more readable and you can use the subsections to keep track of ideas for discussion points while drafting.

This can include subsections on limitations and reflections on your methodology, if you do not have separate sections for those.

Bonus tip, but this is personal preference: In the discussion section, I really like subsection titles that “speak”, i.e. state a finding or interpretation instead of just mentioning what the subsection is about. For example, a subsection such as “Impact of user expertise” would be more revealing if worded as “Experienced users benefit less from [feature X]”. Don’t fear line breaks for titles, the template can cope just fine. What this achieves is that readers see the main findings and thoughts already when skimming through the discussion.

4. A discussion is not a summary

Many discussion sections are largely summaries. A short recap is great and helpful for readers in many cases — with an emphasis on short.

If you want to include a recap, my recommendation is to write a specific subsection for it, probably the first one. Here, you give a short recap of the results and directly state your main interpretations (which you could then elaborate on in other subsections). Even if your study is complex, consider a recap of just 2–4 sentences. If you need more, consider a higher level of abstraction. Naturally, there might be cases where the results are so nuanced that a faithful and understandable summary really does need more space.

Related, don’t introduce entirely new results in the discussion section that have not been already reported in your results section. Splitting results and discussion makes it clear what’s (empirical) result, and what’s your interpretation.

If you catch yourself writing summaries again and again while drafting your discussion, ask yourself…

5. What does it mean?

So A is different than B regarding C, and p<.05? Great, but what does it mean? For users, your research, other researchers and practitioners in this area, and beyond? For the HCI community? For humanity?

The discussion should say something about that, possibly for each key finding. Finding insightful answers here may take some time and external input (see points 1 and 2).

6. Return to your related work

To support the previous point, you can return to your related work: For example, if you would meet those authors whose work had inspired yours, what would you tell them — and why? Why would you be excited to tell them that? Why would you hope that they would be excited to hear it?

A great discussion connects your findings and interpretations with the literature, in detail.

7. Relate on a higher and broader level

If you’ve followed the previous point, now repeat on a higher level, beyond the closest related study. For instance, how does your work speak to larger themes, as found e.g. in calls for papers, workshop or panel themes, future work sections, policy initiatives, social impact considerations, discussions at conferences, etc.

If you could have a conversation with “the community” or “the society”, what would you say about your findings? How do your findings address (broader) questions, or add new ones, or change priorities, potentially? This can be somewhat speculative as long as you clearly mark it as such (e.g. “We envision…”).

8. Consider multiple points of view

Consider multiple points of view. This is possible on all levels, from concrete results to the broader impact.

For example, many discussions of concrete results seem to jump to the desired conclusion, fast. This might be too shallow and unconvincing. Walk the reader through alternatives, building an argument of how/why your results allow us to see the “true path”. If you have a mixed-methods study, this can also be a good place to compare/combine quantitative and qualitative results.

Maybe several alternative interpretations or explanations remain. That’s fine, too. Just don’t end with open questions without giving the community your best advice: After all that you’ve learned from this study — what would you recommend should we do next to clarify this? Be specific here.

9. Dare to be critical

From working with people across disciplines, I’ve gained the impression that the HCI community can be rather cautious when it comes to directly responding to other work, e.g. to discuss conflicting results. I think it is important to clearly point out disagreements or contradictions with related work, along with possible explanations and suggestions on how to clarify this in future research.

10. Be candid about limitations and decisions

Take some time and space to reflect on your own work’s limitations and in particular on your methodological decisions. Even a well-designed study could have been different, as reviewers like to point out. You don’t need to strike a defensive tone here, rather consider a constructive one: You had to make a choice for your study and now you’ve learned something. Let us know: To what extent was it a good choice? Why?

11. Concrete prompts for your next discussion section

To conclude, here are a few concrete ideas for the subsections of your next discussion — adjust as needed, depending on your research topic:

• Discuss the impact of each independent variable or effect in one subsection. Take this space to relate it to other studies on that variable or effect in detail.
• Discuss two ways of interpreting a certain result. Do you favour one explanation? Build a detailed argument why.
• Discuss contrasting/complementary findings for qualitative and quantitative data, if you have both.
• Discuss your study design choices. You likely motivated them earlier in the paper but here you can reflect on them again, now that you’ve done the work and seen the results. Would you recommend your method to others?
• Discuss limitations, e.g. regarding generalisation. Typical aspects include sample diversity and size, study duration, novelty effects, shortcuts you took in your prototype, and (other) aspects of internal vs external validity.
• Discuss broader impact (e.g. social, privacy, biases, etc.).
• Was there one paper in particular that had inspired you to do this work? Maybe relating your findings to that paper, and others who have cited it, is worth a discussion subsection of its own?
• A colleague had a critical or curious question about your work? Maybe addressing this would be interesting to many and is worth a subsection?

Thank you to Florian Lehmann for discussions about discussions, and sharing his ideas for this article!