×

Can AI Give Good Feedback?

Build One and Find Out

Amsterdam Tech Week Hackathon
June 17, 2026 — 18:00–21:00 — FeedbackFruits, Amsterdam

Your Facilitators

Mary-Jo Diepeveen

Mary-Jo Diepeveen

VU Amsterdam

Bas Hintemann

Bas Hintemann

FeedbackFruits

Tonight’s Agenda

  • 18:00–18:15 — Welcome & Introduction
  • 18:15–19:05 — Phase 1: Orientation & Elicitation
  • 19:05–19:50 — Phase 2: Agent Design
  • 19:50–20:15 — Phase 3: Cross-Testing
  • 20:15–20:40 — Phase 4: Iteration
  • 20:40–21:00 — Phase 5: Community Testing & Reflection

Can you build an AI agent that gives better feedback than your peers?

What You’ll Do

Experience

Write summaries and give feedback yourself before building AI

Build

Design an AI feedback agent using GitHub Copilot instructions

Iterate

Test, get peer feedback, improve, and discover what works

About This Research

This hackathon is part of a study on AI-mediated feedback design.

Your participation helps us understand what learners value in AI feedback systems.

Participation is voluntary. See CONSENT.md for details.
18:15 – 19:05

Phase 1: Orientation & Elicitation

Experience the feedback task before building the AI

Step 1: Read & Write (15 min)

1

Read the Article

Mars geology findings in assets/article.md

2

Write a Summary

50–100 words covering the main scientific findings

3

Post in Discussions

Share your summary as a comment in the pinned discussion

Open Discussions Read Article

Steps 2–3: React & Discuss (15 min)

Share Reactions

  • Refresh the discussion page
  • Read your peers’ summaries
  • React with thumbs-up to the strongest ones

Group Discussion

  • What made highly-rated summaries effective?
  • What specific features stood out?
  • How could other summaries improve?

The Universal Science Writing Rubric

Scientific Content

Accuracy and completeness of scientific information

Interpretation

Explaining significance and implications

Audience Targeting

Appropriate language and detail level

Organization

Logical flow and structure

Writing Quality

Grammar and mechanics

Open Full Rubric

Fork & Set Up (12 min)

Fork the Repository

Click Fork at the top of the repo — you’ll use peer summaries as test data for your agent

Fork Repository

You’ve now experienced what AI feedback agents need to do

Recognize quality • Explain why • Suggest improvements

19:05 – 19:50

Phase 2: Agent Design

Build your AI feedback agent using GitHub Copilot instructions

Key Design Decisions

Feedback Type

  • Corrections
  • Questions
  • Guidance
  • Mix of all three?

Rubric Usage

  • Score each dimension?
  • Focus on weakest?
  • Evaluate all five?

Tone & Style

  • Coach / Peer / Editor
  • Quote specific text?
  • All at once or step-by-step?

Create Your Instruction File (20 min)

1

Open Copilot Chat

Click + → Instructions → Configure → + New Instruction file

2

Name Your File

.github/instructions/group[X]-feedback-agent.md

3

Write Your Agent

Define role, process, rubric usage, feedback style, what to include/avoid

Phase 2 Guide

Test & Commit (15 min)

Test Your Agent

  • Paste your Phase 1 summary
  • Ask: “Give me feedback on this summary”
  • Is it helpful? Specific? Uses rubric?
  • Iterate until satisfied

Commit Changes

  • Navigate to your file
  • Click Commit changes
  • Message: “Add feedback agent v1”
  • Your agent is live!
19:50 – 20:15

Phase 3: Cross-Testing

Evaluate another team’s agent

5 Evaluation Criteria

Understandability

Could you comprehend the feedback?

Specificity

Concrete examples vs. vague statements?

Validity

Accurate and aligned with the rubric?

Actionability

Could you use it to actually improve?

Affective Quality

Did it motivate you to improve?

Submit Feedback via GitHub Issue

1

Go to the Main Repository

Not the team’s fork — use the main hackathon repo

2

Create a New Issue

Select the “Agent Feedback (Phase 3)” template

3

Rate & Provide Evidence

Rate all 5 criteria (1–5) with specific examples from the feedback

Submit Feedback Issue
20:15 – 20:40

Phase 4: Iteration

Improve based on feedback received

Common Issues & Fixes

Feedback Received Suggested Fix
“Too vague” Add instructions to quote specific phrases
“Didn’t use the rubric” Require evaluation of each dimension explicitly
“Too harsh” Start with positive observations first
“Not actionable” Require concrete revision suggestions with examples
“Too much at once” Prioritize the 1–2 most important issues first
Find Your Feedback
20:40 – 21:00

Phase 5: Community Testing & Reflection

Test multiple agents and share insights

Open Testing & Vote

Explore Freely (14 min)

  • Navigate between teams’ repositories
  • Test agents with YOUR summary
  • Note what works well and why

Vote for the Best (6 min)

  • Go to Discussions: “Which agent was most effective?”
  • Share which team’s agent you liked best
  • Explain why it was effective
Vote in Discussions

Group Debrief

What Worked?

  • Common patterns in top agents?
  • Specific features that stood out?
  • Any surprising approaches?

What Did You Learn?

  • What’s easy vs. hard for AI?
  • What do humans still do better?
  • How would you design differently now?

What You Accomplished

  • Experienced the challenge of giving good feedback
  • Learned a research-based rubric for science writing
  • Designed an AI agent with specific behavioral instructions
  • Evaluated others’ agents using feedback research dimensions
  • Iterated based on user feedback
  • Explored what makes AI feedback effective
Your designs help us understand what learners value in AI-powered feedback systems.
×

Thank You!

Keep experimenting — this is a powerful way to customize AI behavior