Social VR Research - Collaboration Study Curriculum

Learning Science Paper | Usability Paper | Usability Study | Logistics | Curriculum

Week 1 - Lost at Sea

Goal: Initial exposure to the group to the kinds of activities we will be conducting in groups.

Activity: Lost at sea is a free activity conducted via paper handouts. It involves players rating a number of items as per their usefulness in survival situations. This initial rating is then discussed among the entire group to come up with a group rating. In most cases the group rating is higher than individual rating, exposing participants to the idea that group work will yield better results.

Measure of success:

  • Every member actively participated

  • Participants understood that working in groups can yield better results

  • Participants responded generally positively to the activity

Alternatives: Any “ice breaking” activities that do are depersonalized. Not requiring participants to share personal details on the first week allows them to get to know each other better without forcing it.

Debrief: As far the activity itself, I noticed that the task involving math (number 3 in the list above) grinds the group to stop. This is especially problematic since the “math” task is completely individual and it comes between Group ranking and Final score, both of which are the major opportunities for collaboration in this activity. In the future, we should probably stay away from any activity that breaks flow of collaboration and discussion as it simply defeats the purpose of this intervention.

Groups of 4 appear to work well. We have only had one group of 3. They seem to have been a little faster in breaking the ice. I think there was just fewer lanes of communication and less room to hide. This is based on observation from only one group. Designated Organizer is a role assigned to one participant in each group. The DO was placed in charge of reading the instructions, and keeping the group on task. I think it’s an effective way of encouraging collaboration and leadership, while removing the observer from the activity itself. Short of one group, I didn’t have trouble finding a volunteer to serve as the DO. I plan on rotating the role of DO weekly. Distribution of tasks might be a good strategy for organizing the VR groups.

Week 2 - Study Habits

Goal: Encourage participants to share relevant experiences in the academic domain.

Activity: Participants answer a list of questions regarding their study habits and preferences. The designated organized lead a discussion based on the script provided afterwards. Following script was used (for individual section):

  • What is your favorite class so far?

  • What is your least favorite class so far?

  • What is the class you think will require most work?

  • How are you preparing/studying for your favorite class? (Reading, reviewing notes, study group, programming exercises, etc)

  • How are you preparing/studying for your least favorite class? (Reading, reviewing notes, study group, programming exercises, etc)

Designed organized used the following script: 

  • Was the favorite class chosen by more than 1 student?

    • Did everyone pick their favorite class from their major requirements?

      • Why? Why not?

    • What makes your favorite class your favorite?

      • Within major, related to interests, flexible schedule?

      • Did you expect this class to be your favorite?

  • With this in mind, use the same questions to talk about your least favorite.

  • Think about the differences in learning approaches you use to study and prepare. 

    • Are there techniques that others in your group use that you haven’t thought about?

    • Are there techniques that you think you might try yourself?

    • Do you think study groups are effective for the classes?

      • Which classes are they good for?

      • Which classes do you think they wouldn’t work for?

      • Are you sure?

      • If you are not in a study group, why not?

      • If you are in a study group, how did you find it?

Measure of success:

  • Apparent comfort of the group

  • Attitude towards the activity at hand

  • Willingness to collaborate beyond the activity at hand

Alternatives: Any other set of questions that lead to discussion on differences in preferences or habits in the academic domain. 

Debrief:  The results from this week were satisfactory. Out of the 6 groups schedules, 5 groups ran. Group W3 was cancelled since only 2 participants showed up, with one of the absent participants requesting withdrawal from the study via email. 

Most groups (4 out 5) stuck to the provided discussion guidelines pretty closely. Only one of the designated organizers took effort to divert from the handout. One group was frequently getting slightly off topic but in a positive manner. 

3 out of 5 groups appeared very comfortable with each other and the assignment at hand. This was also a case in the T3 group which ran as a group of 4 for the first time. The new member of the group was immediately accepted. One of the groups performed very poorly for the second week in a row. One of the members inquired about the length of study and the requirement for participation. We should carefully observe this group. 

Student feedback on the activity itself was fairly consistent. Only one participant explicitly stated that she liked this weeks activity over the “Lost at Sea” scenario from week 1. The reminder of participant were split fairly evenly on having no preference or preferring the most problem-solving oriented. 

Week 3 - Ship of Theseus

Goal: Facilitate discussion of an abstract problem
Activity: Participants discuss a thought experiment questioning an identify of an object.  In this case we had the participants discuss the Ship of Theseus thought experiment. 

Measure of success: Participants come to some conclusions even if they do not reach any consensus. Ideally, participants fully explore the topic at hand and “agree to disagree.”

Alternatives: “Is a hot dog a sandwich?”

Debrief: The results were satisfactory but mixed. Only 5 groups participated and one of those groups didn’t participate in a week 2 activity. Success for this week is defined by engagement from participants and feedback they provided afterwards. Out of the 5 groups all were successful, obviously some were more engaged then other. 

One group required the backup activity. This highlights the need for having additional content at hand to best utilize the time available. 

One of our Thursday groups requires additional commentary. 2 participants were missing but instead of cancelling the group, I served as a participant myself and conducted the backup activity. I do not see a significant issue with doing this from time to time, provided that the observer acts only as a “passive” participant and only pushes the group forward instead of taking over the lead.

Week 4 - “Is a hot dog a sandwich?”

Goal: Facilitate discussion of an abstract problem

Activity: (Same as Ship of Theseus) Participants discuss a thought experiment questioning an identify of an object.  

Measure of success: (Same as Ship of Theseus) Participants come to some conclusions even if they do not reach any consensus. Ideally, participants fully explore the topic at hand and “agree to disagree.”

Alternatives: Ship of Theseus

Debrief: Results were comparable to last week. There were some individual members that liked the last weeks activity better but in general, group either prefered this week over last week or reported them as being comparable. Some individual quotes from week 4 feedback:

“Scenario was less confusing, and more engaging.” (refers to this week)

“Not as philosophical.”

“More ethical problems would be fun”

Week 5 - Problem Solving Activities

Goal: Facilitate solving a series of logic puzzles with defined solutions with focus on role / task division. 

Activity: Participants get a list of 4 short puzzles that usually have one solution each. They can often be solved with trial and error by keeping track of individual steps and backtracking them. 

Measure of success: Participants solved all problems in the time allotted. 

Alternatives: Any set of problems that can be accomplished in under 5 minutes each and have a concrete solution or “right” answer. 

Debrief: The results were satisfactory but the feedback we got was very scattered. The participants were split within their groups as to which activities they prefer. This activity highlighted the difference in preferences both within groups and between them. One group reached a consensus that they prefered SoT and HD problems better. That group was able to solve all the problems in 20 minutes (all other groups needed about 30 minutes).

Week 6 - Jenga

Goal: Social game that serves as a morale boost during periods of increased stress and anxiety affecting all members of the group. In our case, mid term exams. 

Activity: A short game of Jenga with house rules encouraged. Observed will adjust rules to competitive / collaborative depending on group preferences and perception of flow. 

Measure of success: 

Every member participates

Morale remains high

Alternatives: UNO, or any other party/social game

Debrief: The group that is currently a week ahead played Jenga in 2 different formats. In the traditional, competitive jenga, the group was somewhat disengaged and quiet. When challenged to work collaboratively the shift in behaviour was dramatic and occurred over a short time. Without being explicitly prompted the group organically explored new strategies and distributed tasks based on perceived competency. Based on the results, collaborative Jenga will be the primary activity for groups in the following week. 

The idea behind this activity was giving participants a gamified experience that can be done collaboratively. This comes following Eric’s push for activities that are more physical and that include object manipulation. Those activities tend to work well in VR and I am trying to maintain some sort of unity between what we are doing in Physical and VR groups.

Week 7 - Moon Landing

Goal: Recognizing progress already made as a group. 

Activity: This activity comes from the same source as week 1 (Lost at Sea). Since first week serves as an icebreaker and the content of the activity can serves largely secondary roles, this reiterates the general trend for groups to outperform individual scores. 

Measure of success:

The group recognized that working collaboratively had better overall results. 

The group recognized that they have made progress since the first meeting. 

Alternatives: Lost at Sea

Debrief: Primary activity for the week was moon landing. This activity is identical in structure to the “Lost at Sea” scenario. This activity was received with mixed results. Groups that liked the activity for week 1 (lost at sea) had similar attitudes toward this one. With that in mind, the activity went much smoother as far as logistics. It was familiar to almost all participants, with less time spend in individual assessment and more groups jumping quickly into the “core,” group ranking part. 

One of the main issues was the dubious, pseudo science presented as “expert” ranking. There was a number of instances were students thought the activity was unfair, or “stupid.” Some pointed out that the more abstract problem solving than lost at sea was more fun. It future activities of this type it would be sensible to either make the activity grounded in reality, with real solutions or make them abstract and openly present them as such. The former would cater to groups that like real life problem solving, the latter would be well suited as a lighter, more casual activity. The middle of the road approach fell apart in this particular activity. 

Additionally, the format of this activity could be a good starting framework for different content with slightly different goals. Groups differ in their preference of concrete problem solving or open ended discussion. Depending on the content this activity could be adjusted to fit either of those goals but not both. The structure is easy to learn and well retained in the participants. 

Week 8 - Pictionary

Alternatives: Charades or any other social “party” game. 

Debrief: This activity was selected based on it’s success in VR groups. This activity is based on a generic, social version and NOT the board game by Mattel. Initially it was planned as a competitive activity with groups being split into 2 teams. The result of that setup was very poor with no flow and half the group sitting idle at any given time. The “free-for-all” setup, were participants compete against each other and take turns worked much better. 

The activity was light and easy to understand, with most of the participants familiar with it. This included some international students. Surprisingly, even the more shy or reserved participants having a good attitude towards it. I would initially attribute it as a result of having designated turn order which ensures that all participants get a designated time to play. This is not the case with a open discussion were participants speak and interact at will. 

Groups that generally like games were very responsive to this week’s activity. Despite getting positive feedback, some groups still stick to their preference for problem solving. 

Upon closer observation, there are some signs suggesting that feedback from groups can be inaccurate with some participants simply half-heartedly agreeing with whatever was already stated. I think individual interviews might shed some light on actual preferences in activities.