Social VR Research - Collaboration Study Logistics

Learning Science Paper | Usability Paper | Usability Study | Logistics | Curriculum

Learning communities are groups of people who meet semi-regularly over shared academic goals. These groups are documented to provide positive impact on the participants’ academic performance. VR has the potential to expand existing learning communities, and create new ones where not previously possible. This research will lead to a better understanding of the strengths and limitations of VR technologies in higher education learning communities. Presenting institutions with data about virtualization technologies that have their impact and benefits well-researched will ease the reservations that administrators and other stakeholders might have.

The central question of this research project focuses on examining whether incoming undergraduate STEM students using collaborative VR in their Freshman Year Seminar Learning Community gain the same skills and benefits for supporting and improving their academic performance as their counterparts in traditional collaborative environments do. We also seek to improve the social connectiveness of underrepresented STEM minority groups more prone to isolation factors such as commuting, working, adult, veteran, female, and ethnic students.

Curriculum was designed for small groups of students to meet once a week for 45 minutes of collaboration training. Groups were set at four students as this was the maximum size of a session in Facebook Spaces. Size of four students was also found to be optimal as it was common for one student from the group to be missing, and groups of two did not work well while groups larger than 4 did not work well either. Activities were selected to maximize the chance that a group would form that group collaboration identity.

Logistics

The study had two groups: Physical group and VR group. During the initial stages of the study, students came for assistance and training. They explored more when they were left with VR sets to play with and was done mostly after the session. Academic and social sheets were filled accordingly. VR group students had to remove jacket and extra clothing, wipe their faces and lens which would help them to coordinate with figured avatar easily. The leader of the group was as to host everybody.. They were asked to go the software and one person would host. For the first session of the group, we taught them how to use the basic VR tools such as using pencil or moving around with gestures, etc.. For VR activities, we used Facebook as the platform.The training days were separately allocated for each group. The activities were ambiguous initially, so new activities were invented as the study proceeded. For every week there were different activities set up. A general calendar should have been made while they were having midterms they were not happy doing activities and we didn’t want them to feel more uncountable, so we end up putting easy activities to them.

Group logistics

There were some groups which were expecting to be democratic but became the usual hierarchical. These groups didn’t run well below 3 members and 4 were ideal and more than that it became too chaotic and the training became ineffective. As time passed they started being formal with mentor and informals with peers.They were a little bit more comfortable with kind of participating in the study and being observed because we know that just being observed and part of the study has an effect. They kind of got over that idea that, you know this is a study and I have to act or behave in a certain way. They talk about, you know, at first, “I was kind of nervous because it’s a study and I don’t know these people. How am I supposed to act?”; This is this early formative process of figuring out those core dynamics for those that are people. And so we’re trying to strike this balance between informal and casual and friendly and open atmosphere. And at the same time there needs to be a structure. We didn’t jump if they didn’t like the activities but gave more time to students and some liked the activities later. During these activities, the students had to be guided in the right direction when the discussion varied way off topic. It would have been more damaging to let be on the wrong track rather its important to guide them. Each group reacted differently to the various activities. Some responded very well but some didn’t. There were times with some toxic elements between the group. Each group had its own strengths and hence, different group gravitated towards different activities. To help students open up, party game is what groups liked and started showing their personalities. First year students are tough to train but later it becomes cycle, meeting every week, filling the social and academic forms etc. As they get comfortable with the environment, the flow of doing the activities becomes smooth.

Focusing on logistics was essential as these decisions would either make or break this kind of study. During the study, the idea of data collection came from the type of data we collected. Maturity plays a really important role in understanding and processing the environment.

Mentor Training

The most important motto for the mentor is to be flexible and light with the students. Mentors have to understand how students are reacting to particular activities and even for the past activities they reacted. They have to judge students from their mood. They are responsible for jumping in when the group is going the wrong way. If the students are not able to solve the problem, the mentor does not have to make it a big deal rather they should just focus on how much have they done.

The mentors started evolving, solving technical and logistic issues. They helped the participants only when they were totally wrong. The mentor has described a few longer, uncomfortable silences. Discussion is difficult to get, the normal rhythm eases back to a creep. In extraordinary cases all collaborative activity stops, and intervention is required. This end case ought to be rendered. It's simpler to guide the gathering when the session begins getting ugly. With restricted time recuperation is tedious and regularly not practical. The activities were tailored according to the students and yet there were three possibilities i.e. they love it, they are fine they will do or they hate it.

Obstacles & Solutions

For every week there were new activities so the students don’t feel restless and stay motivated. Mentors were able to understand students on the based of their reactions or even if they liked the activity or no. It is more damaging to let be on the wrong track rather it is important to guide them. Sometimes re-skin of the activity was done to bring them on the track. Some group reacted very well but some didn’t, a lot of times there were very toxic elements between the group. Participants were coming late and sometimes even missing without informing in advance. Reminders were sent in advance with a picture which they would have uploaded before they come in. Emails were going back and forth. Attendance, lateness was the biggest problem, which was faced daily.

Group Mentoring Guidelines

Observing Active Engagement

To adjust activities you need to get a decent measure of how active your participants are. A short survey between sessions is an intuitive option but it has problems. First, unless you collect data digitally (like via google forms or similar), processing survey data will eat up valuable time. Compliance of participants with an additional weekly task, especially one that from their perspective is likely uninteresting is also a problem. Second, the data in a survey is self-reported so it is subject to a number of biases, such as participants answering what they think will make organizers and facilitators happy, or not being able to self-reflect on their attitude towards the activity. Solution to this is collecting data via direct observation. This had a major disadvantage of requiring the facilitator / mentor to spend the entire session observing, eliminating the option to run multiple sessions concurrently.

Main purpose for this observation is to ensure everyone is engaged and actively participating. One of the issues in groups is a tendency for some participants to disconnect. In groups of 4 or more it is possible for a participant to “check out” without visible affecting the performance of the group. This means that unless the facilitators are actively looking for these signs, this behavior can go unnoticed and therefore uncorrected for an extended length of time.

Good Collaboration Sign - All participant are actively engaged

No participant “checks out.” With a group of 4 (our recommended size) it’s common for participants to respond differently to various activities. Of course you are not aiming for full excitement, all the time. You want your participants to stay on track.

Good Collaboration Sign - Energy and Flow

There are not many pauses, the ones that occur are not overly long. Group continuously flows without needs to be prompted or pushed. If there is a participant that sometimes assumes the role of the leader he does not intervene to keep the group moving.

Poor Collaboration Sign - Activity vs Group Struggling Issues

Differentiate struggling with the activity content and struggling as a collaborative group. The goal of the collaborative sessions is not to finish tasks but to develop collaborative skills and good habits for group activity.

Poor Collaboration Sign - Participants are disengaged

Activities will interest different participants to different degrees, and participants that are somewhat unengaged during some assignments is not necessarily a long term problem. Look for signs that participants that are actively disengaged. This is a potential problem during observations when a single participants disengages. In this scenario a participant “slips thru the cracks” and remains quiet during most of the session. Unless activity effort is undertaken by either the facilitators or the in-group leader, this participant will simply allow to be omitted / ignored for long periods of time.

There is a number of possible causes of this scenario. One is that the participants is naturally reserved or shy and doesn’t feel comfortable speaking in the group, or rather is more comfortable staying silent (disconnected) than my actively participating. Another is that the activity failed to stimulate

Poor Collaboration Sign - No flow

Characterized by longer, awkward pauses. Conversation is hard to pick up, the natural tempo slows to a crawl. In extreme cases all collaborative activity ceases, and intervention is required. This end case should be avoided. It’s easier to steer the group when the session starts taking a turn for the worse. With limited time recovery is time consuming and often not feasible.

Poor Collaboration Sign - Uncivil exchanges between participant

Short, snippy exchanges between participants. Often one sides, with one participant shutting down someone else. Characterized by an unconstructive criticism, often a personal attack. This is a problem that grow to be a significant if uncorrected. It increases a certain toxicity in the group. When those exchanged are targeted towards already sensitive participants, it may cause them to become even less actively engaged in the future. They can be differentiated from playful banter by the hostile or generally unpleasant tone, when they include an expletive.

Misleading Collaboration Sign - Getting off topic (as a group)

Since the objective of the session is not accomplishing the goals of the activity but rather fostering a positive collaborative environment, getting off topic of the activity at hand is not a problem provided that:

The group remains cohesive as in the entire group remains engaged in the off topic activity/ conversation.
An activity / discussion that the group engaged in instead of the provided activity is appropriate for an academic environment.

Activity Notes

Depersonalization of content

Keeping in mind that participants are going likely going to be somewhat uncomfortable, depersonalization of content allows to limit some of the potential resistance that participants might show to the content. This is especially important during the first few sessions where participants are still getting accustomed to their groups and fellow participants.

Depersonalization of content depends on removing any need for participants to share their feelings, thought or opinions on subjects that could be considered personal. Instead activities should focus on puzzles, problem-solving and discussion based problem solving.

Things to avoid

Complex rules and instructions

With the sessions being around 30-45 minutes long it not very efficient to use activities that require long preparation. Ideally, you want participants to start the activity within 5 minutes of starting the session. Any organizational notes should be kept to a minimum. If the group gets off topic at the very beginning of the session it’s going to be an uphill battle to get them “rallied” to get the activity accomplished within the limited session time.

This limitation will be less of an issue if the sessions are longer. Additionally, repeating activities like board games need to have their rules explained only once, so investing times early on would be a good return throughout the seminar.

Identifying group preferences

It is not likely that any activity will fully satisfy needs for all participants, across all groups. Whether or not it is feasible to generate a large pool of activities to select from, it is necessary to identify what are overall preferences of groups. Activities can be classified by a number of qualities. Based on our observations we classify activities on a 2-axis scale of their relative abstract/concrete solution and their tone playful/serious. Include note on depersonalizing activities.

Group vs Individual preferences

While every participant has their individual preferences it is more important to recognize that participants in their group will form a specific identity. Using average group preference is an easier way to adjusting activities.

Activity Classification - Objective (concrete, abstract, reaching a consensus)

The objective is one of the ways we can distinguish different activities and adjust activities to the preferences of the participants.

Activity Classification - Action (discussion, problem modelings, game)

This describes the tasks that participants will take to either accomplish the objective or progress thru the activity until the time allotted runs out.

Activity Classification - Tone (serious, philosophical, light hearted)

Tone plays a smaller role in differentiating activities. The differences in tone rely on identifying personality characteristic of the participants that are less obvious and more nebulous.

Types of Activities - Discussion-based, Problem-solving, Games, Hybrid

Categories we used for classifying around activities are not meant to be rigid. This classification is just a tool for better aligning the session activities to the preferences of the participants.