Developer Blog

Overview | Dev Blog | Design Doc | Code


First User Testing of Prototype - 6/8/19

Today, at Science Park High School in Newark, the team was able to watch several groups of middle schoolers play in the Oculus Quest app. The team who attended this user study consisted of two researchers, a developer, and a designer. Middle school teachers also came to watch the study as their students walked around the gymnasium in the new Oculus Quest headset. Students were first, given a pretest on the subject matter before going into the experience, followed by a post test and interview after participation in the experience. Each group consisted of 5 students who were placed within a 20 x 20 ft outlined boundary space, marked with blue masking tape. Students were then instructed to begin the experience, which would be timed for approximately 15 minutes.

NJIT team members pose in a group shot with the wonderful user test participants from Science Park High School

NJIT team members pose in a group shot with the wonderful user test participants from Science Park High School

The outcome of this user study was highly rewarding with a few students opting to play a second time. The overall impression received from observing the students was that the experience,even in it’s rough state, was engaging and fun. Students made full use of the Quest’s un-tethered feature, breaking out into victory dances and freely moving about in their 20 by 20 foot space. Other than observations and brief verbal feedback between the supervising designers, students took a post test and were interviewed about the experience. The team hopes to learn much more about the feedback from these interviews and plans on moving forward with development based on this feedback.


Locks and Levers- 6/6/19

After a brief feedback session with team researchers, more alterations had to be made. The levers had previously been designed to indicate a locked and unlocked state, by using the colors red and green (red for locked, green for unlocked). This concept was not understood by the team researchers as the concept of something being locked in most digital platforms is typically indicated with a grayed out button or a lock icon. The red and green colors on the lever knobs were confusing due to these facts and after much iteration, the team concluded to use dark gray for locked, as well as providing a lock icon at the base of the lever, for reinforcement.

Other changes were made to the design of the station, such as the color key on the right hand side of the kiosk. The design appeared too busy with the color key adding to this clutter. We rearranged the color swatches so that they are organized into columns of 2 and split into two rows across, thus eliminating the stark contrast in the strip of colors as in the last version. These changes all made the rest of the experience more clear to understand.  


Station Re-designs- 5/25/19

Alterations were made to the number, color, and shape stations. After evaluating the functions of these stations, it seemed that only the number station related to binary math principles, so the color and shape station were changed to have activities related to binary math learning.  The color station was originally designed to have 3 dials, each underneath a corresponding color of RGB. The objective was for the user to find the number value of a certain color, such as aqua marine, however the objective has been redesigned so the user must solve a binary math equation to get their desired color. Eight color swatches are provided, each labeled with their binary representation. This information is on the station itself, in the form of a manual which the user can refer to when completing the activities.

The shape station’s design and objectives was never fully developed, but the team created a more relevant function for it, as it now relates to binary math.


Visual Development- 5/24/19

Visual development for the application began in the hopes of getting the prototype somewhat polished for the June 8th deadline. The look and feel of the experience was decided upon after researching various programming games and VR experiences designed for a wide age group. A few consistencies found within these different games were the bright, playful color palettes, simplistic, almost low poly designs, and cheery atmosphere. After researching the demographics of users for these various titles, it was concluded that these visual elements appealed to both male and female users, as well as diverse age groups. It was important to find an aesthetic that can appeal to a large group of users since one of the objectives of the experience is to attract users of diverse academic backgrounds and skill levels. The application should encourage those who have not considered programming or computer science, to participate so that they may expose themselves to knowledge which they may not under typical academic circumstances.

Building out the Station-5/23/19

More of the stations were modeled. Changed had to be made to the stations with buttons because they were not registering with the hand controllers on the Quest. Due to this, a considerable amount of time went into redesigning the output station and mission station, as the buttons had to be changed into levers. All interactables are levers, most with dials that move as the levers are being moved.


From Sketches to 3D Models- 5/21/19

After our ideation session on the environment, design, and UI development, Eric was able to get a rough model in unity using primitives, based on my sketches. We have made the space smaller, so the user will be in one large room, whereas before, each station was designed to be in its own space.

To begin the design process on the current primitive models, I will be bringing the models into Quill and Medium where I can do rough modeling over the base shapes.


Navigation- 5/20/19

Storyboard by Jessica Ross Nersesian

Storyboard by Jessica Ross Nersesian

The team is currently discussing the options for navigation within the experience. Due to time constraints, the option of having a character guide to bring the user from station to station, is not feasible. This leaves only a few other options which include voice over prompts, text panels or signs, and adding constraints to what the user may or may not be able to do, to prevent him or her from getting distracted and wandering off course. Other possibilities included visual cues such as blinking lights, which may appear at the feet of the user when the experience begins and then projects outward in a dotted path to direct the user where to walk.

Another possibility is to include a station dedicated to help and guidance.This station, (the Help Station), will have elevation controls (up or down), to take the user from one level of the station to another. These features would include an interactive map of the entire play area and other information to help guide the user through the experience such as clues, guides on how to use the different stations, and a guide to binary numbers just in case the user does not understand the number station activity. After working out the current timeline until the June 8th deadline, this idea was archived along with the character guide.

For the time being, the team has decided to stick to either voice overs or text boxes, which are much quicker to implement. These more elaborate options will be revisited after the June 8th deadline.


Ergonomics- 5/16/19

Discussion on the height of the stations in relationship to potential height of the users. Team decided that it would be best to utilize the calibration system within the headset to determine the height per user.


Ergonomics and Scaffolding- 5/9/19

The grabbing motion of the levers need work because the hand goes through the lever and does not properly wrap around when grabbing it. This might be fixed by finding a way to knock it into place with something or telling the user to hit it up or down.

Today was dedicated to developing the educational scaffolding and understanding the ergonomics.

Focusing on these topics revealed a few issues with the team. First, there was a concern that the middle schoolers (our test subjects on June 8th), may not be VR trained and so this would add more work on our part to be sure that the tutorial has some basic VR training. This requires more instruction and prompting within the introduction of the experience and may lead to more confusion for the user if not properly implemented.


Initial Prototyping and First Round of Alterations- 5/8/19

Changes were made to the initial 3D prototype of the stations after the team evaluated current progress on these designs. One of these alterations was the name plate, positioned on top of the station. This nameplate added extra clutter to the kiosk’s interface and if the station is designed well, there would not be a need for a label of what that station is. If after the station is designed and the team feels that reinforcement information is needed for the user, then the written name plate could be supplemented with icons, which offer a quicker way to deliver information and don’t add much clutter.

Each station’s UI was carefully observed in 3D space, then altered accordingly. The number station proved to be the most time consuming, as there are multiple functions and interactables needed to complete the task.

The design team went back to wireframing to brainstorm the best way to organize this information. This was a long, yet rewarding process, resulting in a much improved wireframe of the number station. The design team first drew out the UI of the number station and realized the display screen may be too confusing for the user to read because it contains both the total and the equation.


Space and Ergonomics 5/7/19

UserPathDiagram01.jpg

The next topic focused on was the matter of space and ergonomics. In the prototype, the stations are built like small kiosks or game consoles, about 6 ft tall. There are currently 6 stations all together and are arranged within a circle, to indicate the flow of interaction for the user.

After seeing the interior design of the station in 3D space, the design team felt that the space was too small and cluttered. To solve this issue, the design team altered the original floor plan of this space so that there was more room between stations for the user to walk through. The 3D model was updated to fit the new design, being careful to leave enough room for the transparent domes which are meant to be over each kiosk, making 5 independent small stations placed around the one main Mission Station.

Due to the time constraints, this idea was cut so that all 6 stations are within one dome, making it one whole station. The original layout design for this environment is being archived for the next phase of development.

The next challenge would be to design the interior environment to accommodate our target audience The issue with this target audience is the wide array of heights to consider, due to the varying growth spurts of puberty. Focusing on middle school children is a challenge for this reason, as it encapsulates the beginning of those pubescent years, which are typically 10-15 years old in boys and even earlier in girls, from 8-13 years old. This made the concept of predicting a common height to build around, very daunting because height is particularly hard to predict depending on when the individual’s growth spurt begins.

Currently, the team is spending time on some light research into what average height to build around depending on all of these factors. The objective is to build a space that is “ergonomically sound,” for as many users as possible. This means that the dashboard with the controls need to be placed at a level where the user does not need to bend down or tilt their head up. The width of the dashboard, UI, and display all need to be designed within the scope of the user’s vision and height. For example, If the kiosk is too small or too large it will not be comfortable for the user to interact with.



Began educational scaffolding 5/6/19

The team gathered to collectively agree on priority objectives. First, the main goal was to develop an effective CS education application in VR, using the Oculus Quest. The supporting objectives of this goal focused on ways to collect evidence that this VR application was successful. The parameters for measuring the success of the application would include whether or not the user learned CS principles from the experience and if the user enjoyed the experience.

After establishing these goals, the team was ready to determine the target audience and demographic for this application. STEM is being introduced more regularly in k-12 education, so that students are encouraged to pursue career opportunities in technology related fields. Middle school students seemed to be the best group to focus on for this project for a few reasons, the main reason being the age limit for use of Oculus gear, which is currently set to 13. Most VR applications are designed for users around this age.



Planning the Research 5/3/19

MicroInteractions.JPG

Today’s agenda included how to plan user studies for the CS application. The team discussed many different methods of user testing to consider such as usability studies and use case/heuristic studies. Both methods were feasible for the June deadline, so this would be the starting point for planning the research.

Ideally, the team researchers would have 2 weeks of analyzing the data from around 50-60. How to collect and evaluate this data was the next stage of planning. Methods agreed upon within the team included the following:

  • Surveys

  • Cognitive walkthroughs

  • Observation through ethnographics

  • Direct observation using direct video from the experience


Interaction Design 5/5/19

TaskStations.JPG
  • Macro Interaction Designs- From the perspective of the system

  • Middle Interaction Designs- From the perspective of the stations

  • Micro Interaction Designs- From the perspective of the individual elements (buttons, levers, ect)


Deliverable Roles, Priorities, and Schedules 5/2/19

Development on the UX design of the experience is taking more time than originally estimated, so the team had a meeting to discuss how to make the June deadline. The first plan of action was to limit the number of features and interactions of each station. Once more constraint was added to the design objectives, meeting the deadline was more feasible. The UX designers were able to begin on planning the micro interactions for each station:

Micro Interactions-Categorized by these properties:

  • Triggers

  • Cues

  • Behaviors

  • Actions

  • Modes

  • Mode selectors

  • Feedback

  • System

Once these micro interactions were sorted out, the designers decided to make the stations “single mode.”  For example, Instagram has the ability to browse, post, comment, tile images but in this experience we have not defined micro interactions in VR it has been a challenge because there is nothing to reference. To keep these processes and design within reasonable scope, the elements in the experience mono mode



4/30 Documenting for blog and team communication

Eric shared documents of past development projects so that we could use the same templates for our documentation for the Quest project. From this point forward, there will be a daily blog/journal posting of what was done on each day of production.

Adam tested Eric’s prototype in the Oculus Rift

We planned how to document all of the research and notes; we’ll have 2 documents- a developer journal and a prospectus mm



4/25 Flowcharts of entire run through of experience.

  • Team realized there would need to be a total of 12 steps in the beginning tutorial, just to get the user to the task station.

  • UX designers worked on standardizing all the actions; grabbing, pushing, placing

  • No possible failure points every action is prompted

  • Goal is to standardize all vocabulary when relaying info about the flow charts, mechanics, to prevent confusion

  • For these initial run throughs, we are only focusing on 3 stations; the OPS (output station), MS (mission station), #S (number station)

  • Established that there would need something in charge of prompting, guiding users as well as something to tell the user when they made a mistake. For right now this is called “The System,” until we can agree on what that will be (a character/host, or voice overs coming from each station)

  • Navigation possibilities were discussed to help guide the user, in addition to the other cues. Some possibilities were dotted lines that light up from the user’s position to the desired location, (much like how this is done in the Quest demo).

  • Wireframes were drafted of the 3 stations being focused on; OPS, MS, #S

  • The team designers thought of possible scenarios that would get in the way of the user successfully completing the game. To do this, they planned one such scenario and designed the solution to get the user back on track:

Scenario-User Gets Lost

We thought of what might happen if the user did not understand the flow of action. The resolution would be a timing function (estimated 30-60 seconds) where the user would be prompted if he/she did not get to the next station within a designated time frame.

RunThrough.JPG

Another possible problem point may be when the user arrives at the #S. The user needs to somehow see that the #S action is directly showing up on the OPS screen. We have not yet decided how to do this.



Binary Math Lessons 4/23/19

After the team researchers decided to choose binary math for the learning objective, the designers began whiteboarding possible interactions. First, they would have to figure out how to represent binary numbers in a 3D space. This concept would be represented in the experience as levers the user would pull up or down, to change a number from 1 to a 0 or vice versa.

The levers would have 2 positions so as to not distract the user from the 0 or 1 action. There would also need to be a display at this station which directly reflects the actions of the user.

BinaryMath.JPG

The experience was wireframed in a concentric circle pattern, where the user would start and finish at the same location. There would need to be an introduction to acclimate the user to their virtual environment and teach them what to do. The designers wireframed a user flow similar to an assembly line, where the user would move from one station to another station, to complete a task. These stations became central to the interaction design of the experience:

  • Output Station (OPS)- User would need to train for the OPS first

  • Color Station- RGB the user would be given a color to match with 3 dials. For example the color, aquamarine’s binary representation is (RGB value 66,244,238).

  • Task station- Where user goes to get a task.



Opening Discussion 4/1/19

The team began the discussion of possible approaches for the coding experience in VR. Questions were explored, such as, “What theme would be used, what lessons in CS to teach and at what level of difficulty?”

Various CS learning activities and gaming applications were researched and discussed. This gave the team a starting point to begin brainstorming. Some of the gaming applications included in the brainstorming session were Tynker, Alice, Scratch, and Kano. Most of these digital CS applications used something referred to block code by CS educators, where the user would for one line of code by dragging and placing colored blocks into a sequence to produce the desired outcome. The Harry Potter application from Kano was a perfect example of how to gamify the subject of CS for younger audiences, which uses this block code method. Users have a hand held wand which they physically move in formulated motion sequences, each with a different outcome.

The block code seemed to be the most favored and sensible method of CS learning to implement after trying out these other applications, but after more discussion the team realized that translating this technique into a virtual 3D space would not be effective. One of the main objectives for designing this application is to encourage the user to move freely and be active during the experience. More game mechanics and interactives would be drawn from this objective, so that the user could learn CS through spatial learning, within their immersive world.



First Team Meeting 4/3/19

Today marked the first official day of development on the education app for Oculus Quest. The objective of this project is to develop an educational game application about Computer Science, using the new Quest from Oculus. This project will include a small group of UX/UI and visual designers, CS instructors, HCI researchers, and student volunteers.

Using this new hardware presents many exciting opportunities, due to the completely untethered nature. There are absolutely no wires or cables attached to the head set or controllers and there is even a camera placed within the headset, so there is no need to set up outside sensors. This offers the user the freedom to roam and move more freely around their physical and virtual space, lending yet another level of immersion for the user.

The timeline for this application is not entirely certain, aside from a potential user study on June 8th, with middle school children at a local STEM highschool, here in Newark. The team has began putting together basic parameters for what this application must include and how much of the overall experience should be completed by that time. Realistically, the end product for this deadline will be in a demo state, where the experience only includes the basic game mechanics and features. Setting these parameters of this expected level of completion makes this goal attainable and allows the team to better prioritize this short time period. Some of these first priorities include what CS concept to teach, how to teach it spatially, how to make full use of the Quest’s untethered functionality, and how to make the experience enjoyable, so that users are playing while learning.

The team began to kickoff the project with a brainstorming session, where members can begin exchanging their own ideas about possible genres, subjects, and interactions. Collaboration began to take shape once the researchers gave a few options of CS lessons to include in the experience. Some of the CS lessons included activities called, boolean algebra, building logic states, if statements, and logic puzzles.

The basic interactions and gameplay would need to be designed around these activities. One particular source of inspiration for CS activity lessons is an organization called CS Unplugged, a free collection of lessons and fun learning activities that teach both children and adults about CS. The materials are available from the website, which was put together by Computer Science Research Education Group, at the University of Canterbury in New Zealand. This source was an invaluable starting point for the team to begin building their experience, as the very nature of these activities is utilizing physical space and hands on learning without a computer.