Playtest Summary

The survey indicated that people enjoyed exploring the colonies in the Main View. The art style was also noted as one of the better part of the game. On the negative side, players suggested that more sound be added to the game. The other negative comments about the game centered on bugs in the playest build of the game. This included a bug that was experienced by all of the observed playtesters where after 8-12 minutes of game play, the encounters stopped spawning.

Overall the response to the game seems favorable. People understood they were in Colonial America and asked questions about other aspects of the geography and time period.

Future builds of the game should include a few key items:

•   A richer sound palate.
• More player feedback after encounters.
• More activities for the player during the game.

The inclusion of these items should be done in a way that increases the emphasis on geography, since more people felt they learned more about economics.

Methodology

Testing was open to anyone that was familiar with middle school geography.  The testers were recruited by word of mouth.  The version of the game used in testing contained all of the current gameplay mechanics, but did not include background music or out of game screens (instruction, introduction, etc.).  The instructions were included on the website where the game was embedded.  The playtest consisted of an observed play session and an online survey.  None of the testers had played an earlier version of the game.

Each observed subject was sat at a windows pc and read the Verbal Playtest Instructions.  Once the playtest instructions were given each person was given a consent form to read and sign.  Upon completion of the consent form each was guided to the website were the playtest build of the game was hosted.  The following was read to the player:

"Thank you for attending the play test of my MA project "Round Route".   Once you are seated and comfortable, you can choose a pseudonym (code name) to be identified by during the analysis of the data collected today.  The choosing of this pseudonym is to help protect you identity during the research and if this work is ever to get published.  Once you have selected and informed the observer of you pseudonym please begin to the game "Round Route".  During gameplay, please speak you questions, comments, and concerns about any and all parts of the game aloud, even if they are negative, so that the observer can record your thoughts about the game.  I understand that this is sometimes difficult, so from time to time you may be asked for your opinions about certain game areas.

After you are finished playing the game please complete the accompanying short survey.  Your opinions are very important to the improvement of the game, so please answer the questions a completely as possible.  Once you have finished the survey you will have completed the playtest.

If at any time you feel uncomfortable with a particular question feel free to ignore it and move on.  If at any time you feel uncomfortable with the play test in general please feel free to leave the computer and exit the room.  If you choose to discontinue the playtest ion this fashion none of the information gained from your observation will be used."

Each person played the exact same version of the game.   All but one use the same computer for the test, the person that didn't use the general playtest computer used a laptop PC that was directed to the same website and version of the game.

After the subjects played the game, they were directed to one of two links on the webpage where they could complete the after game survey.  Once the playtest successfully followed the survey link, the observer left the room so that the survey could be completed.  In the event that the playtester was unable to navigate to the survey site, a link to the survey was sent to them via email to be completed at their earliest convenience.

Demographic Information

The observed playtest consisted of five participants, four male and 1 female.  All participants had at least a high school education.  Two of the testers were age 20-24, one was 30-34, and two were over 35.  None of the participants currently work as a geography educator.

Participant One - HH

HH is a 20-24 year old male.  He had seen some of the game graphics but not a working build of the game.  He is currently a graduate student with an emphasis in serious Game Design.

Participant One - Tech One

Tech One is an over 35 year old female.  She had heard of the project before the test and knew of it goals as a game.  She currently is a member of the Board of Education for a local school district.

Participant One - Ace

Ace is a 30-34 year old male. He had no prior knowledge of the game.  He works for a local cable company.

Participant One - T.I.M.Z.

T.I.M.Z. is an over 35 year old male. He had no prior knowledge of the game.  He works in general construction.

Participant One - Blue Crew

Blue Crew is a 20-24 year old male.  He had no prior knowledge of the game.  He recently left a position with a specialty game retailer.

Synthesis of Observations

HH played the game as if performing an analysis of its features.  He activated each Encounter Sprite several times to see if it would have an effect on his inventory.  He was really bothered by the inclusion of the Fishing sign in the Town View since fishing was not included in the playtest build of the game.  He wanted the notification of the award from the Encounter Sprites be more informative.  HH suggested that space under the mini-map be used to give the player more information about what they were getting.

Tech One took a more tentative approach to the game.  She moved the avatar by tapping the arrow keys instead of holding them down.   She didn't understand what was going on when she was getting robbed.  Tech One went to the stores often to avoid getting robbed.

Ace jumped tight into the game and started pressing buttons.  He first comment were about the speed of the "beaver" sprite.  He activated the Encounter Sprite repeated, and wondered if it had an effect on gameplay.  Ace spent some time paying attention to the prices in each town so he could make the most money.

T.I.M.Z. was a fairly quiet tester.  He moved around the Main View slowly to take things in.  T.I.M.Z. also activated the Encounter Sprite repeatedly, but he didn't check the Inventory Screen.   He seemed to sell goods in his inventory without much regard about the amount he was receiving.  He frequently activated the town's people and seemed to pay close attention to what they had to say.

Blue Crew explored the game world more than other testers.  He felt there should be a reward for going to the edge of the map.   He received the comment "Go away, I'm very busy" while in Georgia and wondered if that meant the all people in Georgia at the time were mean.   At one point in the test he asked if there was "fast travel" system in place.

Observations

Earlier bug testing suggested the player may need to be informed about the price differences among the towns and the details of the Store Screen.  Additional instructions were added to the written game instructions to help the players more easily understand the in game economy.  Nothing was changed in the Store Screen.  The additional instructions seemed to have been enough to keep the player moving around the entire colony area during the playtest observations.  Only one of the playtest participants had noticeable difficulties understanding the Store Screen.

One of the few controls not mapped to the space bar is the inventory screen activation.  Despite the game instructions mentioning that the "i" key would open the inventory 60% of the playtesters asked how to open the screen.

60% of playtesters expressed some concern about being how the Encounter Sprites were affecting their player and inventory.  The most common comment was about the robber sprite.  Testers assumed that the red minus sign signified that something was taken from them, but were concerned about exactly what was lost.

Survey Results

Survey Responses

There were a total of eight responses to the online survey.  This is three more than than the number of people observed.

The questions in the survey and responses were as follows.

1.       Please indicate your gender.

o   75% Male

o   25% Female

2.       Please indicate your age group.

o   0.00% - 18-19

o   37.5% - 20-24

o   12.5% - 25-29

o   25.0% - 30-34

o   25.0% - 35 & older

3.       Please indicate your current grade level.

o   12.5% - High School

o   37.5% - Some College

o   0.00% - Associates Degree

o   25.0% - Bachelors Degree

o   12.5% - Some Graduate Work

o   12.5% - Masters Degree

o   0.00% - PhD

4.       How interesting do you find geography?

o   14.3% - Completely uninteresting

o   85.7% - Somewhat interesting

o   0.00% - Very interesting

5.       How many hours per week do you spend playing digital / video games?

o   12.5% - 0 hours

o   37.5% - 1-3 hours

o   12.5% - 4-7 hours

o   37.5% - 8-12 hours

o   0.00% - 13-20 hours

o   0.00% - 20+ hours

6.       What types (genres) of games do you play most often? (select four)

o   12.5% - Board games (selected 1 time)

o   50.0% - Card games (selected 4 times)

o   25.0% - Action - (selected 2 times)

o   25.0% - Adventure - (selected 2 times)

o   62.5% - Arcade - (selected 5 times)

o   25.0% - Fighting - (selected 2 times)

o   0.00% - Flight - (selected 0 times)

o   0.00% - Racing - (selected 0 times)

o   37.5% - Role playing - (selected 3 times)

o   37.5% - Shooter - (selected 3 times)

o   50.0% - Sport - (selected 4 times)

o   37.5% Strategy - (selected 3 times)

7.       How did you feel about the frequency and amount of the goods that popped up during the game?

o   At first there were goods everywhere and the game moved quickly. Then they all disappeared and it became difficult to stock up.

o   [T]he goods needed to vary a little more.

o   Goods were quite plentiful.

o   [A]lot at first then it seemed to get scarce.

o   The frequency was fine. But after several minutes of play, they seemed to stop popping up. I think it happened after I had visited every town.

o   The frequency was fine. But after several minutes of play, they seemed to stop popping up. I think it happened after I had visited every town.

o   Items popped up quite frequently, but not too often, I thought it was a good rate.

o   [I]t was good didn't same like it was too much or too little.

8.       What did you like most about Round Route? Please provide specific examples.

o   I think this game will teach young people about commerce and supply and demand by having to go to different towns and states to get better prices for your goods.

o   I think this game will teach young people about commerce and supply and demand by having to go to different towns and states to get better prices for your goods.

o   The trading aspect of the game.

o   [T]he buying n selling strategy.

o   The graphics and animations look nice and learning the different rates of selling in the towns.

o   I thought it was fun and interesting. You got to go to different towns and meet different people.

o   Art style was clean and entertaining. Liked animation for when items popped in.

o   [T]hat it gave some insight to early pioneering

9.      What did you like least about Round Route? Please provide specific examples.

o   The goods stopped spawning.

o   didn't find any faults in the short time i played

o   Apart from accumulating 3000, I didn't have much else to do.

o   [N]eeds sound of some sort.

o   Obviously since this is alpha version it wouldn't have this, but I would have appreciated more of a set up and options to start the game. I also never knew how much of a good I collected until I went to town.

o   I would have like more on line directions.

o   Needs some more SFX, to give it more life. I wasn't sure what it meant when a guy would come up in a dust cloud and the minus sign appeared. I imagine he was stealing my supplies, but there was no indication of how much, or what triggered this to happen.

o   [T]he goods popped up over water, that seemed a little un real even for seventh graders.

10.  Would you continue to play Round Route?

o   25.0% - No, I would not play again.

o   37.5% - Yes, In a school or instructional setting.

o   25.0% - Yes, at home for fun/leisure.

o   12.5% - Yes, in school or at home.

11.  What (if anything) did you learn from Round Route? Please provide specific examples.

o   Don't take the first offer I get when I want to sell something.

o   [I] really didn't learn anything.

o   [H]ow to buy n sell.

o   I learned a bit of trade business, where the demands for certain goods were most wanted and where to get them.

o   I could see some aspects of how economies worked and how different geographical locations affect supply and demand. Aside from that, not much.

o   [I] was re acclimated about the geographics  of the states in that area

12.  What (if anything) would you change in the game?

o   I would make the fishing part work.

o   [A] little more interaction with towns people

o   [S]ound

o   Make sure the guy can't walk onto the ocean. You can once you get to South Carolina.
Friendlier eyes on the townsfolk. The black eyes make them look a bit zombie-ish. Have the whites of their eyes show with colored pupils so they look friendlier. The "competition" and main character look fine. The black eyes denote sinister or having the hat over the eyes.

o   Needs the final polishes, music, sound effects. Some variety maybe in the different animal furs you can get, the fur commodity can be the same, but maybe allow player to hunt more than beavers. Would have liked to see fishing, or hunting.

o   [I]think for a school setting for seventh graders it seems like a game that would allow them to learn and have fun doing it.

13.  (Optional) If you have any questions, comments, or concerns about Round Route, please list them here.

o   This game would be great for young children to learn about money and the original colonies and how trade may have worked back then.

o   [C]ool concept

o   I think it needs a little more 'fun' elements. Perhaps mini games for item collection? Allowing for fishing, or hunting (maybe oregon trail style) would add more to the interactivity of the game. Something that would give the player more chances to do something before getting robbed seems to happen at random right now.

o   [G]ood job, and efforts emplaced.

Survey Analysis

The survey indicated that people enjoyed exploring the colonies in the Main View.  The art style was also noted as one of the better part of the game.  On the negative side, players suggested that more sound be added to the game.  The other negative comments about the game centered on bugs in the playest build of the game.  This included a bug that was experienced by all of the observed playtesters where after 8-12 minutes of game play, the encounters stopped spawning.

Conclusions

Overall the response to the game seems favorable.  People understood they were in Colonial America and asked questions about other aspects of the geography and time period. 

Future builds of the game should include a few key items:

• A richer sound palate.
• More player feedback after encounters
• More activities for the player during the game.

The inclusion of these items should be done in a way that increases the emphasis on geography, since more people felt they learned more about economics.