Beginner Puzzles

[beenen34]

Hey Folders,

There's been a bit of discussion over the past couple of days (feedback thread: http://fold.it/portal/node/985991) on how to best teach newer players about the basics of competition puzzles. We, the devs, have an idea or two for how to implement this (gathered from suggestions from users), and we would like to receive your feedback, or better yet, other ideas that the community has. We would like feedback both from newer players who have just jumped into the game, and veterans who have been teaching beginners for a long time.

In the feedback post, I said we would post beginner puzzles that would last for a month, with a new one coming up every other week. These puzzles would be available only to those with less than 15 or 150 points and would be simpler proteins that would be easier to improve than the regular competition puzzles. One such puzzle (Beginner Puzzle 1: Mouse) was posted yesterday. The downside to this approach, as LennStar pointed out, is that expert players are less capable of assisting newer players in the beginner puzzles. So we've got a second idea:

There would be two permanent beginner puzzles. Each puzzle would be available for competition for those with less than some number of points (say, 150). Every week, points would be given out and the scoreboard would be reset. The puzzle would also be available to those who have more than 150 points, but the scoreboard would never reset and points would never be given out. Experts would be able to see the structure and assist new players, but not have to spend time competing on these puzzles.

Let us know what you think! Our veteran players have always been crucial to initiating new players, so we'd like to know how you think the initiation process can be smoother.

Thanks,

The Dev Team

[Christoph]

Why not simply add a few competition-like puzzles to the intro puzzles? You don't give the players any hints anymore and let them fold a protein completely from the open game to the end game with all the tools available in the competition mode. I think this is an important step for new players, as it's the first time they have to decide themselves what to do. The intro puzzles before each concentrated on one specific tool or method. The score limit that has to be reached should be of course not too high.

There are many other unclear things like structure mode, note mode, duels and buddies, guides and the ability to share puzzles with your group. I think we should find a way to explain them properly to new players that finished the intro puzzles. Maybe an ingame FAQ would be an option. At the end of the intro puzzles you show the player where to get help and information about the other tools available in competition mode.

I think this could make it easier for newcomers because they now get a better understanding of how the competition mode works.

Another thing I would like to mention is, that although you give a rough estimation of the difficulty of the proteins in the protein selection dialog, I've hardly seen any puzzles that aren't classified as "Intermediate". I propose to keep the current duration of puzzles, but instead release more puzzles with different difficulty levels at the same time. In that way players can choose more easily the puzzles they want (new players -> "Easy" puzzles, pros -> "Advanced" puzzles) and there's a greater variety. So nobody's getting bored. :)

[Migi]

That's certainly a good idea, but if you leave the new players with just a protein and no instructions whatsoever, what's the benefit compared to trying a real competition right away? I think you should open the puzzle with a note encouraging them to try what they want, but every minute or so give pop-up bubbles like "Try to make the protein more compact", or "Don't forget the sidechains!", or "If you're stuck, you can try to nudge & wiggle" with a link to a video explaining how to nudge & wiggle. Of course it should depend on how well the user is doing, not suggesting to look at the sidechains if those are already properly folded.

The idea about "easy" and "advanced" puzzles is a bit vague. After all, what is an easy puzzle and what is a hard puzzle? Some could argue that easy puzzles are the ones with very low points, because you can easily get more points, and the harder ones are those that are near their native state, because those require things like walking the dog. But you could just as well argue that the ones far off the native state require a lot more insight to fold properly and the high-starting-score ones can't really be screwed up.

So instead of "easy" to "advanced" I would classify puzzles as "low RMSD" to "high RMSD", but with a fancier name.

I would provide some variation, like a maximum number of moves, a time limit, try to match the guide, a game where you get 2 minutes to get a score as high as possible (with bronze, silver and gold medals), a game where you guess which of the two given folds has the highest score, a game where you can only tweak the sidechains, a game where you should try to make a cookbook recipe that solves it, there are a million possibilities.

These "games" should not necessarily be part of the tutorials but they can be a classified as "challenges", unlocked when you finish the tutorial. If possible, it would be awesome if there were multiple challenges in every category (time-based, limited-moves, sidechain-only) and completing one would unlock the next, harder one.

I'll stop now, before I write a book about this. But you got me started and I just love FoldIt.

[axcho]

I think the distinction between easy and hard is less vague than it might appear. It's about what's easy or hard for a beginner to start with, and can be determined more or less empirically. We could even use stats on how people actually do in the puzzle to dynamically mark it easy or hard - not that I'm promising anything! ;)

The pop up tips are an interesting idea. I'm not sure yet if that's the best approach. I think the way to find out would be for us to watch some new players trying the science puzzles for the first time, after the intro, and see where they might need extra help or encouragement.

Varying goals, with awards would be very cool. I like that idea.

[LennStar]

I don't think the popups are a good idea. I would hate them earlier then they need for every hint to shown one time :D

The part "I would provide…" sounds very good. You can also take that for duels. Propably with user choosen goals like "only 20 moves and only sidechains" or "only 5 wiggles, rest free to use".

[rednalyerg]

Hello, I am a 99% completely new to Foldit (just did all the intro puzzles yesterday), and I have a number of thoughts on the beginner experience.

In case it makes a difference, I am using the Linux version.

1) What to do after Intro 4-4???

After doing 4-4 (I've seen a level 5 mentioned somewhere in wiki or something, but was not available), it was not clear what to do next.  All I could do was go into the "Science Puzzle"... then found the one that said "Beginner".

My suggestion: Provide a clear progression of puzzles.

2) The Intro Puzzles are incomplete?

The last intro puzzle was 4-4.  I just downloaded this yesterday, so I assume I have most complete version (and/or it should be able to get puzzles online).  I have seen mentioned somewhere (on the wiki?) "Level 5", but there are no level 5 puzzles available.  
When I moved on to "Beginner Puzzle 1 (<15): Mouse", suddenly a great many more buttons, menu items, and so forth appeared that were not available up to 4-4.  I had no idea what these were.  The only apparent option was to look at each button/menu item, read its tool-tip, and fiddle with things by trial and error.  It definitely seems that there should many more "intro puzzles" to introduce all these other features.

My suggestion:  A complete series of tutorial puzzles, that progressively introduce all the features of the game, including all tools for manipulating the protein, as well as all the other options, such as changing the view options, using the chat windows and so forth.  The sudden jump from 4-4 to the full game interface was rather overwhelming.

Another note:  I said 99% new because I actually tried Foldit a year ago (my computer at the time was too slow to handle it).  I vaguely recall the Intro Puzzles back then covering more features (like "freezing" pieces of the model, or showing transparent "ghost" versions of the model to try to achieve).  So I am wondering if some of those intro puzzles have been eliminated, and if so, why?

3) Could not find Web-based tutorial anywhere.

Maybe it is there, but I haven't found it yet if it exists.  When I check out the FAQ, forums, or the wiki, the topics all look rather advanced. There is nothing inviting out there that says "Hey you, New Player!!  Come look here if you are trying to figure out what is going on."  If there is a tutorial out there, it needs to be *prominently* linked from the Foldit home page.  I would likewise recommend such a link on the home page of the wiki.  The web-based tutorial could go hand in hand/step by step along with interactive sequence of tutorial puzzles, but could explain in detail and depth the "what", "how", and "why".

4) Better visualization/feedback on score changes.

This does not exactly relate to the choice of new puzzles, but it relates to the experience of a new player and the development of their skill and intuition.  Working on the Into Puzzles did almost nothing to help me develop an intuition for how moves affect score.  There are three major reasons for this:

a)  The score is just a tiny number at the top of the screen.  I get little or no "feel" for how moves affect score from this -- I would have to make a tiny adjustment, then take my eyes off the model and look at the number and back to the model.  When I explored the "view" options when I got to "Beginner Puzzle 1", I found some options that solve this problem somewhat by highlighting problem areas -- but the tutorial puzzles never taught me about these options.  Mind you, I am not saying that there is no visual feedback -- the spiky clashes and bubbly voids are useful -- there is just not enough.  It would be good to have a more visual/intuitive depiction of the score that can be sensed in the players peripheral vision, so to speak.  The little popup score changes that sometimes occur are only slightly more useful than the score itself.
     Suggestion:  A large "thermometer style" score indicator that both grows and changes color as the score improves.  Because of the huge dynamic range of the scores, a simple linear thermometer may not be sufficient.  It may be necessary to have one indicator of overal score (possible logarithmic to accomodate the large range) and another indicator for small variations in the score.  The scale of each indicator may need to adapt.  These indicators should be the full height or width of the display area, so that the player will perceive the changes without having to actually look at them.

b) The score often changes far too quickly to get any sense of what happened and why it changed.  This particularly happens when the score crashes to zero as a results of introducing several new clashes.
   Suggestion:  I am not sure if clashes (for example) are discrete or continuous score features, but I recommend continuous (this should be possible since the underlying physics is continuous).  Then slow down the rate of change in the model to keep the rate of score change reasonable.  Basically let the action go into "slow motion" so the player can see the score change along with what is happening to the model.  This slow-motion effect could be an adjustable or toggleable feature so advanced players can turn it off (or reduce the effect).

c) Most of the score changes in the Intro Puzzles result from doing "shake" and "wiggle".  This is rather unsatisfying.  I had little feeling that I had done anything myself to improve the score.  I just hit "wiggle" and watch the score climb.
   Suggestion:  A series of simple puzzles in which "shake" and "wiggle" (and other "automatic" tools) are not available.  The player will do everything "manually" in order to see how changes affect the score.

5) The tutorial puzzle sequence needs lots of "toy" puzzles to help better develop intuition AND skill in simply manipulating the model. More models with roughly the complexity of 1-1, but with different tips/objectives for manipulating the model. One interesting twist would be instead of trying to get the highest score possible, to manipulate the model to get a specific sub-optimal score – this will force player to push/pull, rotate back&forth, and so on, thus developing their skill.
Also, I think there is a feature that shows some kind of "ghost" shape for the model, which can be a target for the player to achieve. This was not in any of the Intro Puzzles, but I saw it mentioned somewhere. Such "ghost" targets need not be "optimal". Just configurations that require a player to push/pull/rotate and use the other tools to get the model into that shape.
I think a good idea would puzzles that had a sequence of such shapes – the player manipulates the model into that each successive shape (developing their skill manipulating the model), and as each shape is achieved, they are treated with some explanatory text for why that shape has the score that it has.

---

These are my initial impressions and suggestions. I hope you find them useful.

[rednalyerg]

Just a slight correction to my previous post. When I suggested puzzle "1-1" in section (5)… I meant more like 2-1 or so. Actually for the "sequence of target shapes" idea, even more complex models would be good.

[rednalyerg]

OK, I just went through the Intro Puzzles again. This time I did it "online", and they somehow seemed a little different, and a little bit more instructive. That might just be because having gone through them once, I picked up on things I missed the first time around. Or are some of the online intro puzzles different from the offline set? In any event the intro puzzles still end at 4-4 and then all there is to do is to jump into the Science Puzzles with no guidance. There are many features of the interface which are not covered in the intro puzzles.

One other note. Having gone through the intro puzzles again, I can anticipate that the developers/designers might say that some of my suggestions are already there. My general response would be "not enough or not in the right way". To some extent, just a bit more repetition of the same concept would be useful – maybe a dozen puzzles for each level instead of four.

One thing off the top of my head. The usefulness of freezing was not really clear in the puzzle that included it. In fact I'm quite uncertain precisely what it does (the frozen object still seemed to move when I did a wiggle).

Here are some things that hit me when I first try the Beginner Science Puzzle. Some explanation of these may be available in the Wiki or other documentation. However, most players, even very intelligent and (initially) interested players are going to throw their hands up in frustration.

Actually, before I get into the puzzle there is the list of Science puzzles, the First thing a new player is going to see after working through intro puzzle 4-4… "Grand Challenge"??? Woah… I don't think I'm quite ready for that… don't they have something easier? Oh I have to scroll down… What does "evolver" mean? What are these "(<150)" after the puzzle names? What do the puzzle numbers mean (if anything)? What are these dates for when the puzzle is "over"? Why would a puzzle be "over"? Should players be able to try any puzzle any time?… Oh there it is, a puzzle that says "Beginner"… that must be the one to try next… Here goes…

Woah… where did all this stuff come from? Most of this was not there before…

Cookbook? Where did that come from? What is it? How do I use it?

What are the red arrows making circles around the… are they circling the voids or the side chains? What do they mean? Are they giving me information about the state of the model? Or are they some kind of tool to manipulate the model? Or both?

Reset Structure? What does that do? How is it different from resetting the whole model? Presumably there is some way to select substructures of the model and save/ reset them independently. How? I have never seen this before.

A view menu? Look at all those options. Some sound like they'll be pretty useful. Would be nice to know what they all mean…

A mode menu? What is structure mode… hmmm… maybe something to do with reset structure? What is node mode?

…and so on.

The combination of all these new unexplained features, along with nothing but trial-and-error guesswork to get started on working on the protein model… means only the most interested and dedicated of players are going to take the time to dig through the web pages, wiki, or just "try and see", to figure out how these things work. Many players who were interested and fairly excited (if still a little confused) up through Intro 4-4 are going to hit a wall when they start "Beginner Puzzle 1".

I hope I have made a good case that there needs to be a long, smooth progression of tutorial puzzles that incrementally cover every feature of the user interface (this includes the online interactive/community features, so the player has a clue about duels, contests, puzzle "expiration", etc.), while progressively giving the new player more "hands on" experience fiddling with the models. When the player first tries "Beginner Puzzle 1", you want them to have some idea what to try first, other than just random fiddling or clicking "wiggle".

[axcho]

rednalyerg, I totally agree with everything you've said here. Foldit is still at a very rudimentary stage.

I'd compare it to World of Goo at three and a half months. Ron Carmel writes:

"at first we wondered if we were showing the game to the right people.  why are they so bad at it?  do they simply suck?  it shouldn’t be so difficult to figure this game out!  it would take months of playtesting to get it feeling good and to not lose non-gamers after 30 seconds of play.  if you play this version, pay attention to your frustration and confusion levels as you try to work through the puzzles.  that should give you an indication of how far we still had to go."

That's about how far Foldit has to go, if not further. Maybe another year of playtesting and polishing and it will feel like a real game? :p

There are some cool things coming up, like a new interface that should make it easier to introduce interface elements gradually. I'm hoping that once the design mode is out, we can really start to focus on the playability of the game. Though I'd hope that before then, we could try some of your suggestions such as puzzles without shake or wiggle, and puzzles based on moving the protein to match a guide.

Thanks a lot for your feedback. :)

 

[ChristianK]

As a beginner I would really like to see replays of how advanced folders do it.
It should be possible for closed challenges to have a mode where one can see exactly what a person did to reach those high scores.
Afterwards you would have an idea about what to try.

A replay mode would probably also be good for experts to learn from each other.