The story behind solution diversification (or exploration) puzzles

Started by zoran

zoran Lv 1

As you must have noticed by now, we have introduced a brand new type of puzzle that tries to promote exploration and creation of diversified solutions. In these puzzles, the score has a brand new interpretation, simply described as the originality of the solution when compared to all other known solutions. Of course originality without a realistic protein is not very beneficial, so the new puzzles recognize only the original solutions that are above a certain threshold of energy score (the standard Foldit score that we all know and love). Just to be clear the new protocol that we are currently exploring for tough proteins is as follows:

  1. Exploration phase 1: find the most diversified protein structures above energy threshold T1
  2. Exploration phase 2: find the most diversified protein structures compared to clustered solutions from Phase 1, that are above energy score threshold T2 where T2 > T1
  3. Exploration phase 3: find the most diversified protein structures compared to clustered solutions from Phase 2, above energy threshold T3 where T3 > T2
  4. Final Phase: Old style energy scoring puzzles initiated around best solutions from phase 3

This protocol is subject to change and tweaking, but as you can see all early phases lead to the eventual old style exploration, but try to prevent the premature consensus exploration that is common if we just started from step 4 as in the past.

Obviously this is a lot more complicated process, so why are we doing it? One thing that became obvious as we were analyzing the results from the last CASP is that player's creativity and originality seems to get prematurely stymied by the pull of the initial good solutions. As a result, the common path for puzzle exploration is that the first reasonable configuration quickly wins over, and everyone starts working on it. As a result of all the effort, that solution quickly becomes significantly better than other configurations, making it impractical for anyone to explore other options. In the past, some of you may remember us trying a second stage of exploration where players could start from several good solutions and hopefully merge the best aspects of all solutions. That didn't work, the initial best configuration took over very quickly. Several months ago, as well as during a recent beta test, we also tried combining the energy function with the diversification metric. In both cases we found that this mixture of two objectives was not a sufficient incentive to lead to brand new configurations. As a result we decided to try something radically different, something that we knew would challenge the standard ways of playing Foldit, but something that will invariably lead to new creative solutions. Our small initial beta trial showed very promising results, indicating that many new solutions tend to be closer to the known native structure. Encouraged with this finding, we are eager to find out whether we can enable not just your spatial problem solving skills but also the full power of your creativity. This is particularly important, because as we all know computational methods will always be terrible at creativity. Perhaps a new set of skills will develop within the community, much like your current awesome proteomics skills that we wrote about in the Nature paper. We know this won't happen overnight, and that every new skill needs to be nurtured developed and refined by the community. And we know that there will be a number of folks who will not like it when compared to the old game play. That is completely natural. But our goal is to enable you to be the best that you can be, and after thorough analysis we believe that this is the new step. Hopefully we can evolve this into a new mode of play, especially in synergy with the standard Foldit play. As always, we welcome your comments.

auntdeen Lv 1

We need a way to set our own credit best in this type of puzzle. "Very best" is worthless, as you can set a clashing of .1 and get distracted for a moment - thus running your very best up to over 200,000.

"Credit best" is probably fine for your purposes - but was rarely stable enough for our purpose - i.e. it possibly was a score in the middle of a low clashing that could not be stabilized.

What we really need is either an autosave of best seg score/best expo score that is stable - and probably impossible for you to do - or the ability to set that ourselves. I had so many saves that it became very difficult to find the one that I was looking for (especially since the score was ordered by seg score in the saves) - and had to waste time looking for it frequently.

We were all looking for that delicate balance between the seg score & the expo score - with tools that worked against us, rather than for us. Whatever you can provide us to bridge that gap would be not only appreciated, but would encourage more folders to work this type of puzzle.

Rav3n_pl Lv 1

3 "best score saves" for exploration puzzles:

  • best exploration score <- abs best -> useless, killing evos
  • best exploration score on condition met <-credit best
  • best segment score <- how to get it? -> need it badly

lynnai Lv 1

I was having grand fun…. but I've always been more of a "what happens when I do this" player then a finesse player.

spvincent Lv 1

When these exploration puzzles started I didn't understand how they were supposed to work. No matter, I thought, all will doubtless become clearer after going through a couple of them.

Well we've just started the second exploration puzzle and I'm none the wiser. How are the starting structures for rounds 2,3 etc derived from the (presumably) widely different structures from the previous round? You'd have thought that in subsequent rounds we'd be given more than one starting structure but this doesn't seem to be the case. Instead there's a reference above to 'clustered solutions' but there's no real explanation as to what these are or how they are derived. And the use of the plural also suggests that there should be multiple starting structures. Maybe getting these right is still a work in progress but a little more explanation would be helpful.

And what happened to Stage 4 of Exploration Puzzle 1?