New proposal for social structure of foldit, in a more general, abstract form

[Ignacio]

I got flamed for the last one, but still… I am writing this to include some changes that were suggested from other people's comments + make the proposal more general.
Ideas:

1) Make all recipes available to all players, no group-specific recipes can be allowed for fair competition among all players. Eliminate group competition. Convert groups into social entities, not competitive entities.

This should allow all players to compete in equal conditions. Unfairness is perhaps the worst aspect of the game today. Foldit should not function as a corporation game, in which groups often behave as private corporations under no monopolistic control. Tools must not be hidden. Groups should be structured as collaborative, not competitive efforts. Groups = learning, sharing information, ideas.

2) Differentiate more clearly between soloist efforts and evolver efforts. Eliminate the possibility of using the information from other players to improve soloist scores. Once you start using other people's solutions, you become an evolver. Allow all players to evolve the best solutions of all players at all times.

As I explained in my previous post, I think independent thinking is not sufficiently encouraged or rewarded with this social structure. Top soloist should be only those that come up with independent solutions, not those that efficiently copy other people. Also, it makes no sense to evolve the best solution of a group when it is irrelevant for the very top score. Again, this is due to the idea that groups must compete, which I think is wrong. On the contrary, it is much better to have all players to work on the very top solutions as evolvers at some point of the puzzle.

3) Create a classification for top scriptwriters and another for top programs based on how many times each program is downloaded.

We should recognize the efforts of all scriptwriters and also make them well known for everybody and make all players (not just people within a group) be able to interact with them, suggesting new ideas.

I made a more detailed proposal about how to precisely implement all this before, but the critical ideas are all here. Of course, the implementation could be quite different from what I suggested, even keeping the same ideas.

Given that it is likely this post is also going to be flamed, I will simply not read the answers. Best to all and I hope this helps.

Ignacio

[Rav3n_pl]

Flame on!

1. -1
   Cooperation inside group, competition between groups/players.
   No competitnion - no fun.
   There is very small number of "secret" recipes vs shared ones. Most of group recipes are posted in global when all bugs are removed and script is relly doing something good (at leas I do this way, my team is a debug team:)

2. say what? -100
   Peeking into screenshoot should make me evolver too?
   All-hands puzlles are evil as is described in many places. We need more diversity instead of better score at almost same position (for science).

3. +10
   This one was somehow working long time ago, just scoring funcion showing odd things (number of usage was ok, but how many points gained by script was broken).
   Scripts need description (if ur a writer ADD one finally!) and categorization.

[Ignacio]

Not flamy enough, so I will answer.

1) Competition should mean competition for the top spots, which are the only relevant ones from a scientific point of view and also the only ones that are real fun to play for. Sadly, >50% of our time (even for the strongest players) is passed either working on our own solutions, which are often largely suboptimal (and thus not only scientifically irrelevant but also boring), or in other people's solutions (group) that are often also suboptimal.

Finally, (let's be optimistic) we get to position 15th in the solo, 2nd in the group, about 100 points behind the best score. Then, we get 65 solo points and we feel we have achieved something. But, in fact, we have achieved nothing, we have just lost our time for nothing other than a position in a table of scores. A bit of ego playing, but no science, no big thrill either. This dynamics makes no sense, either scientifically or as players, and should be avoided as much as possible.

Why should not all of us play for the very top spot in each puzzle???. That would be fun.

2) The number of secret recipes is unkown. I can say that several of the recipes that I have, from the Go Science group-only archive, are much better than anything similar in the public repository. Several others are competitive with the best and quite different from what is public, so they provide also an advantage. And still I have had no time to check in detail all what is available, so I think I may be missing some good ones (!). I assume that other groups have some exceptional tools too, otherwise we would be winning all puzzles. I have actually noticed that Go Science as a whole is not competitive in particular types of puzzles, and the simplest hypothesis is that our tools are not good enough, respect to those available in other groups.

This is a real problem. Closed groups with hidden recipes are hampering the progress of the whole community.

3) Screenshots should not be allowed for soloists, only for evolvers!. Especially at the beginning, maximal independence among soloists means maximal variation among solutions, which is in principle best.

However, remember that, according to my proposal, all players can move to use any of the top solutions at all times, so if you think you must peek somebody else work or you are tired of failing short in a particular puzzle, you just become an evolver and keep going. No harm done, except you are not making believe you are a soloist when you are in fact refining other people's work.

4) Description and categorization of scripts are a must, I agree.

In summary, my proposal is based on the idea that independent, creative top soloists, together with top scripts, are the foundations for our success as a community. After that, a strong team of evolvers (which could be all players together) may greatly help to refine the soloist work and improve the final results. Competition should be based mostly on ideas and skills. Now, competition is significantly biased by the share of privileged information within groups and by having private programs. That is, I think, absurd.

Best

Ignacio

[auntdeen]

You make one erroneous assumption about "the top spots, which are the only relevant ones from a scientific point of view".

This is not the case. Foldit's greatest sucess (the monkey HIV) was an evo that was not the number one solution for that particular puzzle - and it wasn't accomplished by scripts, it was late game handwork that nailed it. Good thing that all scripts weren't dumped into global that week… perhaps everyone would have been having too much fun trying them out to get top rank instead of getting it right.

For CASP, while some submissions are top ranked, here are Foldit's submissions for T0722 (puzzle 585 & 589):

585 - rank 6 - petr2 - no group
585 - rank 7 evo - krulon & micheldeweerd & cbwest - foldeRNA (Contenders finished ahead)

589 - rank 15 - smilingone - Beta Folders
589 - rank 33 - auntdeen - Anthropic Dreams
589 - rank 34 - hpaege - no group

Soloists on my team took ranks 1 - 2 - 6 - 7 - 8 - 10, and our high evo was obviously not from my solution in puzzle #589. I only received 32 points and likely brought down my ranking with that finish - but to me it's priceless!

These submissions are not unusual - during CASP, Foldit has used many solutions for submission that were much lower ranked than the "winning" solutions. Please see Madde's excellent work at this page - you can click on any of the puzzles to see the Foldit submissions: http://de.foldit.wikia.com/wiki/CASP10

Our software is far from perfect, and our global top scores are not by a long shot the best gauge of value to science.

[marie_s]

They are other ways to participate to the social life of foldit :

• sharing pictures at the end of puzzles,
• describe your strategies in the wiki,
• helping new ones in the chat,
  these ways give ideas without leading everybody on the same paths.

We dont need:

• to have less freedom : let players share what they want to share,
• to have all the same strategies,
• to have all the same way to participate
• to be on the top in all types of puzzles to be useful but just to try to have on one that, by luck or talent, you may have the good useful idea.

[Ignacio]

I think you are largely right in all what you say, but it does not alter my point of view.

Of course, in CASP you can submit several solutions, so the ones submitted must be the ones that are 1) very good according to standard evaluations, energy etc and 2) the best for a given fold model. It makes no sense to send the same solution 5 times with minimal variations. You must choose as submissions several alternative folds (especially if you get some that are very different) provided that solutions have a reasonable quality, even if some of them are evaluated as quite worse than others. I think that is the main reason for choosing suboptimal solutions.

In any case, I checked (I knew the page, but not that the submissions for some CASP puzzles were already there, thanks for that) and almost always the very top solution is submitted (7/10 in the first 10 cases).
No clue why there are cases in which it is not submitted, but it happens mainly when the same protein has been used in several puzzles, so perhaps the solutions in one puzzle were much better than in the other(s) and those top solutions are the ones that are not included. By the way, I can't evaluate this point well, but you seem to be able to do it. I don't see the CASP submissions for 585 and 589 separatedly, where are those?. You were 7th in 585, and still you say it is the solution in 589 which is submitted. How do you know that?. This would allow me to see more clearly the true level of correlation.

I did not systematically check whether the solutions submitted were always/almost always among the top ones, but in general they are. Otherwise what we do would not make sense. Also, it must be exceptional that a very poor scorer has 1) a peculiar fold (otherwise you would choose a better scorer) and 2) a good energy score.

In any case, community dynamics does not change with all this. My proposal should INCREASE fold diversity by forcing people to work independently. What is decreasing diversity is people massively copying other people solutions, as happens now. And also, I think the only way for us to play is to try to obtain the highest scores for each particular model, generated by soloists as independently as possible, with evolvers concentrating on improving those top solutions. I suggested making available to all people for evolving the top 10 solutions, but this could be increased to 15, 20 or more, if it is better.

It is true that this may not work sometimes, but still we should concentrate on that, because it is the only rational thing that we can do. The fact that sometimes we can get it right by getting it all wrong does not changes the fact that our only way to evaluate what we are doing is by trying to generate top scores and increase the very top scores. Now, I don't think the current social structure is doing that. On the contrary, I think is making people lazy in generating new structures, making many people unable to efficiently improve those structures (due to the lack of the hidden tools that other people have) and inefficient in improving the best structures generated, most often simply because they cannot work on them. I am tired of looking at scores that are 400 points higher than mine and 200 points higher than my group's while I keep twisting my backbone (protein-wise and personally). I doubt the likelihood of my personal solution being a black swann compensates for the points that all top solutions are losing by me not being able to work on them. And the new structures that occasionally I could develop from them. And not only the very top ones, but all those strong competitors that I could SEE have potential,… only I cannot see them at all. Then, multiply that by 25-50 people with about the same skills. What a waste.

In any case, many thanks for the post.

I

[tokens]

There are a couple of things which you are suggesting which I don't believe is true:

1: "The number of secret recipes is unkown. I can say that several of the recipes that I have, from the Go Science group-only archive, are much better than anything similar in the public repository. Several others are competitive with the best and quite different from what is public, so they provide also an advantage. And still I have had no time to check in detail all what is available, so I think I may be missing some good ones (!). I assume that other groups have some exceptional tools too, otherwise we would be winning all puzzles."

I'm in Anthropic Dreams, and I almost exclusively use public recipes. "Magic" recipes is not what gains you a high score, hard manual work and a good knowledge of the public recipes is more important.

1. "What is decreasing diversity is people massively copying other people solutions, as happens now."

I don't believe this is true. I have gained some of my highest ranks without looking at other peoples work: Choosing the template myself and rebuilding the way I believe is right. Clearly if you gain rank 1 in a puzzle, it's not because you copied someone else.

That said, I have gained valuable information studying other peoples solutions. As other people also have suggested, it would be great if all solutions were available for public inspection after a puzzle closes.

[Ignacio]

Answering your criticisms:

We dont need:

• to have less freedom : let players share what they want to share,

Freedom is a typical dialectical weapon: If somebody wants to make me less free, he is naturally mistaken.

Actually, I am one of the players with more freedom in Foldit. Becuase most players lose freedom when they share information and use knowledge from other people's solutions, and I don't. I fully use my freedom to explore my own ideas. Where is your freedom if somebody directs, modifies or constraints your efforts by giving you information?. Keep your ideas free from contaminations!. My suggestion is to make soloists totally independent, i. e. truly free.

Also, according to your idea that freedom = sharing, we would have more freedom if we could share all solutions with all people. Put them in a single repository. Why we don't do that?. Why not a single group where we all roam free?. Why I cannot be in many groups, sharing all their ideas at the same time?. Why the people that thought Foldit does not free us completely?. Because it makes no sense. They want us substantially independent and competing. They constrain us. And they are totally right. I just proposed alternative constraints.

• to have all the same strategies,

I said the same tools, recipes. Strategies are a different thing altogether. Different players play very differently, even with the same tools.

• to have all the same way to participate

Now we all have the same way to participate, I don't understand this, sorry.

• to be on the top in all types of puzzles to be useful but just to try to have on one that, by luck or talent, you may have the good useful idea.

Here I use statistics: if you significantly contribute to many puzzles, it is more likely that you have an useful idea. My proposal was thought to make people able to participate more substantially in the critical moments of the game: development of new ideas and evolution of those ideas.

Thanks for the post.

I

[tokens]

Elaborating on Auntdeen's comments, that diversity, not top score, is the most important:

You said in you opening statement: "Also, it makes no sense to evolve the best solution of a group when it is irrelevant for the very top score. Again, this is due to the idea that groups must compete, which I think is wrong. On the contrary, it is much better to have all players to work on the very top solutions as evolvers at some point of the puzzle."

I think the opposite is true. It's best to have as many solutions evolved as possible (at least if that solution is in the, say, top 30). Unfortunately the evolver scoring system doesn't encourage people to evolve anything other than the very top scoring solution. This is where groups help, by making people evolve a larger number of solutions. Letting everyone evolve all solution will decrease diversity.

That said, I still somehow like you idea of letting everyone evolve everyone's solution. But it would require a different scoring system for evolvers than the current one to make it work in practice.

[Ignacio]

This is a nice feedback. Thanks a lot.

There are no magic recipes, I agree. But still recipes are not the same for everybody. You say you "almost" always use public recipes. I do too. Only that, as you, not always. Sometimes some of our own recipes work better than what is in the public repository. You don't have those and I don't have yours and we both are working below our true abilities. That is my point.

Also trust me if I tell you that AD had a strong lead in recipes about a year ago (I am almost tempted to destrust your humility). I was stunned by the good ideas developed by your group when I saw them for the first time. It was not the recipes, but the IDEAS that were novel when compared with what it was available. And they work much better than anything public, to the point that most recipes today (what I have seen at least) are just refinements of those ideas. It is a matter of time until some group or another gets the same kind of advantage. It may actually be happening right now, only that we don't know. But there is quite a difference in group performances in different types of puzzles. E. g. for some reason Go science is quite bad in design and symmetry puzzles and stronger in the rest. Why?. I have some hypotheses.

You also agree with me in the basic idea that copying is decreasing diversity when you say that you often did best when you did not look at other people solutions. Of course you are right. That is my point also. Make all people work like that and you will see, in average, more and better ideas generated. However, this is not what most people do now.

So I think we fully agree in everything. Best,

I