[NewChapter] performance (speed) issues

Started by Susume

Timo van der Laan Lv 1

Performance measurements results (done after the latest update on 12/1/2014).
Method of measurement: running TvdL DRW 231 with timing (http://fold.it/portal/recipe/47727) on the ED puzzle in main (beginner puzzle) and newchapter, after an initial shake, 1 iteration.
Settings of DRW: only length 3, more options: 1 iteration. Selected to work on: 81-83. Running both minimized and in foreground.

Results:
Newchapter foreground:
Local shake during stabilizing: mean 10 seconds
Global wiggleall(6) during stabilizing: mean 180 seconds
Global shake during fuze: mean 55 seconds
Global wiggleall(2) during fuze: mean 60 seconds
Global wiggleall(6) during fuze: mean 180 seconds
Remark: there was very little variation in running times in the wiggle durations.

Newchapter minimized:
All shakes about 30% faster, all wiggles about 8% faster, same remark as on foreground.

Main foreground:
Local shake during stabilizing: 10 seconds
Global wiggleall(6) during stabilizing: 106-170, mean 140 seconds
Global shake during fuze: mean 50 seconds
Global wiggleall(2) during fuze: 6-68, mean 50 seconds
Global wiggleall(6) during fuze: 31-42, mean 35 seconds

Main minized:
Same kind of numbers but about 20% faster.

Other observations: During the newchapter wiggle all iterations take about the same time and the score is still changing even in iteration 6.

Conclusions:
There is a major change in wiggle behavior, it is now time sliced, not depending on when the algoritm says it is enough.
This is a major game change. Wiggle will not settle after a few iterations.

Final remarks:
I will repeat these measurements on a much faster computer.
These measurements are on the ED puzzle, on ED there are 2 changes. I want to be able to do these kind of comparisons on other types of puzzles. To make that really possible other puzzle types should be available in main/devprev as well as in newchapter AND there should be a way to share from main/devprev to newchapter so beginning positions can be really equal.
I will comment later on what I see as consequences for the game of this major wiggle behavior change.

Susume Lv 1

I have been comparing Blue Fuse v 1.1 in various contexts on the Staph Aureus ED puzzle (158 segments). The average time for the recipe to run on my machine in devprev (less precise) was 178 seconds; in newchapter the average was 396 seconds, a factor of 2.22.

Running the recipe unmodified in newchapter left points on the table, in that wiggle was still getting points when it cut off after the 8 iterations coded in the recipe. I modified it so the last wiggle would loop in blocks of 2 wiggles (you should always wiggle an even number of times) until it stopped gaining whole points. This version had an average run time of 582 seconds, or 3.27 times as long as the original in devprev. The looping version gained 75% more points on average than the original (on the one specific pose I was using - results may vary by how stable the protein is before starting).

I compared running this modified recipe when my CPU was at about 50% vs. 90-100%. While there was a noticeable speed difference, I did not see a significant difference in points.

The recipes we have now will run and gain points in newchapter; they will not be completely broken. If we don't modify the wiggles in the recipes to loop until they stop gaining, the missed gains will not necessarily be permanently lost; rather they will be postponed until later iterations of the recipe or until later recipes. It will become more common to simply run out of time for recipes, and less common to stop because you have found all the points you can.

Timo van der Laan Lv 1

Just have read the discussion in veterans.
It might be better to revert the wiggle to full iterations, this will have speed implications but will not break algoritms.
And add new seperate wiggle functions that do wiggle in chunks like it does now in newchapter.
Also pls do a big effort to optimize the score function as this will help not only wiggle but also mutate and shake.
These are my first thoughts after reading, maybe more when I get back from work.

Bruno Kestemont Lv 1

"(you should always wiggle an even number of times)".
Why?
I thought each step was self-sufficient. If they need 2 steps, why don't they do these 2 steps in one?

Susume Lv 1

jflat told us this some time ago: "<@jflat06> what's actually happening is we idealize the backbone every other step."

I believe this was in a conversation about why iteration 2 had started giving far more points than iteration 1. The only place I know of where it is documented is in the details page for the structure.WiggleAll function in the wiki: http://foldit.wikia.com/wiki/Foldit_Lua_Function_structure.WiggleAll

If you are just wiggling with bands or wiggling at a low CI to lower the score, it's fine to wiggle just once, but if you are trying to get all the points out of a position it's better to wiggle an even number. This seems to be true in newchapter as well, where the odd numbered iterations go by much faster than the even numbered ones.

jeff101 Lv 1

I heard last night that newchapter has been in Rosetta@Home several months already (perhaps since Oct 10, 2013). Have you noticed a drop in speed with Rosetta@Home due to newchapter? If so, by how much?

dekim Lv 1

Hi, I'm a developer for R@h. I haven't noticed a drop in throughput but if there was, it would probably be pretty small.

AsDawnBreaks Lv 1

Yeah, perhaps "deprecate" the old function and replace it with new algorithm & changes that had been planned on being added?

Timo van der Laan Lv 1

Tested puzzle 663.
Loading the puzzle takes a long time
Initial fuzing to get it a bit stable took 1/2 hour.
Following that 1 mini round of DRW231 with timing took 3/4 hour.
Test2.
I could load saved old versions. Repaired the worst parts by hand and then ran a DRW test version where I put back in tail recursion for wiggle. It takes, when the score is not too bad, about 10 rounds of wiggle to get to the stable state, if bad then 30 is not enough. Each round of wiggle taking 35 seconds.
I only selected 2 slots to be stabelized (the best score and the best pairwise score) but not succeeded getting to the fuze state. Only when the gain is getting very low a double iteration of wiggle is getting fast.