Icon representing a recipe

Recipe: Rebuild Worst Spherical Stabilize

created by GaryForbis

Profile


Name
Rebuild Worst Spherical Stabilize
ID
100274
Shared with
Public
Parent
None
Children
Created on
November 29, 2014 at 02:45 AM UTC
Updated on
November 29, 2014 at 02:45 AM UTC
Description

Rebuild Worst Scoring v1.2 Segment then use a Spherical Stabilization routine on the local segments.

Best for


Code


recipeTitle='Rebuild Worst Spherical Stabilize' recipeVersion='1.1' -- 11/23/2014 --[[ Rebuild Worst Spherical Stabilize is a test bed for spherical stabilize after rebuild. revisions: if you release a new version use a new library name for the new stuff 1.0 -- 11/23/2014 by GaryForbis initial version. 1.1 -- 11/25/2014 by GaryForbis finished OptionValidation fixed min and max rebuild attempts implemented allLoops, keep bands, keep frozen, use selected, and use frozen as selection options 1.2 -- 11/27/2014 by GaryForbis fixed rebuild of rwss.scores. fixed when to display keep frozen check box. added alternate scoring functions. added ability to disable slow filters. ]] orig={} -- the puzzle settings when the recipe started. rwss={} -- the stuff in RWSS. function orig.Save() save.SaveSecondaryStructure() orig.settings={} -- stuff gotten from the puzzle at the start of the script. orig.settings.BandStrengthFactor = behavior.GetBandStrengthFactor() orig.settings.BuriedSidechainShakeAccuracy = behavior.GetBuriedSidechainShakeAccuracy() orig.settings.ClashImportance = behavior.GetClashImportance() orig.settings.ExposedSidechainShakeAccuracy = behavior.GetExposedSidechainShakeAccuracy() orig.settings.ShakeAccuracy = behavior.GetShakeAccuracy() orig.settings.SlowFiltersDisabled = behavior.GetSlowFiltersDisabled() orig.settings.WiggleAccuracy = behavior.GetWiggleAccuracy() orig.settings.band={} orig.settings.allBandEnabled=true orig.settings.hasBands=(band.GetCount()>0) orig.settings.hasEnabledBands=false for b = 1,band.GetCount() do orig.settings.band[b]={IsEnabled = band.IsEnabled(b)} if band.IsEnabled(b) then orig.settings.hasEnabledBands=true else orig.settings.allBandEnabled=false end end orig.settings.isFrozen={} orig.settings.isSelected={} orig.settings.note={} orig.settings.allLoops=true for seg = 1,structure.GetCount() do local backbone, sidechain=freeze.IsFrozen(seg) if backbone or sidechain then orig.settings.isFrozen[seg] = {backbone=backbone;sidechain=sidechain} orig.settings.hasFrozen=true end if selection.IsSelected(seg) then orig.settings.isSelected[seg]=true orig.settings.hasSelected=true end orig.settings.note[seg]=structure.GetNote(seg) local ss=structure.GetSecondaryStructure(seg) if (ss=='H') or (ss=='S') then orig.settings.allLoops=false end end end function orig.Restore() save.LoadSecondaryStructure() -- unimplemented commented out --behavior.SetBandStrengthFactor(orig.settings.BandStrengthFactor) --behavior.SetBuriedSidechainShakeAccuracy(orig.settings.BuriedSidechainShakeAccuracy) behavior.SetClashImportance(orig.settings.ClashImportance) --behavior.SetExposedSidechainShakeAccuracy(orig.settings.ExposedSidechainShakeAccuracy) --behavior.SetShakeAccuracy(orig.settings.ShakeAccuracy) behavior.SetSlowFiltersDisabled(orig.settings.SlowFiltersDisabled) --behavior.SetWiggleAccuracy(orig.settings.WiggleAccuracy) if orig.settings.allBandEnabled then band.EnableAll() else for x,b in pairs(orig.settings.band) do if b.IsEnabled then band.Enable(x) else band.Disable(x) end end end freeze.UnfreezeAll() for seg,isFrozen in pairs(orig.settings.isFrozen) do freeze.Freeze(seg,isFrozen.backbone,isFrozen.sidechain) end selection.DeselectAll() -- normal case for seg,IsSelected in pairs(orig.settings.isSelected) do selection.Select(seg) end for seg=1,structure.GetCount() do structure.SetNote(seg,orig.settings.note[seg]) end end function rwss.GetScore() s=current.GetEnergyScore()+.0005 return s-s%.001 end function rwss.OptionValidation(action,value) local aType,label,minValue,maxValue=action[2],action[3],action[5],action[6] if (aType=='Check') or (aType =='Label') then return true,value elseif (aType=='Integer') or (aType=='Number') then local pat if aType=='Integer' then pat="^([+-]?%d+)$" else pat="^([+-]?%d*%.?%d+)$" --no scientific notation. end while true do local n _,_,n=value:find(pat) if n then value=tonumber(value) if not (minValue and (value<minValue)) and not (maxValue and (value>maxValue)) then return true,tonumber(value) end end local dlg=dialog.CreateDialog(aType.." Validation Error") dlg.l1=dialog.AddLabel(label) if minValue and maxValue then dlg.l2=dialog.AddLabel('Must be '..aType..' between '..tostring(minValue).. ' and '..tostring(maxValue)..'.') elseif minValue then dlg.l2=dialog.AddLabel('Must be '..aType..' no less than '..tostring(minValue)..'.') elseif maxValue then dlg.l2=dialog.AddLabel('Must be '..aType..' no more than '..tostring(maxValue)..'.') end dlg.n=dialog.AddTextbox('','') dlg.b1=dialog.AddButton('OK',1) dlg.b0=dialog.AddButton('Cancel',0) if dialog.Show(dlg) == 0 then return false,nil end value=dlg.n.value end else print(aType.." isn't a valid option type.") return false,nil end end function rwss.ChangeScoring() local valid=false local dlg local subscoreNames = puzzle.GetPuzzleSubscoreNames() repeat valid=false dlg=dialog.CreateDialog('Change Scoring Function') dlg.l1=dialog.AddLabel('One Scoring method only') dlg.scoreSeg=dialog.AddCheckbox('Score segment',true) dlg.scoreDelta=dialog.AddCheckbox('Score difference from neighbor',false) dlg.scoreSphere=dialog.AddCheckbox('Score sphere average',false) dlg.l2=dialog.AddLabel('No Checks below for Total Segment Energy Score') for i =1,#subscoreNames do dlg['score'..subscoreNames[i]]=dialog.AddCheckbox('Include '..subscoreNames[i],false) end dlg.b1=dialog.AddButton('OK',1) dlg.b0=dialog.AddButton('Cancel',0) if dialog.Show(dlg) == 0 then print ('Score using: Segment Energy Score') return end local methods=0 if dlg.scoreSeg.value then methods=methods+1 end if dlg.scoreDelta.value then methods=methods+1 end if dlg.scoreSphere.value then methods=methods+1 end if methods==1 then valid = true end until valid if dlg.scoreDelta.value then rwss.options.scoreDelta=true print('Score using: Greatest score difference from neighbor.') elseif dlg.scoreSphere.value then rwss.options.scoreSphere=true print('Score using: Average of segment scores in the sphere.') else print ('Score using: Segment Energy Score.') end rwss.options.scoreSubComponents={} for i =1,#subscoreNames do if dlg['score'..subscoreNames[i]].value then table.insert(rwss.options.scoreSubComponents,subscoreNames[i]) print('Include: '..subscoreNames[i]..'.') end end if #rwss.options.scoreSubComponents == 0 then rwss.options.scoreSubComponents=nil -- don't score subcomponents end end function rwss.GetOptions() local optionsDialog = { -- {condition, -- {index,type,label,default,(optional)minimum,(optional)maximum}} {not orig.settings.allLoops, {'allLoops','Check','Run as all loops',false}}, {orig.settings.hasSelected, {'rebuildSelected','Check','Only rebuild selected segments',true}}, {orig.settings.hasFrozen and not orig.settings.hasSelected, {'selectFrozen','Check','Use frozen as if selected',true}}, {orig.settings.hasFrozen, {'leaveFrozen','Check','Leave frozen during the run',orig.settings.hasSelected}}, {orig.settings.hasEnabledBands, {'leaveBands','Check','Leave bands enabled during the run',false}}, {true, {'l1','Label','Score using Segment Energy Score'}}, {true, {'changeScoring','Check','Changing Scoring Method',false}}, {true, {'disableSlowFilters','Check','Disable slow filters (bad idea)',false}}, {true, {'maxRebuildAttempts','Integer','Segment Rebuild Attempts',6,1,10}}, {true, {'rebuildsBeforeRetrying','Integer', 'Successful Rebuilds before clearing list',5,0,structure.GetCount()}}, {true, {'minRebuildPoints','Number','Minimum Points for a Successful Rebuild',2.0}}, {true, {'minScorePoints','Number','Score improvement clears rebuild list',40,0}}, {true, {'radius','Number','Radius of sphere to local wiggle',9,1.9}}, {true, {'wiggles','Integer','Cycles to wiggle',10,1}}, {true, {'lowCI','Number','Low CI for local wiggle',0.03,0,1}} } local dlg=dialog.CreateDialog(recipeTitle) for i=1,#optionsDialog do if optionsDialog[i][1] then local action=optionsDialog[i][2] local actiontype=action[2] if actiontype == 'Label' then dlg[action[1]]=dialog.AddLabel(action[3]) elseif actiontype == 'Check' then dlg[action[1]]=dialog.AddCheckbox(action[3],action[4]) else dlg['l'..action[1]]=dialog.AddLabel(action[3]) dlg[action[1]]=dialog.AddTextbox('',tostring(action[4])) end end end dlg.b1=dialog.AddButton('OK',1) dlg.b0=dialog.AddButton('Cancel',0) rwss.options={} if dialog.Show(dlg) == 0 then return false end local valid for i=1,#optionsDialog do if optionsDialog[i][1] then local action=optionsDialog[i][2] valid,rwss.options[action[1]]=rwss.OptionValidation(action,dlg[action[1]].value) if not valid then return false end end end while rwss.options.selectFrozen and rwss.options.leaveFrozen do local fdlg=dialog.CreateDialog("Fix Incompatibly Frozen") fdlg.l1=dialog.AddLabel('Uncheck one of the following') fdlg.s=dialog.AddCheckbox('Use frozen as if selected',true) fdlg.l=dialog.AddCheckbox('Leave frozen during the run',true) fdlg.b1=dialog.AddButton('OK',1) fdlg.b0=dialog.AddButton('Cancel',0) if dialog.Show(fdlg) == 0 then return false end rwss.options.selectFrozen=fdlg.s.value rwss.options.leaveFrozen=fdlg.l.value end if rwss.options.changeScoring then rwss.ChangeScoring() else print ('Score using: Segment Energy Score') end rwss.options.changeScoring=nil -- don't display for i=1,#optionsDialog do local action=optionsDialog[i][2] if rwss.options[action[1]] ~= nil then print(action[3]..': '..tostring(rwss.options[action[1]])) end end return true end function rwss.GetSegScore(seg) if (seg < 1) or (seg > structure.GetCount()) then return nil end if rwss.options.scoreSubComponents then local x = 0 local subComponents=rwss.options.scoreSubComponents for i=1,#subComponents do x=x+ current.GetSegmentEnergySubscore(seg,subComponents[i]) end return x else return current.GetSegmentEnergyScore(seg) end end function rwss.GetWorkingSegScore(seg) -- handle the various scoring function. -- delta and sphere scoring are currently mutually exclusive. if rwss.options.scoreDelta then local x,y=rwss.GetSegScore(seg-1),rwss.GetSegScore(seg+1) if x and y then return rwss.GetSegScore(seg) - math.max(x,y) else return rwss.GetSegScore(seg) - (x or y) end elseif rwss.options.scoreSphere then local j = 0 local x = 0 local radius = rwss.options.radius for i = 1,structure.GetCount() do if structure.GetDistance(seg,i)<=radius then j = j + 1 x = x + rwss.GetSegScore(seg) end end return x/j else return rwss.GetSegScore(seg) end end function rwss.GetScores() -- lowest score to highest score local scores={} local i=0 if rwss.options.rebuildSelected then for seg,IsSelected in pairs(rwss.IsSelected) do if not (structure.IsLocked(seg) or freeze.IsFrozen(seg) or (structure.GetSecondaryStructure(seg) == 'M')) then i=i+1 scores[i]={seg,rwss.GetWorkingSegScore(seg)} end end else for seg=1,structure.GetCount() do if not (structure.IsLocked(seg) or freeze.IsFrozen(seg) or (structure.GetSecondaryStructure(seg) == 'M')) then i=i+1 scores[i]={seg,rwss.GetWorkingSegScore(seg)} end end end table.sort(scores,function(a,b) return a[2]<b[2] end) return scores end function rwss.Setup() orig.Save() recentbest.Save() rwss.startingScore=current.GetEnergyScore() rwss.bestScore=rwss.startingScore local startingScore = rwss.GetScore() print(recipeTitle.." Starting Score: "..tostring(startingScore)) if not rwss.GetOptions() then print('You cancelled out of the option dialogue.') return false -- user cancelled end if rwss.options.allLoops then selection.SelectAll() structure.SetSecondaryStructureSelected('L') end if rwss.options.rebuildSelected then rwss.IsSelected=orig.settings.isSelected elseif rwss.options.selectFrozen then rwss.options.rebuildSelected=true -- fake out the scoring function rwss.IsSelected={} for seg,isFrozen in pairs(orig.settings.isFrozen) do rwss.IsSelected[seg]=true print(seg) end end if not rwss.options.leaveFrozen then freeze.UnfreezeAll() end if not rwss.options.leaveBands then band.DisableAll() end rwss.scores = rwss.GetScores() rwss.rebuildsRemaining = rwss.options.rebuildsBeforeRetrying rwss.scoreBeforeRetrying=rwss.bestScore+rwss.options.minScorePoints rwss.triedSeg={} -- initialize the list of segments tried. return true end function rwss.GetSeg() for i = 1,#rwss.scores do seg=rwss.scores[i][1] if not rwss.triedSeg[seg] then print("Trying Segment "..tostring(seg).." scored "..tostring(rwss.scores[i][2])..".") return seg end end return nil end function rwss.RebuildSegment(seg) selection.DeselectAll() selection.Select(seg) for i =1,rwss.options.maxRebuildAttempts do structure.RebuildSelected(i) if current.GetEnergyScore()~=rwss.bestScore then rwss.rebuildScore=current.GetEnergyScore() if current.GetEnergyScore() > rwss.bestScore then print("Rebuild of segment "..tostring(seg).." improved score to "..rwss.GetScore()..".") end return true end end return false end function rwss.SphericalStabilize(seg) local radius = rwss.options.radius selection.DeselectAll() for i = 1,structure.GetCount() do if structure.GetDistance(seg,i)<=radius then selection.Select(i) end end -- this is a very simple Spherical Stabilize routine local wiggles=rwss.options.wiggles behavior.SetSlowFiltersDisabled(rwss.options.disableSlowFilters) behavior.SetClashImportance(1) structure.ShakeSidechainsSelected(1) structure.WiggleSelected((wiggles+1)/2) structure.WiggleAll((wiggles+1)/2) behavior.SetClashImportance(rwss.options.lowCI) structure.ShakeSidechainsSelected(1) structure.WiggleSelected(wiggles*2) behavior.SetClashImportance(1) structure.ShakeSidechainsAll(1) structure.WiggleAll(wiggles) behavior.SetSlowFiltersDisabled(false) -- the rest is administrative stuff about score keeping. recentbest.Restore() local currentScore=current.GetEnergyScore() if currentScore > rwss.bestScore then rwss.scores = rwss.GetScores() if currentScore > rwss.rebuildScore then print("Stabilize of segment "..tostring(seg).." improved score to "..rwss.GetScore()..".") end if currentScore > (rwss.bestScore + rwss.options.minRebuildPoints) then rwss.rebuildsRemaining=rwss.rebuildsRemaining - 1 print("Rebuild counted as a success. Need "..tostring(rwss.rebuildsRemaining).. " more.") end if currentScore > rwss.scoreBeforeRetrying then print("Reached goal score.") end if (rwss.rebuildsRemaining <= 0) or (currentScore > rwss.scoreBeforeRetrying) then rwss.rebuildsRemaining = rwss.options.rebuildsBeforeRetrying rwss.scoreBeforeRetrying=currentScore+rwss.options.minScorePoints print("Resetting list to try. New goal score is ".. tostring(rwss.scoreBeforeRetrying)..".") rwss.triedSeg={} end rwss.bestScore = currentScore return true else return false end end function rwss.Cleanup(message) print("Total gain: "..tostring(current.GetEnergyScore()-rwss.startingScore)) print(recipeTitle.." "..message..". Final Score: "..tostring(rwss.GetScore())) orig.Restore() end function rwss.Main () if not rwss.Setup() then orig.Restore() return true end local seg=rwss.GetSeg() while seg do rwss.triedSeg[seg]=true if rwss.RebuildSegment(seg) then if not rwss.SphericalStabilize(seg) then print("Rebuild of segment "..tostring(seg).." didn't improve things.") end else print("Couldn't rebuild segment "..tostring(seg)) end seg=rwss.GetSeg() end rwss.Cleanup('completed') return true end function rwss.EarlyExit(err) if rwss.earlyExited then print(err) -- handle error if already earlyExited else rwss.earlyExited=true -- prevent runaway in EarlyExit local message if err:find('Cancelled')~=nil then message = 'cancelled' else message = 'aborted' print(err) end behavior.SetSlowFiltersDisabled(false) recentbest.Restore() if current.GetEnergyScore() > rwss.bestScore then print("Score improved to "..rwss.GetScore().." as recipe was "..message..".") end rwss.Cleanup(message) end end xpcall(rwss.Main,rwss.EarlyExit)

Comments


GaryForbis Lv 1

This recipe is primarily a test bed for administrative routines around the functionality.

It is best for late in the game but can be use in middle game.

Version 1.0 Updated on: Mon, 11/24/2014 - 01:08
The option validation routines are stubs, don't try to enter bad numbers.
Several nifty options haven't been implemented.

Next version 1.1 will have:
Options: all Loops, only rebuild selected, use frozen as selected, leave frozen during run, leave bands enabled during run.

GaryForbis Lv 1

Version 1.1 Thu, 11/27/2014 - 00:21

  If it detects secondary structures it asks if you want to run all loops.
  If it detects selected segments it asks if you want to limit to them.
  If it detects frozen but not selected it asks if you want to limit to them.
  If it detects frozen it asks if you want to leave segments frozen.
  If it detects enabled bands it asks if you want to leave them enabled.

It returns frozen, bands, secondary structure, and selection as it found them.

This is primarily a late game script. I've found a larger sphere, 15 to 20, and higher low ci, .2 to .5 can produce some results.

PetWolverine Lv 1

Very effective script! Thanks for sharing this.

One thing I've noticed, though, is that it doesn't recalculate the scores when starting over in the list. This means that if, for example, it makes great gains on the first segment, such that it's no longer the worst, it will still start with it every time it reaches its goal and starts over.

GaryForbis Lv 1

You're right. Thanks. I'll fix this logic:


function rwss.SphericalStabilize(seg)
...
  -- the rest is administrative stuff about score keeping.
  recentbest.Restore()
  local currentScore=current.GetEnergyScore()
  if currentScore > rwss.bestScore then
    rwss.GetScores()
...
    rwss.bestScore = currentScore
    return true
  else
    return false
  end
end

offending line should be:


    rwss.scores = rwss.GetScores()

The table rwss.scores should have been getting updated every time there is a gain either during the rebuild or stabilizing function. This handles the case where segments switch place in the list during the cycle.

What you are seeing is that the same segment is still worst. There are three factors to reduce how often a segment is attempted to be rebuilt: 1) increase the number of successful rebuilds before clearing the table rwss.triedSeg; 2) increase the gain before a rebuild cycle is counted as successful; 3) increase the over all gain before clearing the table rwss.triedSeg.

In version 1.2 I will be adding logic to let you control the scoring function by subcomponents and/or delta relative to neighbor and/or sphere average.

The delta relative to neighbor will subtract the highest scoring seg+1 or seg-1 (if they exist) from the segment's score (i.e. segment score -5 and neighbor 5 would yield -10 score).

Sphere scoring will average the scores of the segments inside the sphere centered on the segment.

I currently do a wiggle all after the rebuild. I may change that to selecting the segment and the one on either side and doing a wiggle selected in bursts as long as it gains more than a certain number of points. There is something to be said for wiggling in bursts and checking the outcome but short bursts produces different behavior than the same number of cycles in a single wiggle.

I don't know if this complexity is worth it. I'll add what people request, including the local wiggle with freeze on either side as I borrowed from somewhere and put into gary Minima Finder years ago. For all I remember I may have developed from a mix of concepts in kawaga Repeat Settle and rav3n's wiggle routine in Lucky LWS.

GaryForbis Lv 1

Changes:

    fixed rebuild of rwss.scores.
    fixed when to display keep frozen check box.
    added alternate scoring functions.
    added ability to disable slow filters.

Helpful hints:

1.  You can observer more of what's going on by using the selection interface
    and watching the recipe output.
2.  The lower the low CI the more the sphere distorts the fold.
    Use .3 to .5 CI and larger radius, 15 to 20, to start.  Observe how close the recovery and adjust.
3.  I've added disable slow filters during the settling after rebuild but it's probably not worth it.
4.  Scoring sphere tends to score nearby segments similarly.
5.  Scoring difference from neighbor focuses on segments relatively bad compared to their neighbors.
6.  The wiggles at low CI are twice the wiggles at CI = 1 and you specify the wiggles at CI 1.
    Other recipes use 6.  Try different values to determine your preferences.
7.  If you have segments selected it will give you the option to only work on those segements.
8.  If you have segments frozen and none selected it will give you the option to work on those
    segments as if you selected them.

GaryForbis Lv 1

I've waffled about how to handle the mix of selected and frozen.
I don't remember what I finally did. I'd have to check to find out.
I don't know what other recipes don't clean up. I know gary Rebuild Worst and gary Minima Finder fail to clean up when cancelled, heck they don't even restore recent best.

While I prefer the original interface most of the time I'm starting to switch back and forth between them. When one switches to the selection interface one loses selections one had in the original interface. One of the features of the selection interface is notes appear as bubbles. I was using a note I kept in segment 1 to store the selected options. I did a Quicksave to slot 90 then repeated this when a solution gained points. When I switched to the selection interface the note was plastered across the screen. I realized I had to clear the note after the Quicksave so it wasn't junking up the screen. I've thought about using this feature of the selection interface to show people the first 10 segments selected by a particular set of selection options. I may add that and saving/retrieving the options from one of the save slots to version 1.3. If I retrieve the settings I'll need to add a "reset defaults" button.