Icon representing a recipe

Recipe: MicroIdealize 4.0 test

created by LociOiling

Profile


Name
MicroIdealize 4.0 test
ID
102586
Shared with
Public
Parent
MicroIdealize 4.0
Children
Created on
December 19, 2017 at 17:42 PM UTC
Updated on
December 19, 2017 at 17:42 PM UTC
Description

This version deals with locked segments. It's designed to test for conditions observed in the aflatoxin puzzles.

Best for


Code


--[[ MicroIdealize 4.0 Test version - LociOiling - 2017/12/18 For use on puzzle 1440 and perhaps other aflatoxin puzzles, attempt to determine whether idealizing locked segments has any effect. Lots of TimoTech inserted to make it happen. Also some Loci code Maybe some other code got hurt. It happens. ]]-- Recipe = "Microidealize" Version = "4.0 test" ReVersion = Recipe .. " " .. Version kOriginalStructureOrNewBest = 1 -- the starting structure, or any subsequent improvement, will be stored in this quicksave slot best_score = 0 idealize_length = 3 min_residue = 1 max_residue = 999 worst_first = true score_type = 1 original_slow_filters_setting = 0 kDefaultWiggle = 12 kSphereRadius = 10 segCnt = 0 segCnt2 = 0 dolocked = false -- if locked sections present, work on locked sections only locksphr = false -- if locked sections present, include locked sections in wiggle sphere ulcksphr = true -- if locked sections present, include unlocked sections in wiggle sphere lockedL = {} -- list of locked segments lockedS = {} -- set of locked segments unlockedS = {} -- set of unlocked segments idealz = {} -- table of ideality by segment -- -- worklist offsets -- WLSEG1 = 1 WLSEG2 = 2 WLIDEAL = 3 WLAFTER = 4 -- -- ideal list offsets -- IDSEG = 1 IDLCK = 2 IDIDEAL = 3 IDAFTER = 4 function r3 ( i ) return i - i % 0.001 end -- -- Segment set and list module v0.3 -- -- Find lists and sets of segments with various properties, such as selected or frozen. -- -- A "list" is one-dimensional table containing segment numbers. -- -- A "set" is a two-dimensional table containing segment number ranges. -- -- For example, given a list of segments: -- -- list = { 1, 2, 3, 7, 8, 11, 13, 14, 15 } -- -- the corresponding set is: -- -- set = { { 1, 3 }, { 7, 8 }, { 11, 11 }, {13, 15 } } -- -- Most functions assume that the sets are well-formed, -- meaning they are ordered and have no overlaps -- -- 02-05-2012 TvdL Free to use for non commercial purposes -- -- v0.2 - LociOiling - 2017/11/03 -- + add primary FindUnlocked function -- v0.3 - LociOiling -- + add FindRotamers function -- function SegmentListToSet ( list ) -- retirer doublons local result = {} local f = 0 local l = -1 table.sort ( list ) for ii = 1, #list do if list [ ii ] ~= l + 1 and list [ ii ] ~= l then -- note: duplicates are removed if l > 0 then result [ #result + 1 ] = { f, l } end f = list [ ii ] end l = list [ ii ] end if l > 0 then result [ #result + 1 ] = { f, l } end return result end function SegmentSetToList ( set ) -- faire une liste a partir d'une zone local result = {} for ii = 1, #set do for k = set [ ii ] [ 1 ], set [ ii ] [ 2 ] do result [ #result + 1 ] = k end end return result end function SegmentCleanSet ( set ) -- Makes it well formed return SegmentListToSet ( SegmentSetToList ( set ) ) end function SegmentInvertSet ( set, maxseg ) -- Gives back all segments not in the set -- maxseg is added for ligand local result={} if maxseg == nil then maxseg = structure.GetCount () end if #set == 0 then return { { 1, maxseg } } end if set [ 1 ] [ 1 ] ~= 1 then result [ 1 ] = { 1, set [ 1 ] [ 1 ] - 1 } end for i = 2, #set do result [ #result + 1 ] = { set [ i - 1 ] [ 2 ] + 1, set [ i ] [ 1 ] - 1} end if set [ #set ] [ 2 ] ~= maxseg then result [ #result + 1 ] = { set [ #set ] [ 2 ] + 1, maxseg } end return result end function SegmentInvertList(list) table.sort(list) local result={} for i=1,#list-1 do for j=list[i]+1,list[i+1]-1 do result[#result+1]=j end end for j=list[#list]+1,segCnt2 do result[#result+1]=j end return result end function SegmentInList ( s, list ) -- verifier si segment est dans la liste table.sort ( list ) for ii = 1, #list do if list [ ii ] == s then return true elseif list [ ii ] > s then return false end end return false end function SegmentInSet ( set, s ) --verifie si segment est dans la zone for ii = 1, #set do if s >= set [ ii ] [ 1 ] and s <= set [ ii ] [ 2 ] then return true elseif s < set [ ii ] [ 1 ] then return false end end return false end function SegmentJoinList ( list1, list2 ) -- fusionner 2 listes de segments local result = list1 if result == nil then return list2 end for ii = 1, #list2 do result [ #result + 1 ] = list2 [ ii ] end table.sort ( result ) return result end function SegmentJoinSet ( set1, set2 ) --fusionner (ajouter) 2 zones return SegmentListToSet ( SegmentJoinList ( SegmentSetToList ( set1 ), SegmentSetToList ( set2 ) ) ) end function SegmentCommList ( list1, list2 ) -- chercher intersection de 2 listes local result = {} table.sort ( list1 ) table.sort ( list2 ) if #list2 == 0 then return result end local j = 1 for ii = 1, #list1 do while list2 [ j ] < list1 [ ii ] do j = j + 1 if j > #list2 then return result end end if list1 [ ii ] == list2 [ j ] then result [ #result + 1 ] = list1 [ ii ] end end return result end function SegmentCommSet ( set1, set2 ) -- intersection de 2 zones return SegmentListToSet ( SegmentCommList ( SegmentSetToList ( set1 ), SegmentSetToList ( set2 ) ) ) end function SegmentSetMinus ( set1, set2 ) return SegmentCommSet ( set1, SegmentInvertSet ( set2 ) ) end function SegmentPrintSet ( set ) print ( SegmentSetToString ( set ) ) end function SegmentSetToString ( set ) -- pour pouvoir imprimer local line = "" for ii = 1, #set do if ii ~= 1 then line = line .. ", " end line = line .. set [ ii ] [ 1 ] .. "-" .. set [ ii ] [ 2 ] end return line end function SegmentSetInSet ( set, sub ) if sub == nil then return true end -- Checks if sub is a proper subset of set for ii = 1, #sub do if not SegmentRangeInSet ( set, sub [ ii ] ) then return false end end return true end function SegmentRangeInSet ( set, range ) --verifier si zone est dans suite if range == nil or #range == 0 then return true end local bb = range [ 1 ] local ee = range [ 2 ] for ii = 1, #set do if bb >= set [ ii ] [ 1 ] and bb <= set [ ii ] [ 2 ] then return ( ee <= set [ ii ] [ 2 ] ) elseif ee <= set [ ii ] [ 1 ] then return false end end return false end function SegmentSetToBool ( set ) --vrai ou faux pour chaque segment utilisable ou non local result = {} for ii = 1, structure.GetCount () do result [ ii ] = SegmentInSet ( set, ii ) end return result end --- End of Segment Set module -- Module Find Segment Types function FindMutablesList () local result = {} for ii = 1, segCnt2 do if structure.IsMutable ( ii ) then result [ #result + 1 ] = ii end end return result end function FindMutables() return SegmentListToSet ( FindMutablesList () ) end function FindFrozenList () local result = {} for ii = 1, segCnt2 do if freeze.IsFrozen ( ii ) then result [ #result + 1 ] = ii end end return result end function FindFrozen () return SegmentListToSet ( FindFrozenList () ) end function FindLockedList () local result = {} for ii = 1, segCnt2 do if structure.IsLocked ( ii ) then result [ #result + 1 ] = ii end end return result end function FindLocked () return SegmentListToSet ( FindLockedList () ) end function FindUnlockedList () local result = {} for ii = 1, segCnt2 do if not structure.IsLocked ( ii ) then result [ #result + 1 ] = ii end end return result end function FindUnlocked () return SegmentListToSet ( FindUnlockedList () ) end function FindZeroScoreList () local result = {} for ii = 1, segCnt do local sub = 0 for jj = 1, #SubScores do sub = sub + current.GetSegmentEnergySubscore ( ii, SubScores [ jj ] [ 1 ] ) end if sub == 0 then result [ #result + 1 ] = ii end end return result end function FindZeroScore () return SegmentListToSet ( FindZeroScoreList () ) end function FindRotamersList () local result = {} for ii = 1, segCnt do local rots = rotamer.GetCount ( ii ) if rots > 1 then result [ #result + 1 ] = ii end end return result end function FindRotamers () return SegmentListToSet ( FindRotamersList () ) end function FindSelectedList () local result = {} for ii = 1, segCnt do if selection.IsSelected ( ii ) then result [ #result + 1 ] = ii end end return result end function FindSelected() return SegmentListToSet ( FindSelectedList () ) end function FindAAtypeList ( aa ) local result = {} for ii = 1, segCnt2 do if structure.GetSecondaryStructure ( ii ) == aa then result [ #result + 1 ] = ii end end return result end function FindAAtype ( aa ) return SegmentListToSet ( FindAAtypeList ( aa ) ) end function FindAminotype ( at ) --NOTE: only this one gives a list not a set local result={} for ii = 1, segCnt2 do if structure.GetAminoAcid ( ii ) == at then result [ #result + 1 ] = ii end end return result end -- -- end Segment set and list module v0.3 -- -- Thanks too Rav4pl function ShuffleTable(tab) --randomize order of elements local cnt=#tab for i=1,cnt do local r=math.random(cnt) tab[i],tab[r]=tab[r],tab[i] end return tab end function GetScore () if ( score_type == 1 ) then score = current.GetEnergyScore () elseif ( score_type == 2 ) then behavior.SetFiltersDisabled ( false ) score = current.GetScore () behavior.SetFiltersDisabled ( true ) end return score end function DoesPuzzleHaveFilters () local init_setting = behavior.GetFiltersDisabled () if ( init_setting == false ) then score_without_sf = current.GetScore () behavior.SetFiltersDisabled ( true ) score_with_sf = current.GetScore () else score_with_sf = current.GetScore () behavior.SetFiltersDisabled ( false ) score_without_sf = current.GetScore () end behavior.SetFiltersDisabled ( init_setting ) if ( math.abs ( score_without_sf - score_with_sf ) > 1e-3 ) then return true else return false end end function SphereSelect ( start_idx, end_idx ) selection.DeselectAll () for ii = 1, segCnt2 do for jj = start_idx, end_idx do if lockedL [ ii ] then if locksphr then if structure.GetDistance ( ii , jj ) < kSphereRadius then selection.Select ( ii ) end end else if ulcksphr then if structure.GetDistance ( ii , jj ) < kSphereRadius then selection.Select ( ii ) end end end end end end -- -- GetWorklist -- -- return set of segments to work on -- -- sSet - segment set -- lenX - length -- gMinR - minimum residue -- gMaxR - maximum residue -- function GetWorklist ( sSet, lenX, gMinR, gMaxR ) local lsx = {} for uu = 1, #sSet do local minR = sSet [ uu ] [ 1 ] local maxR = sSet [ uu ] [ 2 ] if ( #sSet == 1 and minR >= gMinR and maxR <= gMaxR ) or ( #sSet > 1 and minR > gMinR and maxR <= gMaxR ) then for ii = minR, maxR - lenX + 1 do lsx [ #lsx + 1 ] = { ii, ii + lenX - 1, nil, nil, nil } end end end return lsx end function GoSegment ( start_idx, end_idx ) save.Quickload ( kOriginalStructureOrNewBest ) if ( start_idx > 1 ) then structure.InsertCut ( start_idx ) end if ( end_idx < segCnt2 ) then structure.InsertCut ( end_idx ) end selection.DeselectAll () selection.SelectRange ( start_idx , end_idx ) structure.IdealizeSelected () if ( start_idx > 1 ) then structure.DeleteCut ( start_idx ) end if ( end_idx < segCnt2 ) then structure.DeleteCut ( end_idx ) end SphereSelect ( start_idx, end_idx ) structure.WiggleSelected ( kDefaultWiggle ) local score = GetScore () if ( score > best_score ) then best_score = score print ( "Improvement to ".. r3 ( best_score ) ) save.Quicksave ( kOriginalStructureOrNewBest ) end end function ScoreIdealitySection ( seg1, seg2 ) local ideal = 0 for ii = seg1, seg2 do ideal = ideal + current.GetSegmentEnergySubscore ( ii, "ideality" ) end return ideal end function ScoreIdeality ( wlist ) for ii = 1, #wlist do wlist [ ii ] [ WLIDEAL ] = ScoreIdealitySection ( wlist [ ii ] [ WLSEG1 ], wlist [ ii ] [ WLSEG2 ] ) end return wlist end function SortIdealAsc ( xyzzy ) table.sort ( xyzzy, function ( a, b ) return a [ WLIDEAL ] < b [ WLIDEAL ] end) return xyzzy end function Idealize () local inc local worklist = {} if dolocked then worklist = GetWorklist ( lockedS, idealize_length, min_residue, max_residue ) else worklist = GetWorklist ( unlockedS, idealize_length, min_residue, max_residue ) end worklist = ScoreIdeality ( worklist ) if worst_first then worklist = SortIdealAsc ( worklist ) else worklist = ShuffleTable ( worklist ) end for ii = 1, #worklist do print ( "Idealizing " .. worklist [ ii ] [ WLSEG1 ] .. "-" .. worklist [ ii ] [ WLSEG2 ] .. ", ideality = " .. r3 ( worklist [ ii ] [ WLIDEAL ] ) .. " (" .. ii .. "/" .. #worklist .. ")" ) GoSegment ( worklist [ ii ] [ WLSEG1 ], worklist [ ii ] [ WLSEG2 ] ) worklist [ ii ] [ WLAFTER ] = ScoreIdealitySection ( worklist [ ii ] [ WLSEG1 ], worklist [ ii ] [ WLSEG2 ] ) local ggain = worklist [ ii ] [ WLAFTER ] - worklist [ ii ] [ WLIDEAL ] if ggain >= 0.001 then print ( "ideality gained " .. r3 ( ggain ) ) end end end function GetParameters () local dlog = dialog.CreateDialog ( ReVersion ) dlog.idealize_length = dialog.AddSlider ( "Idealize length" , idealize_length , 1 , 20 , 0 ) dlog.min_residue = dialog.AddSlider ( "Min residue" , 1 , 1 , segCnt2 , 0 ) dlog.max_residue = dialog.AddSlider ( "Max residue" , segCnt2 , 1 , segCnt2 , 0 ) dlog.worst_first = dialog.AddCheckbox ( "Worst first?" , worst_first ) if #lockedS > 0 then dlog.dolocked = dialog.AddCheckbox ( "Do locked segments only?" , dolocked ) dlog.locksphr = dialog.AddCheckbox ( "Include locked in wiggle?" , locksphr ) dlog.ulcksphr = dialog.AddCheckbox ( "Include unlocked in wiggle?" , ulcksphr ) end if DoesPuzzleHaveFilters () == true then score_type = 2 end dlog.score_type = dialog.AddSlider ( "Score type" , score_type , 1 , 2 , 0 ) dlog.ok = dialog.AddButton ( "OK" , 1 ) dlog.cancel = dialog.AddButton ( "Cancel" , 0 ) if ( dialog.Show ( dlog ) > 0 ) then idealize_length = dlog.idealize_length.value min_residue = dlog.min_residue.value max_residue = dlog.max_residue.value worst_first = dlog.worst_first.value score_type = dlog.score_type.value if #lockedS > 0 then dolocked = dlog.dolocked.value locksphr = dlog.locksphr.value ulcksphr = dlog.ulcksphr.value end return true else return false end end -- -- Ident - print identifying information at beginning and end of recipe -- -- slugline - first line to print - normally recipe name and version -- function Ident ( slugline ) local function round ( ii ) return ii - ii % 0.001 end print ( slugline ) print ( "Puzzle: " .. puzzle.GetName () .. " (" .. puzzle.GetPuzzleID () .. ")" ) print ( "Track: " .. ui.GetTrackName () ) gname = user.GetGroupName () if gname ~= nil then gname = " (" .. gname .. ")" else gname = "" end print ( "User: " .. user.GetPlayerName () .. gname ) local scoretype = scoreboard.GetScoreType () local scort = "" if scoretype == 0 then scort = "soloist" elseif scoretype == 1 then scort = "evolver" elseif scoretype == 2 then scort = "all hands" elseif scoretype == 3 then scort = "no score" else scort = "unknown/error" end print ( "Rank: " .. scoreboard.GetRank ( scoretype ) .. " (" .. scort .. ")" ) local sGroup = scoreboard.GetGroupScore () if sGroup ~= nil then print ( "Group rank / score: " .. scoreboard.GetGroupRank () .. " / " .. round ( 10 * ( 800 - sGroup ) ) ) end end function Initialize () segCnt = structure.GetCount() segCnt2 = segCnt while structure.GetSecondaryStructure ( segCnt2 ) == "M" do segCnt2 = segCnt2 - 1 end print ( "initial segment count: " .. segCnt ) if segCnt ~= segCnt2 then print ( "ligand found, adjusted segment count: " .. segCnt2 ) end for ii = 1, segCnt2 do lockedL [ #lockedL + 1 ] = structure.IsLocked ( ii ) end lockedS = FindLocked () print ( #lockedS .. " locked sections found" ) unlockedS = FindUnlocked () print ( #unlockedS .. " unlocked sections found" ) -- -- record locked status and ideality of each segment -- for ii = 1, segCnt2 do local ideal = current.GetSegmentEnergySubscore ( ii, "ideality" ) idealz [ #idealz + 1 ] = { ii, lockedL [ ii ], ideal, 0, } end end function main () Ident ( ReVersion ) save.Quicksave ( kOriginalStructureOrNewBest ) best_score = GetScore () print ( "Start score " .. r3 ( best_score ) ) Initialize () original_slow_filters_setting = behavior.GetFiltersDisabled () if ( GetParameters () == false ) then return -- graceful exit end if ( score_type == 2 ) then behavior.SetFiltersDisabled ( true ) end print ( "Idealize length: " .. idealize_length ) if #lockedS > 0 then print ( "Idealize locked: " .. tostring ( dolocked ) ) print ( "Include locked in sphere: " .. tostring ( locksphr ) ) print ( "Include unlocked in sphere: " .. tostring ( ulcksphr ) ) end print ( "Worst first: " .. tostring ( worst_first ) ) print ( "--" ) Idealize () for ii = 1, segCnt2 do idealz [ ii ] [ IDAFTER ] = current.GetSegmentEnergySubscore ( ii, "ideality" ) end print ( "--spreadsheet output--" ) print ( "segment\tlocked\tbefore\tafter\tdiff" ) for ii = 1, #idealz do local locked if lockedL [ idealz [ ii ] [ IDSEG ] ] then locked = "L" else locked = "U" end local before = r3 ( idealz [ ii ] [ IDIDEAL ] ) local after = r3 ( idealz [ ii ] [ IDAFTER ] ) local diff = r3 ( after - before ) print ( idealz [ ii ] [ IDSEG ] .. "\t" .. locked .. "\t" .. before .. "\t" .. after .. "\t" .. diff ) end cleanup () end function cleanup ( errmsg ) if CLEANUPENTRY ~= nil then return end CLEANUPENTRY = true print ( "---" ) local reason local start, stop, line, msg if errmsg == nil then reason = "complete" else -- -- model 120 - civilized error reporting, -- thanks to Bruno K. and Jean-Bob -- start, stop, line, msg = errmsg:find ( ":(%d+):%s()" ) if msg ~= nil then errmsg = errmsg:sub ( msg, #errmsg ) end if errmsg:find ( "Cancelled" ) ~= nil then reason = "cancelled" else reason = "error" end end -- -- model 100 - print recipe name, puzzle, track, time, score, and gain -- Ident ( ReVersion .. " " .. reason ) if reason == "error" then print ( "Unexpected error detected" ) print ( "Error line: " .. line ) print ( "Error: \"" .. errmsg .. "\"" ) end -- -- model 130 - reset clash importance, clear selections, restore structures, etc. -- behavior.SetClashImportance ( 1.0 ) behavior.SetFiltersDisabled ( original_slow_filters_setting ) save.Quickload ( kOriginalStructureOrNewBest ) if errmsg == nil then end end --main () xpcall ( main, cleanup )

Comments


LociOiling Lv 1

In the first aflatoxin challenge puzzle, Puzzle 1440, some players observed ideality changes in locked segments.

This version of Microidealize looks at the issue in detail, checking for locked segments. If no locked segments are found, the recipe should behave almost like the base version, Microidealize 4.0.

If locked segments are detected, the recipe offers three new options:

  • Do locked segments only?
  • Include locked in wiggle?
  • Include unlocked in wiggle?

The "do locked" option is unchecked by default, meaning the recipe attempts to idealize only the unlocked segments.

The "include locked" option is unchecked by default, meaning the recipe does not attempt to wiggle any locked segments after idealizing.

The "include unlocked" option is checked by default, meaning the recipe wiggles unlocked segments around the idealized section.

So far, this version has only been tested on Puzzle 1440. With all three of the options checked, the ideality scores of the locked segments immediately before or immediately after an unlocked section can change. From the starting position of 1440, locked segment 291 gained nearly 20 points in ideality with all three of these options checked.

The recipe includes a tab-delimited report of the ideality changes for each segment, which is produced at the end of the recipe. The report can be copied from the scriptlog file and pasted into a spreadsheet.

(The recipe print protein 2.7 can be used before and after this recipe for more detailed analysis, but unfortunately it doesn't identify locked segments. Look for an updated print protein 2.8 in the near future.)

Some limited additional testing shows that the gains from idealizing locked segments are all from wiggling the adjacent unlocked segments. If "do locked" is checked, and "include unlocked" is not checked, the recipe concludes very quickly, and no gain is reported. This is because unlocked segments can't be selected, and idealize applies only to selected segments. With "do locked" checked, the idealize function (structure.IdealizeSelected) doesn't do anything. The recipe attempts to select locked segments, but this has no effect. This means that "include locked in wiggle" also does nothing.

To summarize, the subscores of locked segments in puzzle 1440 can change due to changes in nearby locked segments. For example, the clashing subscore of a locked segment can improve is shake moves an unlocked sidechain away from it. Changes in ideality of a locked segment seem to result from changes to nearby unlocked segments.

The testing so far does not necessarily explain all issues seen with puzzle 1440. Some players have reported seeing backbone issues in locked sections when the "show backbone issues" view option is selected. I did not observe any backbone issues on locked segments, but there were lots of them on the unlocked segments.

As a side note, on some puzzles with locked segments, the sidechains of some locked segments are actually unlocked. This does not appear to be the case in puzzle 1440. Using "ligand specific" coloring, all the sidechains of the locked segments in 1440 appear in the same gray color as the backbone. Any unlocked sidechains would appear in the score-based coloring seen on the unlocked segments. There is no direct way for a recipe to detect unlocked sidechains of a locked segment.

Internally, this version of the recipe uses the segment set and list functions developed by Timo van der Laan, lifted and slightly modified from his EDRW family of recipes. These functions are used during initialization, and there's a little duplicated effort there. The sets of locked or unlocked segments are used to generate "work lists". The work lists are sets of segment ranges to be idealized.

There are also some other minor internal changes. The use of work lists made it easy to combine two different "Go" functions into one. Table sorting has been replaced by Rav3n_pl's "ShuffleTable", and a standard Lua "table.sort". The cleanup method has been updated to a standard version to provide error reporting. The recipe now uses standard "Timo logic" to detect ligands at the end of a protein. Any such ligands are automatically excluded.

The logic used for detecting locked segments can be extended to other recipes. In general, just working on the unlocked segments may be good enough for most puzzles. A complete method would need to allow for locked segments with unlocked sidechains, and ligands with multiple rotamers, which have been seen in some recent puzzles.