2360: CACHE SARS Nsp3 Reranking challenge: Round 1

Closed since over 2 years ago

Summary

• Created: September 21, 2023
• Expires: October 06, 2023 at 23:00 UTC
• Max points: 100

Description

This puzzle is a followup to the SARS-CoV-2 Nsp3 macrodomain design challenges. Help us rank potential active compounds!

Phase 2 of the CACHE challenge #3 is a ranking of all 1739 molecules which the various participants submitted for testing. In this puzzle you don't have free design with the Small Molecule Design tool, but you instead have the Ligand Queue which will provide you with a small number of possible compounds. You can then use the compound library panel to find related compounds specifically from the library of tested compounds. Then attempt to find the best binding position for the compounds.

Note: To get the most out of the small molecule design tools, we recommend changing you view settings to the Small Molecule Design Preset.

Ranking designed compounds and figuring out which bind are an important part of designing new molecules, and with this puzzle, we're testing out how well the Ligand Queue and Compound Library tools work for evaluating medium-sized libraries. Bounce back and forth between the ligand queue and the compound library search to seek out compounds you think bind best. (Note that all compounds should be accessible within two compound library jumps from a ligand queue molecule.)

Participation in CACHE puzzles is subject to the CACHE Terms of Participation, in particular “the Challenge IP [including Challenge Compounds] will be made freely available in the public domain pursuant to Creative Commons Attribution Only (CC-BY 4.0 or subsequent versions) licensing terms, with the intent that such Challenge IP may be Used and practiced by Users for any purpose”.

Top groups

• 1. Contenders 100 pts. 16,389
• 2. Anthropic Dreams 71 pts. 16,280
• 3. Go Science 49 pts. 15,964
• 4. Void Crushers 33 pts. 14,939
• 5. AlphaFold 22 pts. 14,891
• 6. Gargleblasters 14 pts. 14,886
• 7. Australia 8 pts. 14,879
• 8. L'Alliance Francophone 5 pts. 14,836
• 9. FoldIt@Netherlands 3 pts. 14,809
• 10. FamilyBarmettler 2 pts. 14,767

Top soloists

• 1. BootsMcGraw 100 pts. 16,389
• 2. Galaxie 94 pts. 16,264
• 3. Sandrix72 88 pts. 15,703
• 4. Artoria2e5 83 pts. 15,430
• 5. Bruno Kestemont 78 pts. 15,330
• 6. ZeroLeak7 73 pts. 15,310
• 7. dcrwheeler 68 pts. 15,193
• 8. MicElephant 63 pts. 15,127
• 9. ucad 59 pts. 15,126
• 10. LociOiling 55 pts. 15,109

Comments

[LociOiling]

After some more testing, I'm finding that restarting Foldit gets you a new shuffle and a new draw, resulting in a new ligand queue. Some of the entries in the new queue may be the same as ones in a previous queue, but they will probably be in a different position. Other entries may be different.

It doesn't appear that the ligand queue is a random pick from 1,739 entries. Too many entries remain the same, and some entries seem to appear, disappear, and reappear. So maybe there's a smaller list of compounds being used for this puzzle.

Resetting the puzzle may affect the first ligand queue entry, resetting it to the compound with weight 411.414 Da. Entries 2 to 30 are not changed by a reset.

Reloading the puzzle without a restart also doesn't change entries 2 to 30. Entry 1 will be the last molecule opened before the reload.

So far, I haven't been able to load a different puzzle, then reload 2360. Trying to load 2358 after 2360 causes a crash.

[jeff101]

If there will be another puzzle like this, why not give each ligand a number like 1 to 1739 in the ligand queue and in the library tool? Also, please tell us which of the 1739 ligands is the starting one. You can still list similarity scores, molecular weights, and other ligand properties, but having the #’s from 1 to 1739 would give us a pretty simple way to tell things apart. Perhaps adding a lua command to read this # would help too. A lua command to pick the ligand using the # 1 to 1739 would help too.

[jeff101]

If it is possible to make compounds not in the desired list of 1739 for this puzzle, can you eliminate the scores for these non-desired compounds from the scoreboards and rankings for this puzzle?

[BootsMcGraw]

Hey, Foldit Overlords. Are you going to pull this puzzle and re-release it with the game upgrade intact? Or are you going to let me keep my (undeserved) first place ranking?

[LociOiling]

After still more experimentation, the crash when switching puzzles happens when any of the small molecule properties are selected. Deselecting all properties lets you swap 2360 and 2358, then reload 2360.

Reloading has no effect on ligand queue entries 2 to 30. They'll be the same as before you switched puzzles. Ligand queue entry 1 will be whatever molecule was in view when you switched puzzles.

[rmoretti]

Is the starting ligand for puzzle 2360 one of the ligands you want us to dock?

It isn't. It's just provided as a convenient "known" starting point. (We don't have experimental structures for any of the ligands to redock.) The staring ligand should not be present in the Compound Library results, nor in the ligand queue panel (except potentially for the "initial" entry.)

the rest of the ligand queue is not fixed.

That's deliberate. To avoid giving too much emphasis on the ligands earlier in the list, we randomize the order. We tried to make the ordering consistent for any individual run (so resetting/reloading the puzzle shouldn't change the order), but if you have other clients and/or restart, the list may get re-randomized. (Different people having different orders is completely intentional.)

ligand queue entries 2 to 30 are perhaps a random sample of a larger population of molecules

That shouldn't be the case. For this puzzle, there should be the same set of 29(+1) compounds which are reordered for the ligand queue.

The entry #1 is the "inital" molecule when you launched the ligand queue tool (this time), so it will change based on the state of the structure at the time when you pressed the ligand queue button.

the "ligand ID" for ligand queue entries 2 to 30 doesn't seem to be in SMILES format

They should indeed be SMILES. We heard the concerns that the arbitrary IDs in e.g. the Compound Library panel were hard to work with, so we attempted to use SMILES for the IDs in the ligand queue. – That said, there are multiple ways to write a SMILES string for the same compound, so the SMILES used in the ligand queue may or may not match that listed in the log file. I'm not sure what Jmol isn't liking about the one used in for the Ligand ID. (The inability to copy/paste the ID may be complicating things.)

[rmoretti]

@jeff101 can you eliminate the scores for these non-desired compounds from the scoreboards and rankings for this puzzle?

@BootsMcGraw Are you going to pull this puzzle and re-release it … Or are you going to let me keep my (undeserved) first place ranking?

I've been contemplating how best to do this. I think I could add a Compound Library objective to the puzzle with sufficient weight to make any match better than the current high score, though I think this would mean that everyone would have to restart their clients (to pull down the updated objective definitions) and then re-load their top scoring structures, as well as submit the query to the compound library to get it "approved".

I certainly will have the Compound Library objective on during the next round, but I don't know if people think that "fixing" the current round warrants the hassle – but I'd be guided by your opinions.

[jeff101]

You could make this round worth zero global points but let us load our results from it into the next round.

[BootsMcGraw]

As much as I'd love a #1 on a puzzle, I am in full agreement with jeff101. Make # 2360 worth zero points, but let us keep our structures that meet the puzzle guidelines.

[LociOiling]

@rmoretti They should indeed be SMILES.

Yes, those are SMILES, it's just copying them by hand is not a guarantee of accuracy. I was using 0 (zero) in spots where I meant O for oxygen, which explains the problems I was having. I blame the font.

The "fluffy" version of the SMILES string also appears in log.txt. This version explicitly lists all the hydrogens.

The SMILES strings in log.txt can of course be copied and pasted. The problem is that log.txt is buffered, meaning it doesn't get written to disk immediately. Closing Foldit makes sure that you have the complete log.txt to look at. Even then, it can be difficult to determine which compound is which.