ESM3 IS OUT: A FIRST SUMMARY

On June 25th 2024 the ESM authors placed a preprint online (also on BioRxiv1 on July 2nd) titled “Simulating 500 million years of evolution with a language model”. Key takeaways Using a masked language model, not an autoregressive model, with multiple tracks (sequence, structure) allows one to generate samples from any track Diffusion is not used for sampling structures, unlike Alphafold32. The tokenization, and generative procedure is performed via an attentive VQVAE model with iterative decoding scheme.