Georeactor Blog

Pangenome LLMs exist

In September 2025 I posted about TomatoTomato, an LLM which I trained on two tomato genomes simultaneously. When I'm reading the post and model README with fresh eyes, I'm not super happy with my text. But essentially:

• a pangenome collects multiple genomes to capture variance in a population
• instead of training a model on one genome and then another very similar one, I align two sequences and encode them in input tokens (A-A, A-C, A-G, A-T, A-none

Unfortunately this would be difficult to scale to 5+ genomes, so I wrote up some ideas about leveraging loss to express the ratio of options.

A few weeks ago I was surprised to read a paper on bioRxiv about TomatoPGFM, "a graph-conditioned model" trained on 66 tomato genomes by researchers in China. I anticipated that researchers would be drawn to a pangenome, and obviously a professional or academic collaborative effort would have a better product.
Claude tells me that graph data is encoded in a different way than I expected (similar to the position of words being encoded alongside tokens in text). That has got to be position data because revealing other genomes' sequences would be too easy to copy to predict the next token in the reference genome.
There are some oddities: each token is 6-nucleotides long, like AgroNT. Their model's evals are above DNABERT-2, but below PlantDNAMamba. Their cited research papers are often from other Chinese labs (OneGenome-Rice and OryzaG3).

Of course it would be nice if someone threw in a link to TomatoTomato as related work, but no one has ever cited my bio projects, so I'm not part of that system so much.

Quinoa

Also this fall, Frontiers in Plant Science published an article about the quinoa pangenome based on 11 genomes (which I think you can all find on QuinoaDB.org). I have wanted to do something new with chenopodium.org to include huazontle and cañihua from the same genus, so I was feeling a little extra left-out.

One thing which I hadn't considered in my original process is that the pangenome analysis removes transposable elements / transopons. There are some cases where these affect traits of their organisms (a recent paywalled paper is titled Deflecting the parasitic paradigm: new insights into mutualistic transposons in plant genomes), but the conventional view is that transopons mostly help themselves exist, so:

Transcripts containing transposable elements were subsequently removed from the gene sets.

The paper uses the multiple samples to find genes which might be the most valuable:

these positively selected genes may represent candidate molecular adaptations contributing to quinoa's environmental resilience

… and to identify what regions are the most similar across accessions. I've previously discussed how quinoa is polyploid (where we have two copies of each chromosome, quinoa merged two species and has two from species A and two from species B). In this case they confirm a trait in other polyploid species:

one subgenome dominates in gene retention

Here are their future goals:

Future research should focus on functional validation of candidate adaptive genes, exploration of epigenetic regulatory mechanisms, and integration of pan-genomic data into molecular breeding programs to improve quinoa's yield, quality, and stress tolerance.

Inspiration

Attribution, inspiration, and collaboration are all tied together.
I know, or think I know, the norms around sharing code and knowledge in the coding space. There are places for open source and things that I'm not supposed to talk about. Science is going to have its own norms. Who do I need to read, or to reach out to? Do I publish a more evolved pangenome model, an interactive that consumes bio papers and highlights their locations on the genome…? Do I need to publish my own paper with someone on bioRxiv? Who do I reach out to and set up a collaboration? Whatever happened to the Duluth professor? Etc.

Instead of me pretending to know a lot about those, lets talk music:

While buried in my latest work project, I listened to so many Olivia Rodrigo tracks, that I now get Olivia-centric credit card and fast food ads. Sorry I am basic, and I'm down the rabbit hole.

In the past Rodrigo was plagued with less friendly interactions -three songs added writing credits to Taylor Swift, and one to Paramore (UNC law school article) in part because she was quoted comparing her songs to theirs. The author of the law article notes the long and difficult path of bringing a music case through court, so it's in both teams' interest to settle a challenge.
These days sometimes a friendly call does the work. Rodrigo reaches out to the artists, a Gen Z daughter or associate will underline the importance of this connection, and next thing you know they have a collab. Chris Butler recently posted a note about Rodrigo's outreach ahead of her take me home (4 christmas), and Sarah McLachlan performed at Rodrigo's Daisy Chain Fields festival, which was inspired by MacLachlan's own 90s female-forward festival Lilith Fair.
The new tactics are working.

So maybe what I want to do in the next 2–3? months is have something interesting to share with the TomatoPGFM team, the Duluth professor, the quinoa team, maybe pointing me in a good direction? And this is really a lot on me… I have to figure out what would prove that a pangenome model has value, and that I'm not just churning LLM slop.