As a Research Software Engineer, I work with Professor Reed Maxwell’s group, helping scientists better understand how water moves through the environment. Our lab studies computational hydrology, which means we use mathematical models to simulate how groundwater and surface water flow across landscapes. These simulations help researchers explore questions related to current groundwater availability, future scenario predictions, and long‑term water sustainability.

Recently our team, together with collaborators at the University of Arizona, has been building a continental‑scale model of water movement across the entire United States using the integrated groundwater model ParFlow. A model this large produces quite a bit of data, and one of my goals was to make it easier for the research team to explore and interpret the results. To help with this, I built a small interactive dashboard using Dash, a Python framework that lets you create web apps with interactive plots and maps. The dashboard lets us directly compare model results with real‑world observations: for example, streamflow measurements from monitoring stations. This helps researchers quickly see where the model is performing well and where it may need adjustment.

While there were some quirks to get the initial dashboard set up, the step that proved to be more challenging was in how to share it with all of the collaborators.

Our group works on a high‑performance computing (HPC) cluster, because our simulations are too large to run on a laptop. But HPC environments aren’t naturally designed for running web applications, especially not ones that multiple people may want to launch on demand.

We needed a solution that would:

• Let anyone on the team launch the dashboard without installing software
• Scale up if we wanted to start visualizing the project’s larger datasets
• Be friendly to both programmers and non‑programmers

That’s where Open OnDemand came in.

Open OnDemand (OOD) is a web platform that sits on top of an HPC cluster and gives users the ability to run jobs, move files, and launch applications, all from a browser. Instead of everyone needing to memorize Unix commands or write job scripts, they can launch the dashboard in just a couple of clicks.

What I didn’t know at first was that OOD supports two distinct ways of deploying custom applications. Choosing between them turned out to be a key technical decision.

Open OnDemand supports:

1. Passenger Applications

These behave like traditional web apps.

• They run on the cluster’s login or “head” node.
• They’re ideal for small, lightweight applications with modest computing needs.
• They automatically stay online, which is convenient.

For our dashboard, this approach wasn’t ideal. Because our visualizations can involve large model outputs, running everything on the head node could slow things down or interfere with other users.

2. Interactive Applications

These work differently:

• Each time a user launches the app, OOD submits a job to the cluster
• That job reserves compute resources on one of the worker nodes
• OOD then connects the user’s browser to that job so the app runs inside the HPC environment

This design has important advantages:

• The dashboard can use as much memory or computing power as the user requests
• Each user gets their own isolated instance
• It keeps our options flexible in case we want to add more computational complexity to the dashboard in the future

For our use case, we decided the Interactive App approach gave us the most flexibility for future development.

In the final setup, running the dashboard looks like this:

1. A collaborator logs into our cluster’s Open OnDemand page
2. They click on the dashboard’s icon under “Interactive Apps”
3. They choose how many hours their session will be
4. They click “Launch”
5. Within a minute or two, a unique dashboard session opens right in their browser

This means hydrologists who may not write code every day can still explore large datasets and investigate model behavior, without ever touching the command line.

Setting this up taught me a lot about how Open OnDemand handles custom applications. To help others who might be trying something similar, I put together a small GitHub repository with several “Hello World” examples to show how both Passenger and Interactive apps can be developed, both with and without Dash.

You can find it here: https://github.com/amy-defnet/hello-world-ood

Making scientific tools accessible is just as important as building them. By turning complex datasets into an interactive dashboard that anyone on the team can launch with a few clicks, we’ve made it easier for researchers to analyze results, spot issues, and ask new scientific questions. I’m excited to continue improving this tool and exploring new ways to help scientists use computational models more effectively.

The following work received a spotlight at NeurIPS:

John Yang, Kilian Lieret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, Diyi Yang

Despite recent progress in Language Models (LMs) for software engineering, collecting training data remains a significant pain point. Existing datasets are small, with at most 1,000s of training instances from 11 or fewer GitHub repositories. The procedures to curate such datasets are often complex, necessitating hundreds of hours of human labor; companion execution environments also take up several terabytes of storage, severely limiting their scalability and usability. To address this pain point, we introduce SWE-smith, a novel pipeline for generating software engineering training data at scale. Given any Python codebase, SWE-smith constructs a corresponding execution environment, then automatically synthesizes 100s to 1,000s of task instances that break existing test(s) in the codebase. Using SWE-smith, we create a dataset of 50k instances sourced from 128 GitHub repositories, an order of magnitude larger than all previous works. We train SWE-agent-LM-32B, achieving 40.2% Pass@1 resolve rate on the SWE-bench Verified benchmark, state of the art among open source models. We open source SWE-smith (collection procedure, task instances, trajectories, models) to lower the barrier of entry for research in LM systems for automated software engineering. All assets available at swesmith.com.

https://arxiv.org/abs/2504.21798

Last month many members of the RSE Group and other Princeton colleagues attended USRSE’25, the third annual conference from US-RSE. Hosted this year in Philadelphia, the conference theme was “Code, Practices, and People.” Princeton University (authors in bold) contributions included:

1. Accelerating Research: Strategies from the Field – Jen Rosiere Reynolds, Lance Parsons, Gail Rosenbaum, Joost Wagenaar, and Sarah Stevens (BoF)
2. Sustainable Models of RSE Support: The Prospects of Centralization in Institutional Research – Eric Manning, Lori Bougher, Colin Swaney, and Sangyoon Park (BoF)
3. Undate: computing with uncertain and partially-unknown dates – Rebecca S. Koeser (notebook)
4. Integrating ATR Software with University HPC Infrastructure: balancing diverse compute needs – Christine Roughan and Rebecca S. Koeser (paper)
5. INnovative Training Enabled by a Research Software Engineering Community of Trainers (INTERSECT) – Jeffrey Carver and Ian Cosden (poster)
6. Building Scientific Python Packages – Henry Schreiner (poster)
7. Community Code Review in the Digital Humanities – Julia Damerow, Rebecca S. Koeser, Jeffrey C. Carver, and Malte Vogl (poster)
8. Surveying the Digital Humanities Research Software Engineering Landscape – Rebecca S. Koeser and Julia Damerow (poster)
9. Ten Simple Rules for Catalyzing Collaborations and Building Bridges between Research Software Engineers and Software Engineering Researchers – Nasir Eisty, Jeffrey Carver, Johanna Cohoon, Ian Cosden, Carole Goble, and Samuel Grayson (poster)
10. Developing a Machine Learning-Augmented Solver for the Hydrologic Model ParFlow – Georgios Artavanis, Laura Condon, Andrew Bennett, and Reed Maxwell (talk)
11. Everything, All at Once, Yesterday: Creating Research Software with Humanities Faculty – Jeri Wieringa and Mary Naydan (talk)
12. What happened to Curt’s arm? – Curt Hillegas (RAM)
13. Agile Foundations for RSEs: Building an AI Assistant with Agile – Tisha Charles and David Luet (workshop)

Additionally, Princeton University Professor Reed Maxwell delivered the first keynote address on Accelerating Continental-Scale Groundwater Simulation With a Fusion of Machine Learning, Integrated Hydrologic Models and Community Platforms. His keynote highlighted three of his lab’s software projects centered around hydrologic data, simulations, and visualizations, and he noted contributions to those projects from five current and past RSE Group members (Vineet Bansal, Calla Chenault, Georgios Artavanis, Amy Defnet, and Bill Hasling). Professor Maxwell stated that not only RSE contributions to software, but additionally that “RSEs enable digital education and outreach content.”

All in all, it was inspiring to convene with RSEs from all over the country. We already look forward to next year’s conference to be hosted in the San Francisco Bay Area!

John Yang, Kilian Lieret, Joyce Yang, Carlos E. Jimenez, Ofir Press, Ludwig Schmidt, Diyi Yang

Current benchmarks for coding evaluate language models (LMs) on concrete, well-specified tasks such as fixing specific bugs or writing targeted tests. However, human programmers do not spend all day incessantly addressing isolated tasks. Instead, real-world software development is grounded in the pursuit of high-level goals, like improving user retention or reducing costs. Evaluating whether LMs can also iteratively develop code to better accomplish open-ended objectives without any explicit guidance remains an open challenge. To address this, we introduce CodeClash, a benchmark where LMs compete in multi-round tournaments to build the best codebase for achieving a competitive objective. Each round proceeds in two phases: agents edit their code, then their codebases compete head-to-head in a code arena that determines winners based on objectives like score maximization, resource acquisition, or survival. Whether it’s writing notes, scrutinizing documentation, analyzing competition logs, or creating test suites, models must decide for themselves how to improve their codebases both absolutely and against their opponents. We run 1680 tournaments (25,200 rounds total) to evaluate 8 LMs across 6 arenas. Our results reveal that while models exhibit diverse development styles, they share fundamental limitations in strategic reasoning. Models also struggle with long-term codebase maintenance, as repositories become progressively messy and redundant. These limitations are stark: top models lose every round against expert human programmers. We open-source CodeClash to advance the study of autonomous, goal-oriented code development.

https://arxiv.org/abs/2511.00839

By Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei, Jason D. Lee, Sanjeev Arora

The success of Reinforcement Learning from Human Feedback (RLHF) critically depends on the quality of the reward model. However, while this quality is primarily evaluated through accuracy, it remains unclear whether accuracy fully captures what makes a reward model an effective teacher. We address this question from an optimization perspective. First, we prove that regardless of how accurate a reward model is, if it induces low reward variance, then the RLHF objective suffers from a flat landscape. Consequently, even a perfectly accurate reward model can lead to extremely slow optimization, underperforming less accurate models that induce higher reward variance. We additionally show that a reward model that works well for one language model can induce low reward variance, and thus a flat objective landscape, for another. These results establish a fundamental limitation of evaluating reward models solely based on accuracy or independently of the language model they guide. Experiments using models of up to 8B parameters corroborate our theory, demonstrating the interplay between reward variance, accuracy, and reward maximization rate. Overall, our findings highlight that beyond accuracy, a reward model needs to induce sufficient variance for efficient~optimization.

Read the paper: https://arxiv.org/abs/2503.15477

By Luis Linares-Otoya, Jaden D. Shirkey, Bhuwan Khatri Chhetri, Amira Mira, Abhishek Biswas, Samuel L. Neff, Maria V. Linares-Otoya, Ye Chen, Julio V. Campos-Florian, Mayar L. Ganoza-Yupanqui, Philip D. Jeffrey, Frederick M. Hughson & Mohamed S. Donia

Considerable advances have been made in characterizing bioactive molecules secreted by bacteria, yet the regulatory elements controlling their production remain largely understudied. Here we identify and characterize the N-acyl-cyclolysine (ACL) system—a cell-density-dependent chemical signalling system specific to and widespread in the phylum Bacteroidota (formerly Bacteroidetes)—and show that it regulates the expression of co-localized operons encoding diverse secreted molecules. Using genetic and biochemical analyses, combined with structural studies of a key biosynthetic enzyme, AclA, we elucidate the molecular structure of various ACLs and their complete biosynthetic pathway involving l-lysine acylation and ATP-dependent cyclization. Furthermore, we find that secreted ACLs are sensed by a dedicated transcription factor, AclR, resulting in the expression of associated operons and the autoinduction of ACL biosynthesis. Moreover, we show that different Bacteroidota strains produce structurally diverse ACLs and encode transcription factors with varying ligand specificities. Finally, we find that the acl circuit is widely distributed and transcribed in human gut and oral microbiome samples, with clear evidence for an active role in regulating associated operons under host colonization conditions. Understanding the function of the ACL system in different contexts has the potential to reveal details about the biology, ecology and chemistry of the Bacteroidota and how members of this phylum interact with their environments and hosts.

Read the paper: https://www.nature.com/articles/s41586-025-09418-9

By Jiayu Zhang, Elizabeth C. Chavez, Melina Winkler, Jianche Liu, Sebastian Carver, Aaron E. Lin, Abhishek Biswas, Tomokazu Tamura, Anna Tseng, Danyang Wang, Aaron Benhamou, Aoife K. O’ Connell, Mao Matsuo, Jack E. Norton, Devin Kenney, Britt Adamson, Ralph E. Kleiner, Benjamin Burwitz, Nicholas A. Crossland, Florian Douam & Alexander Ploss

The live-attenuated yellow fever 17D vaccine strain differs genetically only minimally from its virulent parent. However, it remains unclear which sequence differences lead to virulence or attenuation. Here we demonstrate, using SHAPE-MaP, that these mutations do not induce global RNA structure changes and show that protein sequence mutations are mostly responsible for the phenotypic differences between 17D and virulent YFV. Using a highly modular, combinatorial genetic approach, we identified key mutations in the envelope (E) and non-structural 2A (NS2A) proteins that increase 17D’s ability to spread and enhance host antiviral responses. Introducing these mutations into infectious clones of virulent YFV genomes results in viral attenuation in vitro and in two mouse models. Collectively, our results define the genetic basis for 17D attenuation and highlight a potentially general approach for creating live-attenuated vaccines by introducing mutations resulting in similar phenotypic changes in other pathogenic viruses.

Read the paper: https://www.nature.com/articles/s41564-025-02047-y

By Sofia A. Quinodoz, Lifei Jiang, Aya A. Abu-Alfa, Troy J. Comi, Hongbo Zhao, Qiwei Yu, Lennard W. Wiesner, Jordy F. Botello, Anita Donlic, Elizabeth Soehalim, Prashant Bhat, Christiane Zorbas, Ludivine Wacheul, Andrej Košmrlj, Denis L. J. Lafontaine, Sebastian Klinge & Clifford P. Brangwynne

Biomolecular condensates are key features of intracellular compartmentalization. As the most prominent nuclear condensate in eukaryotes, the nucleolus is a multiphase liquid-like structure in which ribosomal RNAs (rRNAs) are transcribed and processed, undergoing multiple maturation steps to form the small (SSU) and large (LSU) ribosomal subunits. However, how rRNA processing is coupled to the layered organization of the nucleolus is poorly understood owing to a lack of tools to precisely monitor and perturb nucleolar rRNA processing dynamics. Here we developed two complementary approaches to spatiotemporally map rRNA processing and engineer de novo nucleoli. Using sequencing in parallel with imaging, we found that rRNA processing steps are spatially segregated, with sequential maturation of rRNA required for its outward movement through nucleolar phases. By generating synthetic nucleoli in cells using an engineered rDNA plasmid system, we show that defects in SSU processing can alter the ordering of nucleolar phases, resulting in inside-out nucleoli and preventing rRNA outflux, while LSU precursors are necessary to build the outermost layer of the nucleolus. These findings demonstrate how rRNA is both a scaffold and substrate for the nucleolus, with rRNA acting as a programmable blueprint for the multiphase architecture that facilitates assembly of an essential molecular machine.

Read the paper: https://www.nature.com/articles/s41586-025-09207-4