GenePrep

Unified Genomic Data Formatter for Statistical Analysis

By Yue Xu¹, Sirihaasa Nallamothu², Haoyang Liu², Haohan Wang² — ¹Columbia University, ²UIUC

Get Started View Code

Overview

GenePrep is an automated multi‑agent tool to streamline preprocessing and analysis of large‑scale gene expression data (GEO, TCGA). Provide a dataset and trait–condition pairs; GenePrep validates data, selects pairs, performs statistical tests, and outputs reproducible logs and CSV results with minimal scripting.

Workflow

End-to-end workflow for GenePrep.

Quick Start

CLI Preview

$ conda create -n agent python=3.10
$ pip install -r requirements.txt
$ python main.py --version 1 \
    --model gemini-2.0-flash-002 --api 1

Commands shown for illustration; see manual for full options.

Features

Automation

End‑to‑end workflow: validation, pair selection, tests, and result generation.

Modular Agents

Plan, execute, and debug with minimal manual scripting using modular agents.

Reproducibility

Terminal‑style logs and CSV outputs for transparent, repeatable results.

Gallery

Real-time resource consumption in terminal.

Generated code and CSV files saved to output folder.

Inspect agent action and communication logs.

Sample result CSV output.

Paper

Read the full GenePrep paper below:

You can also open the PDF in a new tab.

### 1. Data preparation
Download the input data from our Google Drive [folder](https://drive.google.com/drive/u/0/folders/1A25gqaIpcahle6TLJ81Qnd2VoluJ2NEe). \
You can verify data integrity with:
```bash
python validator.py --data-dir /path/to/data --validate
```

### 2. Set up environment
Create a conda environment with Python 3.10 and install the required packages:
```bash
conda create -n agent python=3.10
conda activate agent
pip install -r requirements.txt
```

### 3. Set up API key File(Optional)
Go to the folder of the package, create a ".env" file(please make sure the name is exactly ".env") and stores your API key and organization info.\
The following is an example for OpenAI API key and organization, so please adjust if you are using other platform.\
Note: Use newline to separate, no space should be used.
```bash
OPENAI_API_KEY_1=sk-sample-key
OPENAI_ORGANIZATION_1=org-sample-xxx
```

### 3. Run the code
#### If you have set up the API key file
Modify `in_data_root` in `main.py` to set input data path on different devices.\
Run an experiment like this:
```bash
python main.py --version 1 --model gemini-2.0-flash-002 --api 1
```

The first time you run this, you'll get an error asking you to set up an API key, e.g. `GOOGLE_API_KEY_1` in a `.env` file. You'll need to get this API key from the LLM provider.

If you type a wrong model name, the error message will show you all the model names that work with this code.

For open-source LLMs, you can either run them locally with [Ollama Python](https://github.com/ollama/ollama-python), or use APIs (use the `--use-api` flag).

#### If you chose not to set up the File
You can use the following code template to pass your API key and org info in command line
```bash
python main.py --version 1 --model o1-mini-2024-09-12 --api_key xxxxx --organization xxxxx
```

If you type a wrong model name, the error message will show you all the model names that work with this code.

For open-source LLMs, you can either run them locally with [Ollama Python](https://github.com/ollama/ollama-python), or use APIs (use the `--use-api` flag).

BibTeX

@inproceedings{nallamothu2025geneprep,
        title        = {GenePrep: Unified Genomic Data Formatter for Statistical Analysis},
        author       = {Yue Xu and Sirihaasa Nallamothu and Haoyang Liu and Haohan Wang},
        booktitle    = {Proceedings of the Machine Learning in Computational Biology Conference (MLCB)},
        year         = {2025},
        month        = jun,
        address      = {Vancouver, Canada},
        publisher    = {MLCB},
        url          = {https://www.mlcb.io/2025},
        abstract     = {GenePrep is an automated multi-agent system that streamlines preprocessing and analysis of large-scale gene expression data from GEO and TCGA. By simply installing the GenePrep package, users can perform end-to-end workflows including data validation, trait–condition pair selection, statistical testing, and result generation. Given a dataset and trait–condition pairs, GenePrep identifies genes associated with traits while accounting for conditions. Its modular agents enable iterative planning, execution, and debugging with minimal manual scripting. GenePrep reduces preprocessing overhead and improves reproducibility. This work extends on tools to create teams of AI scientists and automate gene expression data analysis, as presented in Liu et al. (2024, 2025).}
      }