diff --git a/docs/src/en/enrichr.md b/docs/src/en/enrichr.md index 6df88088d..aa8504217 100644 --- a/docs/src/en/enrichr.md +++ b/docs/src/en/enrichr.md @@ -42,6 +42,21 @@ Short names (gene symbols) of background genes to perform enrichment analysis on Alternatively: use flag `--ensembl_background` to input a list of Ensembl gene IDs. See [this Tweetorial](https://x.com/ChiHoangCaltech/status/1689679611335155712?s=20) to learn why you should use a background gene list when performing an enrichment analysis. +`-gl` `--get_library` +Instead of running an enrichment analysis, fetch the gene sets (members) of this Enrichr gene-set library, e.g. `MSigDB_Hallmark_2020`. This is the recommended way to retrieve [MSigDB](https://www.gsea-msigdb.org/gsea/msigdb/) gene sets (search for "MSigDB" in the [Enrichr library list](https://maayanlab.cloud/Enrichr/#libraries), e.g. `MSigDB_Hallmark_2020`, `MSigDB_Oncogenic_Signatures`, `MSigDB_Computational`). When set, the `genes` argument and `--database` are not required. +Python: `gget.enrichr_library("MSigDB_Hallmark_2020")` + +`-gs` `--gene_set` +With `--get_library`: only return the genes of this single gene set (term) within the library, e.g. `Hypoxia`. (Default: None -> return all gene sets in the library.) + +`-ll` `--list_libraries` +List the available Enrichr gene-set libraries (to discover library names), then exit. Optionally pass a substring to filter, e.g. `--list_libraries MSigDB`. +Python: `gget.enrichr_libraries(filter="MSigDB")` + +`-desc` `--descriptions` +With `--get_library`: also include each gene set's description column. +Python: `descriptions=True` + `-o` `--out` Path to the file the results will be saved in, e.g. path/to/directory/results.csv (or .json). (Default: Standard out.) Python: `save=True` will save the output in the current working directory. @@ -222,6 +237,25 @@ df |> xlab("-log10(adjusted P value)") ``` +

+ +**Fetch the gene sets of an MSigDB collection (e.g. the Hallmark gene sets):** +```bash +gget enrichr --get_library MSigDB_Hallmark_2020 --csv +``` +```python +# Python +import gget +gget.enrichr_library("MSigDB_Hallmark_2020") +``` +→ Returns the 50 MSigDB Hallmark gene sets and their member genes as a long-format data frame (`gene_set`, `gene`). Add `--gene_set Hypoxia` (Python: `gene_set="Hypoxia"`) to return only one gene set. Use `--list_libraries MSigDB` (Python: `gget.enrichr_libraries(filter="MSigDB")`) to discover the available MSigDB collections. + +| gene_set | gene | +|--------------------------------|--------| +| TNF-alpha Signaling via NF-kB | JUNB | +| TNF-alpha Signaling via NF-kB | CXCL2 | +| ... | ... | + # Tutorials [Using `gget enrichr` with background genes](https://github.com/pachterlab/gget_examples/blob/main/gget_enrichr_with_background_genes.ipynb) @@ -238,3 +272,11 @@ If you use `gget enrichr` in a publication, please cite the following articles: If working with non-human/mouse datasets, please also cite: - Kuleshov MV, Diaz JEL, Flamholz ZN, Keenan AB, Lachmann A, Wojciechowicz ML, Cagan RL, Ma'ayan A. modEnrichr: a suite of gene set enrichment analysis tools for model organisms. Nucleic Acids Res. 2019 Jul 2;47(W1):W183-W190. doi: [10.1093/nar/gkz347](https://doi.org/10.1093/nar/gkz347). PMID: 31069376; PMCID: PMC6602483. + +If retrieving MSigDB collections, please also cite: +- Subramanian A, Tamayo P, Mootha VK, Mukherjee S, Ebert BL, Gillette MA, Paulovich A, Pomeroy SL, Golub TR, Lander ES, Mesirov JP. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci USA. 2005;102(43):15545-15550. doi: [10.1073/pnas.0506580102](https://doi.org/10.1073/pnas.0506580102) +- Liberzon A, Subramanian A, Pinchback R, Thorvaldsdóttir H, Tamayo P, Mesirov JP. Molecular signatures database (MSigDB) 3.0. Bioinformatics. 2011;27(12):1739-1740. doi: [10.1093/bioinformatics/btr260](https://doi.org/10.1093/bioinformatics/btr260) +- Liberzon A, Birger C, Thorvaldsdóttir H, Ghandi M, Mesirov JP, Tamayo P. The Molecular Signatures Database (MSigDB) hallmark gene set collection. Cell Syst. 2015;1(6):417-425. doi: [10.1016/j.cels.2015.12.004](https://doi.org/10.1016/j.cels.2015.12.004) + +If using mouse MSigDB, please also cite: +- Castanza AS, Recla JM, Eby D, Thorvaldsdóttir H, Bult CJ, Mesirov JP. Extending support for mouse data in the Molecular Signatures Database (MSigDB). Nat Methods. 2023;20:1619-1620. doi: [10.1038/s41592-023-02014-7](https://doi.org/10.1038/s41592-023-02014-7) diff --git a/docs/src/en/updates.md b/docs/src/en/updates.md index 89be22281..0c8abdba3 100644 --- a/docs/src/en/updates.md +++ b/docs/src/en/updates.md @@ -5,6 +5,9 @@ #### *gget* officially became part of [*scverse*](https://scverse.org/) on June 9, 2026. 🥳🥳🥳 **Version ≥ 0.30.9** (XXX XX, 2026): +- [`gget enrichr`](enrichr.md): Added support for retrieving gene sets, including [MSigDB](https://www.gsea-msigdb.org/gsea/msigdb/) collections (fixes [issue 139](https://github.com/scverse/gget/issues/139)). + - New `gget.enrichr_library()` function (command line: `gget enrichr --get_library`/`-gl`) fetches the gene sets (members) of any Enrichr gene-set library, e.g. `MSigDB_Hallmark_2020`, `MSigDB_Oncogenic_Signatures`, `MSigDB_Computational`. + - Returns a long-format DataFrame (`gene_set`, `gene`), or a `{gene_set: [genes]}` dictionary with `json=True`. Use `gene_set=` (command line: `--gene_set`/`-gs`) to return a single gene set, and the `species` argument for the non-human Enrichr variants. **Version ≥ 0.30.8** (Jun 28, 2026): - [`gget g2p`](g2p.md): Either `gene` or `--uniprot_id` is now sufficient — whichever is missing is resolved via UniProt and cached. Gene→UniProt picks the canonical reviewed human Swiss-Prot entry; the resolution and its limitations are logged. The canonical pair is **always** prepended to the result as `gene_name` / `uniprot_id` columns (and stored on `df.attrs`), so the output schema is invariant regardless of input mode. Existing call sites continue to work. diff --git a/docs/src/es/enrichr.md b/docs/src/es/enrichr.md index 18cb6308b..cc7e685ab 100644 --- a/docs/src/es/enrichr.md +++ b/docs/src/es/enrichr.md @@ -39,6 +39,21 @@ Opciones: Lista de nombres cortos (símbolos) de genes de 'background' (de fondo/control), p. NSUN3 POLRMT NLRX1. Alternativamente: usa la bandera `--ensembl_background` para ingresar IDs tipo Ensembl. +`-gl` `--get_library` +En lugar de ejecutar el análisis de enriquecimiento, obtén los conjuntos de genes (miembros) de esta biblioteca de Enrichr, p. ej. `MSigDB_Hallmark_2020`. Es la forma recomendada de obtener conjuntos de genes de [MSigDB](https://www.gsea-msigdb.org/gsea/msigdb/) (busca "MSigDB" en la [lista de bibliotecas de Enrichr](https://maayanlab.cloud/Enrichr/#libraries)). Con esta opción, `genes` y `--database` no son necesarios. +Para Python: `gget.enrichr_library("MSigDB_Hallmark_2020")` + +`-gs` `--gene_set` +Con `--get_library`: solo regresa los genes de este único conjunto de genes (término) dentro de la biblioteca, p. ej. `Hypoxia`. (Por defecto: None -> regresa todos los conjuntos de la biblioteca.) + +`-ll` `--list_libraries` +Lista las bibliotecas de conjuntos de genes disponibles en Enrichr (para descubrir sus nombres) y termina. Opcionalmente pasa un texto para filtrar, p. ej. `--list_libraries MSigDB`. +Para Python: `gget.enrichr_libraries(filter="MSigDB")` + +`-desc` `--descriptions` +Con `--get_library`: también incluye la columna de descripción de cada conjunto de genes. +Para Python: `descriptions=True` + `-o` `--out` Ruta al archivo en el que se guardarán los resultados, p. ruta/al/directorio/resultados.csv (o .json). Por defecto: salida estándar (STDOUT). Para Python, usa `save=True` para guardar los resultados en el directorio de trabajo actual. @@ -221,6 +236,19 @@ df |> #### [Más ejemplos](https://github.com/pachterlab/gget_examples) +

+ +**Obtén los conjuntos de genes de una colección de MSigDB (p. ej. los conjuntos Hallmark):** +```bash +gget enrichr --get_library MSigDB_Hallmark_2020 --csv +``` +```python +# Python +import gget +gget.enrichr_library("MSigDB_Hallmark_2020") +``` +→ Regresa los 50 conjuntos de genes Hallmark de MSigDB y sus genes miembros como un Dataframe en formato largo (`gene_set`, `gene`). Agrega `--gene_set Hypoxia` (Python: `gene_set="Hypoxia"`) para regresar un solo conjunto. Usa `--list_libraries MSigDB` (Python: `gget.enrichr_libraries(filter="MSigDB")`) para descubrir las bibliotecas MSigDB disponibles. + # Citar Si utiliza `gget enrichr` en una publicación, favor de citar los siguientes artículos: @@ -234,3 +262,11 @@ Si utiliza `gget enrichr` en una publicación, favor de citar los siguientes art Si trabaja con conjuntos de datos no humanos/ratón, cite también: - Kuleshov MV, Diaz JEL, Flamholz ZN, Keenan AB, Lachmann A, Wojciechowicz ML, Cagan RL, Ma'ayan A. modEnrichr: a suite of gene set enrichment analysis tools for model organisms. Nucleic Acids Res. 2019 Jul 2;47(W1):W183-W190. doi: [10.1093/nar/gkz347](https://doi.org/10.1093/nar/gkz347). PMID: 31069376; PMCID: PMC6602483. + +Si recupera colecciones de MSigDB, cite también: +- Subramanian A, Tamayo P, Mootha VK, Mukherjee S, Ebert BL, Gillette MA, Paulovich A, Pomeroy SL, Golub TR, Lander ES, Mesirov JP. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci USA. 2005;102(43):15545-15550. doi: [10.1073/pnas.0506580102](https://doi.org/10.1073/pnas.0506580102) +- Liberzon A, Subramanian A, Pinchback R, Thorvaldsdóttir H, Tamayo P, Mesirov JP. Molecular signatures database (MSigDB) 3.0. Bioinformatics. 2011;27(12):1739-1740. doi: [10.1093/bioinformatics/btr260](https://doi.org/10.1093/bioinformatics/btr260) +- Liberzon A, Birger C, Thorvaldsdóttir H, Ghandi M, Mesirov JP, Tamayo P. The Molecular Signatures Database (MSigDB) hallmark gene set collection. Cell Syst. 2015;1(6):417-425. doi: [10.1016/j.cels.2015.12.004](https://doi.org/10.1016/j.cels.2015.12.004) + +Si utiliza MSigDB de ratón, cite también: +- Castanza AS, Recla JM, Eby D, Thorvaldsdóttir H, Bult CJ, Mesirov JP. Extending support for mouse data in the Molecular Signatures Database (MSigDB). Nat Methods. 2023;20:1619-1620. doi: [10.1038/s41592-023-02014-7](https://doi.org/10.1038/s41592-023-02014-7) diff --git a/gget/__init__.py b/gget/__init__.py index f56cbcddc..3f02ea425 100644 --- a/gget/__init__.py +++ b/gget/__init__.py @@ -14,7 +14,7 @@ from .gget_cosmic import cosmic from .gget_diamond import diamond from .gget_elm import elm -from .gget_enrichr import enrichr +from .gget_enrichr import enrichr, enrichr_libraries, enrichr_library from .gget_g2p import g2p from .gget_gpt import gpt from .gget_info import info diff --git a/gget/constants.py b/gget/constants.py index 38f90119f..b234b0a14 100644 --- a/gget/constants.py +++ b/gget/constants.py @@ -60,6 +60,20 @@ } GET_ENRICHR_URLS["human"] = GET_ENRICHR_URL +# Enrichr endpoint to download the gene sets (membership) of a library, e.g. MSigDB collections +GENESETLIBRARY_ENRICHR_URLS = { + f"{typ}": f"https://maayanlab.cloud/{typ.capitalize()}Enrichr/geneSetLibrary" + for typ in ["fly", "yeast", "worm", "fish"] +} +GENESETLIBRARY_ENRICHR_URLS["human"] = "https://maayanlab.cloud/Enrichr/geneSetLibrary" + +# Enrichr endpoint listing all available gene-set libraries (for library discovery) +DATASETSTATISTICS_ENRICHR_URLS = { + f"{typ}": f"https://maayanlab.cloud/{typ.capitalize()}Enrichr/datasetStatistics" + for typ in ["fly", "yeast", "worm", "fish"] +} +DATASETSTATISTICS_ENRICHR_URLS["human"] = "https://maayanlab.cloud/Enrichr/datasetStatistics" + # ARCHS4 API endpoints GENECORR_URL = "https://maayanlab.cloud/matrixapi/coltop" EXPRESSION_URL = "https://maayanlab.cloud/archs4/search/loadExpressionTissue.php?" diff --git a/gget/gget_enrichr.py b/gget/gget_enrichr.py index deb2c75b7..f1bf6a5df 100644 --- a/gget/gget_enrichr.py +++ b/gget/gget_enrichr.py @@ -16,7 +16,9 @@ from .compile import PACKAGE_PATH from .constants import ( + DATASETSTATISTICS_ENRICHR_URLS, DEFAULT_REQUESTS_TIMEOUT, + GENESETLIBRARY_ENRICHR_URLS, GET_BACKGROUND_ENRICHR_URL, GET_ENRICHR_URLS, POST_BACKGROUND_ID_ENRICHR_URL, @@ -64,6 +66,196 @@ def clean_genes_list(genes_list: list[Any]) -> list[Any]: return genes_clean +def enrichr_library( + library: str, + species: str = "human", + gene_set: str | None = None, + descriptions: bool = False, + json: bool = False, + save: bool = False, + verbose: bool = True, +) -> pd.DataFrame | dict[str, Any]: + """Fetch the gene sets (members) of an Enrichr gene-set library. + + This is useful for retrieving the gene sets of a collection such as the + [MSigDB](https://www.gsea-msigdb.org/gsea/msigdb/) libraries hosted by Enrichr + (e.g. "MSigDB_Hallmark_2020", "MSigDB_Oncogenic_Signatures", "MSigDB_Computational"). + + Args: + - library Name of the Enrichr gene-set library to fetch, e.g. "MSigDB_Hallmark_2020". + See the full list at https://maayanlab.cloud/Enrichr/#libraries + (search for "MSigDB" for the MSigDB collections). + - species Enrichr variant to query: 'human' (default), 'mouse' (uses the human/Enrichr variant), + 'fly', 'yeast', 'worm', or 'fish'. + - gene_set If provided, only return the genes of this single gene set (term) within the library + (default: None -> return all gene sets in the library). + - descriptions If True, also include each gene set's description (the source often leaves this + empty). Adds a 'description' column to the data frame; with json=True the dict becomes + {gene_set: {'description': str, 'genes': [genes]}} (default: False). + - json If True, returns a dictionary instead of a pandas DataFrame (default: False). + - save If True, saves the result to 'gget_enrichr_library_{library}.csv' + (or .json if json=True) in the current working directory (default: False). + - verbose True/False whether to print progress information (default: True). + + Returns a long-format pandas DataFrame with one row per (gene set, gene) pair and the columns + 'gene_set' and 'gene' (plus 'description' if descriptions=True). With json=True, returns a + {gene_set: [genes]} dictionary (or {gene_set: {'description', 'genes'}} if descriptions=True). + """ + if species not in ["human", "mouse", "fly", "yeast", "worm", "fish"]: + raise ValueError("Argument 'species' must be one of 'human', 'mouse', 'fly', 'yeast', 'worm', or 'fish'.") + + # Mouse uses the human (Enrichr) variant, consistent with gget.enrichr + species_key = "human" if species in ("human", "mouse") else species + url = GENESETLIBRARY_ENRICHR_URLS[species_key] + + if verbose: + logger.info(f"Fetching gene sets for Enrichr library '{library}'...") + + response = requests.get( + url, + params={"mode": "text", "libraryName": library}, + timeout=DEFAULT_REQUESTS_TIMEOUT, + ) + # Enrichr returns an HTML 404 page (with HTTP status 200) for unknown library names + text = response.text + if not response.ok or text.lstrip()[:1] == "<" or "HTTP Status 404" in text[:500]: + raise RuntimeError( + f"Enrichr did not return a valid gene-set library for '{library}'. " + "Please double-check the library name (see https://maayanlab.cloud/Enrichr/#libraries)." + ) + + # Each line: "\t\t\t\t..." + library_dict: dict[str, list[str]] = {} + description_map: dict[str, str] = {} + for line in text.splitlines(): + if not line.strip(): + continue + fields = line.split("\t") + set_name = fields[0].strip() + # The second field is an (often empty) description; genes start at index 2 + genes = [g.strip() for g in fields[2:] if g.strip()] + if set_name: + library_dict[set_name] = genes + description_map[set_name] = fields[1].strip() if len(fields) > 1 else "" + + # A valid library has at least one gene set with member genes + if not library_dict or not any(genes for genes in library_dict.values()): + raise RuntimeError( + f"No gene sets returned for Enrichr library '{library}'. " + "Please double-check the library name (see https://maayanlab.cloud/Enrichr/#libraries)." + ) + + # Optionally restrict to a single gene set + if gene_set is not None: + if gene_set not in library_dict: + raise ValueError( + f"Gene set '{gene_set}' not found in Enrichr library '{library}'. " + f"The library contains {len(library_dict)} gene sets." + ) + library_dict = {gene_set: library_dict[gene_set]} + + if verbose: + n_genes = sum(len(g) for g in library_dict.values()) + logger.info(f"Retrieved {len(library_dict)} gene set(s) containing {n_genes} gene entries.") + + if json: + result_dict: dict[str, Any] + if descriptions: + result_dict = {s: {"description": description_map.get(s, ""), "genes": g} for s, g in library_dict.items()} + else: + result_dict = library_dict + if save: + with open(f"gget_enrichr_library_{library}.json", "w", encoding="utf-8") as f: + json_package.dump(result_dict, f, ensure_ascii=False, indent=4) + return result_dict + + if descriptions: + rows = [ + {"gene_set": s, "description": description_map.get(s, ""), "gene": gene} + for s, genes in library_dict.items() + for gene in genes + ] + df = pd.DataFrame(rows, columns=["gene_set", "description", "gene"]) + else: + rows = [{"gene_set": s, "gene": gene} for s, genes in library_dict.items() for gene in genes] + df = pd.DataFrame(rows, columns=["gene_set", "gene"]) + + if save: + df.to_csv(f"gget_enrichr_library_{library}.csv", index=False) + + return df + + +def enrichr_libraries( + species: str = "human", + filter: str | None = None, + json: bool = False, + save: bool = False, + verbose: bool = True, +) -> pd.DataFrame | list[dict[str, Any]]: + """List the gene-set libraries available from Enrichr (for discovering library names). + + Handy for finding the exact library name to pass to `enrichr_library`, e.g. searching for the + MSigDB collections with filter="MSigDB". + + Args: + - species Enrichr variant to list: 'human' (default), 'mouse' (uses the human/Enrichr variant), + 'fly', 'yeast', 'worm', or 'fish'. + - filter If provided, only return libraries whose name contains this substring + (case-insensitive), e.g. "MSigDB" (default: None -> all libraries). + - json If True, returns a list of dictionaries instead of a pandas DataFrame (default: False). + - save If True, saves the result to 'gget_enrichr_libraries.csv' + (or .json if json=True) in the current working directory (default: False). + - verbose True/False whether to print progress information (default: True). + + Returns a pandas DataFrame (or list of dicts if json=True) with one row per library and the + columns 'library', 'num_terms', 'gene_coverage', 'genes_per_term'. + """ + if species not in ("human", "mouse", "fly", "yeast", "worm", "fish"): + raise ValueError("Argument 'species' must be one of 'human', 'mouse', 'fly', 'yeast', 'worm', or 'fish'.") + + # Mouse uses the human (Enrichr) variant, consistent with gget.enrichr + species_key = "human" if species in ("human", "mouse") else species + + if verbose: + logger.info(f"Fetching the list of Enrichr gene-set libraries ({species})...") + + response = requests.get(DATASETSTATISTICS_ENRICHR_URLS[species_key], timeout=DEFAULT_REQUESTS_TIMEOUT) + if not response.ok: + raise RuntimeError( + f"Enrichr returned error status code {response.status_code} while listing libraries. Please try again." + ) + + stats = response.json().get("statistics", []) + rows = [ + { + "library": s.get("libraryName"), + "num_terms": s.get("numTerms"), + "gene_coverage": s.get("geneCoverage"), + "genes_per_term": s.get("genesPerTerm"), + } + for s in stats + ] + if filter: + needle = filter.lower() + rows = [r for r in rows if r["library"] and needle in r["library"].lower()] + rows.sort(key=lambda r: (r["library"] or "").lower()) + + if verbose: + logger.info(f"Found {len(rows)} gene-set library/libraries.") + + if json: + if save: + with open("gget_enrichr_libraries.json", "w", encoding="utf-8") as f: + json_package.dump(rows, f, ensure_ascii=False, indent=4) + return rows + + df = pd.DataFrame(rows, columns=["library", "num_terms", "gene_coverage", "genes_per_term"]) + if save: + df.to_csv("gget_enrichr_libraries.csv", index=False) + return df + + @overload def enrichr( genes: str | list[str], diff --git a/gget/main.py b/gget/main.py index 7a2944b09..f6d585530 100644 --- a/gget/main.py +++ b/gget/main.py @@ -29,7 +29,7 @@ from .gget_cosmic import cosmic # noqa: E402 from .gget_diamond import diamond # noqa: E402 from .gget_elm import elm # noqa: E402 -from .gget_enrichr import enrichr # noqa: E402 +from .gget_enrichr import enrichr, enrichr_libraries, enrichr_library # noqa: E402 from .gget_g2p import g2p # noqa: E402 from .gget_gpt import gpt # noqa: E402 from .gget_info import info # noqa: E402 @@ -1016,20 +1016,62 @@ def main() -> None: parser_enrichr.add_argument( "genes", type=str, - nargs="+", - help="List of gene symbols or Ensembl gene IDs to perform enrichment analysis on.", + nargs="*", + help="List of gene symbols or Ensembl gene IDs to perform enrichment analysis on. " + "(Not required with --get_library.)", ) parser_enrichr.add_argument( "-db", "--database", type=str, - required=True, + required=False, help=( "'pathway', 'transcription', 'ontology', 'diseases_drugs', 'celltypes', 'kinase_interactions'" "or any database listed at: https://maayanlab.cloud/Enrichr/#libraries" " or the species-specific libraries listed in the documentation" ), ) + parser_enrichr.add_argument( + "-gl", + "--get_library", + type=str, + default=None, + required=False, + help=( + "Instead of running enrichment, fetch the gene sets (members) of this Enrichr gene-set library, " + "e.g. 'MSigDB_Hallmark_2020'. Useful for retrieving MSigDB gene sets. " + "See https://maayanlab.cloud/Enrichr/#libraries" + ), + ) + parser_enrichr.add_argument( + "-gs", + "--gene_set", + type=str, + default=None, + required=False, + help="With --get_library: only return the genes of this single gene set (term) within the library.", + ) + parser_enrichr.add_argument( + "-ll", + "--list_libraries", + type=str, + nargs="?", + const="", + default=None, + required=False, + help=( + "List the available Enrichr gene-set libraries (to discover library names), then exit. " + "Optionally pass a substring to filter, e.g. --list_libraries MSigDB." + ), + ) + parser_enrichr.add_argument( + "-desc", + "--descriptions", + default=False, + action="store_true", + required=False, + help="With --get_library: also include each gene set's description column.", + ) parser_enrichr.add_argument( "-s", "--species", @@ -3504,6 +3546,62 @@ def main() -> None: ## enrichr return if args.command == "enrichr": + # List available gene-set libraries (discovery) instead of running enrichment + if args.list_libraries is not None: + libraries_results = enrichr_libraries( + species=args.species, + filter=args.list_libraries or None, + json=args.csv, + verbose=args.quiet, + ) + if isinstance(libraries_results, pd.DataFrame): + if args.out: + directory = "/".join(args.out.split("/")[:-1]) + if directory != "": + os.makedirs(directory, exist_ok=True) + libraries_results.to_csv(args.out, index=False) + else: + libraries_results.to_csv(sys.stdout, index=False) + elif args.out: + directory = "/".join(args.out.split("/")[:-1]) + if directory != "": + os.makedirs(directory, exist_ok=True) + with open(args.out, "w", encoding="utf-8") as f: + json.dump(libraries_results, f, ensure_ascii=False, indent=4) + else: + print(json.dumps(libraries_results, ensure_ascii=False, indent=4)) + return + + # Fetch the gene sets of a library (e.g. MSigDB) instead of running enrichment + if args.get_library: + library_results = enrichr_library( + library=args.get_library, + species=args.species, + gene_set=args.gene_set, + descriptions=args.descriptions, + json=args.csv, + verbose=args.quiet, + ) + + if isinstance(library_results, pd.DataFrame): + if args.out: + directory = "/".join(args.out.split("/")[:-1]) + if directory != "": + os.makedirs(directory, exist_ok=True) + library_results.to_csv(args.out, index=False) + else: + library_results.to_csv(sys.stdout, index=False) + elif args.out: + directory = "/".join(args.out.split("/")[:-1]) + if directory != "": + os.makedirs(directory, exist_ok=True) + with open(args.out, "w", encoding="utf-8") as f: + json.dump(library_results, f, ensure_ascii=False, indent=4) + else: + print(json.dumps(library_results, ensure_ascii=False, indent=4)) + + return + # Handle deprecated flags for backwards compatibility if args.genes_deprecated and args.genes: logger.warning("The [-g][--genes] argument is deprecated, using positional argument [genes] instead.") @@ -3512,6 +3610,8 @@ def main() -> None: logger.warning("The [-g][--genes] argument is deprecated, please use positional argument [genes] instead.") if not args.genes_deprecated and not args.genes: parser_enrichr.error("the following arguments are required: genes") + if not args.database: + parser_enrichr.error("the following arguments are required: -db/--database (or use --get_library)") ## Clean up args.genes genes_clean = [] diff --git a/tests/fixtures/test_enrichr.json b/tests/fixtures/test_enrichr.json index 3e284d90f..33ba7daaa 100644 --- a/tests/fixtures/test_enrichr.json +++ b/tests/fixtures/test_enrichr.json @@ -1,4 +1,37 @@ { + "test_enrichr_library": { + "type": "code_defined", + "args": { + "library": "MSigDB_Hallmark_2020", + "verbose": false + }, + "expected_n_sets": 50 + }, + "test_enrichr_library_gene_set": { + "type": "code_defined", + "args": { + "library": "MSigDB_Hallmark_2020", + "gene_set": "Hypoxia", + "verbose": false + }, + "expected_n_genes": 200 + }, + "test_enrichr_library_json": { + "type": "code_defined", + "args": { + "library": "MSigDB_Hallmark_2020", + "json": true, + "verbose": false + }, + "expected_n_sets": 50 + }, + "test_enrichr_library_bad": { + "type": "code_defined", + "args": { + "library": "NOT_A_LIBRARY_xyz" + }, + "expected_result": "RuntimeError" + }, "test_enrichr_pathway": { "type": "assert_equal", "args": { diff --git a/tests/test_enrichr.py b/tests/test_enrichr.py index 3df50fa34..c3368ac25 100644 --- a/tests/test_enrichr.py +++ b/tests/test_enrichr.py @@ -1,16 +1,21 @@ import json import math +import os +import tempfile import unittest +from unittest.mock import patch import matplotlib import matplotlib.pyplot as plt import pandas as pd +import requests from .from_json import from_json # Prevent matplotlib from opening windows matplotlib.use("Agg") -from gget.gget_enrichr import enrichr +import gget.gget_enrichr as gget_enrichr +from gget.gget_enrichr import enrichr, enrichr_libraries, enrichr_library # Load dictionary containing arguments and expected results with open("./tests/fixtures/test_enrichr.json") as json_file: @@ -48,6 +53,47 @@ def test_enrichr_background_ensembl(self): self.assertListEqual(result_to_test, expected_result) + def _live_library(self, **kwargs): + """Call enrichr_library live, skipping (not failing) on transient network/Enrichr issues. + + A genuine network error, or Enrichr transiently returning a non-data response (e.g. a + rate-limit/HTML page, which enrichr_library raises as RuntimeError), is treated as a skip + so these live tests don't go red on upstream hiccups. The exact-count anchors below are + stable because MSigDB_Hallmark_2020 is a frozen (2020) snapshot. + """ + try: + return enrichr_library(**kwargs) + except requests.RequestException as e: + self.skipTest(f"Network error reaching Enrichr: {e}") + except RuntimeError as e: + self.skipTest(f"Enrichr did not return library data (transient): {e}") + + def test_enrichr_library(self): + td = enrichr_dict["test_enrichr_library"] + df = self._live_library(**td["args"]) + self.assertListEqual(list(df.columns), ["gene_set", "gene"]) + self.assertEqual(df["gene_set"].nunique(), td["expected_n_sets"]) + + def test_enrichr_library_gene_set(self): + td = enrichr_dict["test_enrichr_library_gene_set"] + df = self._live_library(**td["args"]) + self.assertEqual(set(df["gene_set"]), {td["args"]["gene_set"]}) + self.assertEqual(len(df), td["expected_n_genes"]) + + def test_enrichr_library_json(self): + td = enrichr_dict["test_enrichr_library_json"] + result = self._live_library(**td["args"]) + self.assertIsInstance(result, dict) + self.assertEqual(len(result), td["expected_n_sets"]) + + def test_enrichr_library_bad(self): + # A bad library name must raise RuntimeError; a real network error is a skip, not a failure. + try: + with self.assertRaises(RuntimeError): + enrichr_library("NOT_A_LIBRARY_xyz", verbose=False) + except requests.RequestException as e: + self.skipTest(f"Network error reaching Enrichr: {e}") + def test_enrichr_plot(self): # Number of plots before running enrichr plot num_figures_before = plt.gcf().number @@ -74,3 +120,152 @@ def test_enrichr_plot(self): num_figures_before, "No matplotlib plt object was created.", ) + + +class _FakeResponse: + """Minimal stand-in for a requests.Response used to test enrichr_library offline.""" + + def __init__(self, text, ok=True): + self.text = text + self.ok = ok + + +# A valid Enrichr gene-set-library text payload (tab-separated, with a blank line). +_LIBRARY_TEXT = "SET_A\t\tGENE1\tGENE2\tGENE3\n\nSET_B\tdescription\tGENE4\tGENE5\n" + + +class TestEnrichrLibraryOffline(unittest.TestCase): + """Network-free tests of enrichr_library parsing/branches (issue #139).""" + + def test_invalid_species_raises(self): + with self.assertRaises(ValueError): + enrichr_library("MSigDB_Hallmark_2020", species="martian", verbose=False) + + @patch.object(gget_enrichr.requests, "get") + def test_parse_verbose(self, mock_get): + # Covers verbose logging, the blank-line skip, and full parsing. + mock_get.return_value = _FakeResponse(_LIBRARY_TEXT) + df = enrichr_library("MSigDB_Hallmark_2020", verbose=True) + self.assertEqual(list(df.columns), ["gene_set", "gene"]) + self.assertEqual(set(df["gene_set"]), {"SET_A", "SET_B"}) + self.assertEqual(df.shape[0], 5) + + @patch.object(gget_enrichr.requests, "get") + def test_bad_library_html_raises(self, mock_get): + # Enrichr returns an HTML page for unknown library names. + mock_get.return_value = _FakeResponse("HTTP Status 404") + with self.assertRaises(RuntimeError): + enrichr_library("DoesNotExist", verbose=False) + + @patch.object(gget_enrichr.requests, "get") + def test_empty_library_raises(self, mock_get): + # A response whose sets contain no member genes is treated as empty. + mock_get.return_value = _FakeResponse("SET_A\tdescription\n") + with self.assertRaises(RuntimeError): + enrichr_library("EmptyLib", verbose=False) + + @patch.object(gget_enrichr.requests, "get") + def test_gene_set_filter(self, mock_get): + mock_get.return_value = _FakeResponse(_LIBRARY_TEXT) + df = enrichr_library("MSigDB_Hallmark_2020", gene_set="SET_A", verbose=False) + self.assertEqual(set(df["gene_set"]), {"SET_A"}) + + @patch.object(gget_enrichr.requests, "get") + def test_gene_set_not_found_raises(self, mock_get): + mock_get.return_value = _FakeResponse(_LIBRARY_TEXT) + with self.assertRaises(ValueError): + enrichr_library("MSigDB_Hallmark_2020", gene_set="NOPE", verbose=False) + + @patch.object(gget_enrichr.requests, "get") + def test_json_and_save(self, mock_get): + # Covers the json return, json+save, and CSV save branches. + mock_get.return_value = _FakeResponse(_LIBRARY_TEXT) + result = enrichr_library("MSigDB_Hallmark_2020", json=True, verbose=False) + self.assertIsInstance(result, dict) + self.assertEqual(result["SET_A"], ["GENE1", "GENE2", "GENE3"]) + with tempfile.TemporaryDirectory() as tmp: + cwd = os.getcwd() + os.chdir(tmp) + try: + enrichr_library("MSigDB_Hallmark_2020", save=True, verbose=False) + self.assertTrue(any(f.endswith(".csv") for f in os.listdir("."))) + enrichr_library("MSigDB_Hallmark_2020", json=True, save=True, verbose=False) + self.assertTrue(any(f.endswith(".json") for f in os.listdir("."))) + finally: + os.chdir(cwd) + + @patch.object(gget_enrichr.requests, "get") + def test_descriptions(self, mock_get): + # descriptions=True keeps the (often empty) description field. + mock_get.return_value = _FakeResponse(_LIBRARY_TEXT) + df = enrichr_library("MSigDB_Hallmark_2020", descriptions=True, verbose=False) + self.assertEqual(list(df.columns), ["gene_set", "description", "gene"]) + self.assertEqual(df[df["gene_set"] == "SET_B"]["description"].iloc[0], "description") + self.assertEqual(df[df["gene_set"] == "SET_A"]["description"].iloc[0], "") + # json + descriptions -> {set: {"description", "genes"}} + result = enrichr_library("MSigDB_Hallmark_2020", descriptions=True, json=True, verbose=False) + self.assertEqual(result["SET_B"], {"description": "description", "genes": ["GENE4", "GENE5"]}) + + +class _FakeJsonResponse: + """Minimal stand-in for a requests.Response returning JSON (for enrichr_libraries).""" + + def __init__(self, payload, ok=True, status_code=200): + self._payload = payload + self.ok = ok + self.status_code = status_code + + def json(self): + return self._payload + + +_STATS_PAYLOAD = { + "statistics": [ + {"libraryName": "MSigDB_Hallmark_2020", "numTerms": 50, "geneCoverage": 4383, "genesPerTerm": 146}, + {"libraryName": "KEGG_2021_Human", "numTerms": 300, "geneCoverage": 8000, "genesPerTerm": 90}, + {"libraryName": "MSigDB_Computational", "numTerms": 858, "geneCoverage": 10061, "genesPerTerm": 106}, + ] +} + + +class TestEnrichrLibrariesOffline(unittest.TestCase): + """Network-free tests of enrichr_libraries (library discovery, issue #139).""" + + def test_invalid_species_raises(self): + with self.assertRaises(ValueError): + enrichr_libraries(species="martian", verbose=False) + + @patch.object(gget_enrichr.requests, "get") + def test_list_columns_and_sort(self, mock_get): + mock_get.return_value = _FakeJsonResponse(_STATS_PAYLOAD) + df = enrichr_libraries(verbose=True) + self.assertEqual(list(df.columns), ["library", "num_terms", "gene_coverage", "genes_per_term"]) + # Sorted case-insensitively by library name + self.assertEqual(list(df["library"]), ["KEGG_2021_Human", "MSigDB_Computational", "MSigDB_Hallmark_2020"]) + + @patch.object(gget_enrichr.requests, "get") + def test_filter_case_insensitive(self, mock_get): + mock_get.return_value = _FakeJsonResponse(_STATS_PAYLOAD) + df = enrichr_libraries(filter="msigdb", verbose=False) + self.assertEqual(set(df["library"]), {"MSigDB_Hallmark_2020", "MSigDB_Computational"}) + + @patch.object(gget_enrichr.requests, "get") + def test_json_return(self, mock_get): + mock_get.return_value = _FakeJsonResponse(_STATS_PAYLOAD) + result = enrichr_libraries(filter="Hallmark", json=True, verbose=False) + self.assertIsInstance(result, list) + self.assertEqual(result[0]["library"], "MSigDB_Hallmark_2020") + + @patch.object(gget_enrichr.requests, "get") + def test_bad_status_raises(self, mock_get): + mock_get.return_value = _FakeJsonResponse({}, ok=False, status_code=503) + with self.assertRaises(RuntimeError): + enrichr_libraries(verbose=False) + + @patch.object(gget_enrichr.requests, "get") + def test_mouse_maps_to_human(self, mock_get): + # 'mouse' has no dedicated Enrichr variant; it must query the human endpoint, + # consistent with enrichr_library / enrichr. + mock_get.return_value = _FakeJsonResponse(_STATS_PAYLOAD) + enrichr_libraries(species="mouse", verbose=False) + self.assertEqual(mock_get.call_args[0][0], gget_enrichr.DATASETSTATISTICS_ENRICHR_URLS["human"])