The Abstractinator

The Abstractinator

Immunology & virology literature, searched across databases at once.

abstractinator.me · Connect your AI · Self-host it

---

The Abstractinator turns a literature sweep into one search. It queries Europe PMC, OpenAlex, ClinicalTrials.gov, bioRxiv/medRxiv, NIH RePORTER and NSF Awards in parallel, removes duplicates across sources, and tags every record with its most prominent immune cell type and virus using a weighted title/abstract scoring matrix.

A new search takes about 5–10 seconds; repeat searches within 7 days are instant.

✨ Features

📚 Data sources

Keyed sources are bring-your-own-key: enter keys in Settings (☰). They're held only in your browser session, never logged or written to disk, and never shared with other visitors.

🤖 Use it with AI

The Abstractinator is available as a remote MCP server, the open standard AI assistants use to connect to tools.

https://mcp.abstractinator.me/mcp

In Claude: Settings → Connectors → *Add custom connector* → paste the URL above, and choose No sign-in for authentication. Then just ask, e.g. *"Search the Abstractinator for oncolytic virus trials in melanoma."* Any MCP client that supports remote (streamable HTTP) servers works too.

The server exposes one tool:

`search_literature(query, max_results=25, abstract_chars=1500, deep=False)` returns total_found, source_counts, and a source-balanced list of records with title, abstract, authors, publication_date, doi, url, source, primary_cell, primary_virus and is_bioinformatics.

The AI connector uses open-access sources only (no keys needed or spent) and is rate-limited to keep the service available for everyone.

🏗️ How it works

Browser ──► Cloudflare ──► abstractinator.me      ──► Shiny Server :3838 (app.R) ─────────┐

├─► search engine ──► shared cache AI ──► Cloudflare ──► mcp.abstractinator.me ──► MCP server :8200 (agent/mcp_server.py)│ (orchestrate_extraction.R) └──► REST API :8100 (agent/api.R) ┘

• One search engine, two front doors. orchestrate_extraction.R checks the cache, starts every source as a background job (future), then combines, deduplicates and tags the results. The website calls it asynchronously through Shiny's ExtendedTask, so one visitor's search never freezes the page for anyone else. The REST API calls the same code synchronously.
• Shared cache. Results are cached for 7 days in .cache/s3_mimic/ (one .rds per search, keyed by term, depth and which keyed sources were used). A search by a person makes the same search instant for an AI, and vice versa.
• Local preprints. bioRxiv/medRxiv are searched from biorxiv_local_db/ (one Parquet file per year) via Arrow, instead of hitting their API on every search.
• Hit detection. Titles and abstracts are matched against immune-cell, virus and bacteria alias lists (aliases.R); title hits weigh 20, abstract hits 3, and the top scorer becomes primary_cell / primary_virus / primary_bacteria. Aliases are matched as whole words, so "salmonella" doesn't match "salmonellosis".
• Swappable pools. The Plotinator's pathogen axis is not hard-wired to viruses: a pool switch selects which tagged category it plots, and the same search can be viewed through either lens. Adding a pool (fungi, parasites, drug classes, a lab's own vocabulary) means defining an alias list in aliases.R, tagging it with tag_with_alias_list() in orchestrate_extraction.R, and adding one entry to PATHOGEN_POOLS in app.R — contributions welcome.
• Logging. Website searches write a small JSON record (term, duration, result count, cache hit, and *whether* keys were present, never the keys) to .cache/s3_mimic/logs/. Feedback submitted through the app is saved to .cache/s3_mimic/feedback/ and, if DATA_WEBHOOK_URL is set, forwarded to Discord.
• Display safety. Text from external databases is HTML-escaped before display, with only basic formatting tags (italics, bold, sub/superscript, line breaks) allowed through, and links restricted to plain http(s) URLs.

🖥️ Self-hosting

Requirements

• Linux (tested on Ubuntu), R ≥ 4.1, Python ≥ 3.10 (for the MCP server)
• System libraries:

  sudo apt install -y r-base r-base-dev libcurl4-openssl-dev libssl-dev libxml2-dev \

libsqlite3-dev libfontconfig1-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev \ libpng-dev libtiff5-dev libjpeg-dev libglpk-dev libgmp3-dev libsodium-dev cmake gfortran

- R packages (install into the system library so Shiny Server can see them):

  sudo /usr/bin/Rscript -e 'install.packages(c("shiny","bslib","shinyjs","shinythemes","shinyWidgets",

"tidyverse","DT","plotly","htmltools","htmlwidgets","htmlTable","jsonlite","digest","future", "promises","parallelly","arrow","httr","httr2","RSQLite","DBI","uuid","lubridate","ggbeeswarm", "plumber"), repos = "https://cloud.r-project.org", Ncpus = 8)'

Using conda? Shiny Server and the services use the system R at /usr/bin/R. Install

packages with /usr/bin/Rscript (or conda deactivate first) so they land in the right place.

1. Get the code and configure

git clone https://github.com/ckangas2/abstractinator.git

cd abstractinator

Optional .Renviron in the project root:

INTEGRATE_NIH_NSF=1      # include NIH & NSF grants (the app also sets this)

USER_EMAIL=you@example.org # sent to OpenAlex for its "polite pool" ENABLE_SCOPUS=0 # set to 1 only on a subscribing institution's network DATA_WEBHOOK_URL= # optional Discord webhook for updater notifications

No API keys are needed on the server: the website is strictly bring-your-own-key and never uses keys from .Renviron, and the AI connector uses open-access sources only. (Keys in .Renviron are only read by command-line scripts you run yourself.)

2. Build the preprint store (once, takes several hours)

tmux new -s biorxiv

Rscript write_parquet_bioRxiV.R # resumable: finished years are skipped on rerun

Detach with Ctrl+B, D. Searches pick up each year as soon as it's written.

3. Run locally

/usr/bin/Rscript -e 'shiny::runApp(port = 4000)'   # website

/usr/bin/Rscript agent/run_api.R # REST API on 127.0.0.1:8100 (run from repo root)

4. Production setup (what abstractinator.me runs)

Shiny Server, serving the app at the root and only on localhost (/etc/shiny-server/shiny-server.conf):

run_as shiny;

server { listen 3838 127.0.0.1; location / { app_dir /srv/shiny-server/abstractinator; log_dir /var/log/shiny-server; app_idle_timeout 0; } }

API and MCP services (run as the shiny user):

sudo /usr/bin/python3 -m venv /opt/abstractinator-mcp

sudo /opt/abstractinator-mcp/bin/pip install "mcp[cli]" # MCP Python SDK 2.x sudo cp agent/abstractinator-api.service agent/abstractinator-mcp.service /etc/systemd/system/ sudo systemctl daemon-reload sudo systemctl enable --now abstractinator-api abstractinator-mcp

Daily preprint updates (after the initial build finishes):

sudo -u shiny mkdir -p logs

sudo cp deploy/abstractinator.cron /etc/cron.d/abstractinator # runs update_biorxiv_db.R at 4:15am

Public access via a Cloudflare Tunnel (no open ports): route abstractinator.me → http://localhost:3838 and mcp.abstractinator.me → http://localhost:8200. The REST API stays private.

Updating the live site

cd /srv/shiny-server/abstractinator

sudo -u shiny git pull sudo systemctl restart shiny-server # app.R, www/, search scripts sudo systemctl restart abstractinator-api # agent/api.R, search scripts sudo systemctl restart abstractinator-mcp # agent/mcp_server.py

Running tests

# Fast offline tests (these also run on GitHub with every push)

Rscript -e 'testthat::test_dir("tests/testthat")'

Plus live end-to-end searches against the real databases (needs internet, ~1-2 min)

ABSTRACTINATOR_LIVE_TESTS=1 Rscript -e 'testthat::test_dir("tests/testthat")'

Usage logs

Every search, from the website or an AI agent, is logged without personal data. Searches per day, split by channel:

cd /srv/shiny-server/abstractinator

sudo -u shiny /usr/bin/Rscript -e 'source("R/log_utils_local.R"); print(summarize_search_logs())'

REST API reference (local only)

GET /search?q= &limit=25&abstract_chars=1500&deep=false

• limit: results returned (1–100), picked round-robin across sources
• abstract_chars: truncate abstracts; 0 omits them
• deep: true for up to 250 results per source
• Optional headers X-CORE-Key and X-USPTO-Key enable those sources for the request
• Server-wide limit of 20 searches/minute (HTTP 429 when exceeded)

GET /health returns {"status":"ok"}.

🗂️ Repository layout