How is Arctom AI Mode Better than OpenAI GPT-5.5 and Google Gemini for Scientific Deep Research?

Technical Specification & Benchmark Report  |  Model: ARCTOM-AI-SCIENTIFIC-ADVANCED-001  |  Production  |  May 2026

Abstract

We evaluated Arctom AI Mode against four Gemini configurations and two OpenAI GPT-5.5 configurations on 30 scientific queries spanning drug mechanisms, PK/PD interactions, structure-activity relationships, and emerging therapeutics. Arctom AI achieves 96–98% citation fidelity compared to 20–66% for plain Gemini and 76–92% for GPT-5.5. On complex PK queries, Gemini 2.5 Flash drops to 8%. Arctom AI Fast and Standard deliver first tokens in 13 seconds — vs 110–125 seconds for GPT-5.5.

How citation fidelity is measured: Each reference cited by Gemini is checked programmatically. DOIs are resolved via CrossRef and PMIDs via PubMed to retrieve the real paper. An independent AI verifier then compares the resolved paper against what was claimed — checking whether the topic, authors, and study match. If the DOI returns a 404 (paper does not exist) or the resolved paper is on a different topic, the citation is marked as hallucinated. Arctom AI citations are verified by the model's built-in quality assurance layer.

1. Citation Fidelity

Figure 1: Overall Citation Fidelity

Figure 1. 30 scientific queries. Top: from model weights only. Bottom: with Google Search grounding via Gemini API.

Table 1. Arctom AI achieves 99–100% on complex queries where Gemini 2.5 Flash drops to 8%.

2. Reference Density

Figure 2: Verified References Per Response

Figure 2. Each Arctom AI reference is verified. Gemini references include 44–84% fabricated citations.

3. Reference Quality & Recency

Per-System Breakdown

Citation volume, recency, and journal breadth across all 7 systems.

Total references cited across 30 queries. "Unique Journals" = verified via PubMed for Arctom AI; claimed from text for Gemini (20–66% of Gemini DOIs resolve to wrong papers).

Journal Quality

Arctom AI top journals (verified via PubMed): The New England Journal of Medicine (6), Clinical Pharmacology & Therapeutics (5), Journal of Medicinal Chemistry (4), Pharmacology & Therapeutics (4).

Gemini journal claims: Gemini models collectively claim 131 citations to the New England Journal of Medicine across 30 queries. When we verified each DOI, 58% resolved to real NEJM papers — exclusively landmark trials the model memorized. The remaining 42% were fabricated DOIs with valid NEJM prefix format (10.1056/NEJMoa...) that do not correspond to any existing paper.

GPT-5.5 recency cliff: Plain GPT-5.5 cites 0 papers from 2025–26 across all 30 queries. Even with the web_search tool enabled, only 1% of citations are recent (4 of 360). GPT-5.5's January 2026 training cutoff is structural — web grounding rarely promotes new literature into the response. Arctom AI Standard cites 43% from 2025–26 on the same queries.

4. Speed of Answer

Figure 3: Time to First Token (lower is better)

Figure 3. *Non-Arctom systems = total response time (no streaming). Arctom AI Fast/Standard are ~8× faster than GPT-5.5 while achieving higher citation fidelity.

5. Arctom AI Tier Comparison

6. Head-to-Head Comparisons

Direct comparisons between matched tiers — same-class models, all metrics.

Arctom AI Fast vs Gemini 3.0 Flash

Arctom AI Premium vs Gemini 3.0 Pro

Arctom AI Fast vs Gemini 3.0 Flash + Google Search

Arctom AI Premium vs Gemini 3.0 Pro + Google Search

Arctom AI Standard vs GPT-5.5

Arctom AI Premium vs GPT-5.5 + Web Search

7. Summary

Complex queries are where it matters most. On PK/drug interaction questions, Gemini 2.5 Flash drops to 8% citation fidelity. Arctom AI stays at 99–100%.

2.4× more references. Arctom AI Premium cites 18 verified papers per response versus 8 for Gemini.

GPT-5.5 findings (added May 2026)

GPT-5.5 is the strongest external model we have evaluated against Arctom AI — but still trails Arctom AI by 4–6 points on citation fidelity (92% vs 96–98%) while running ~8× slower.

GPT-5.5 with web_search beats Gemini 3.0 Pro + Search (92% vs 89% CF) and lifts the verified reference count to 15.5 per response — close to Arctom AI Premium's 18.3. Without the web tool, plain GPT-5.5 falls to 76% CF, still ahead of every plain Gemini variant but well below Arctom AI.

Recency is the structural ceiling. GPT-5.5's January 2026 training cutoff means plain calls cite zero papers from 2025–26. Even web_search grounding lifts that to only 1%. Arctom AI Standard cites 43% recent literature on the same queries.

Latency penalty is severe. GPT-5.5 takes 110–125s per query because ~65% of output tokens are invisible reasoning. Arctom AI Standard finishes the same query in 12.8s.

Failure mode for plain GPT-5.5: fabricated identifiers. The model often gets author, journal, year, and title correct but invents the matching PMID. Example from this evaluation: GPT-5.5 cited PMID 18315556 as Wong et al.'s apixaban Factor Xa paper — but that PMID actually resolves to a von Willebrand factor study (the real Wong PMID is 18315548). The DOI for the same reference was correct. This is a structural risk for any clinical workflow that relies on the PMID for traceability.

Appendix A: Evaluation Setup

Eleven systems evaluated: three Arctom AI tiers, four plain Gemini configurations, two Gemini + Google Search configurations, and two OpenAI GPT-5.5 configurations (plain and with the web_search tool). All received identical queries. Non-Arctom systems were prompted to cite sources with DOI/PubMed links. Clinical safety scored by Gemini 3.0 Pro (temp=0, batch). Citation fidelity verified by resolving each DOI via CrossRef and each PMID via PubMed esummary, then matching the resolved paper against the cited claim with an independent LLM verifier (Gemini Flash-Lite, temp=0).

GPT-5.5 calls used OpenAI's /v1/responses endpoint (Chat Completions rejects the web_search tool). GPT-5.5 was added to this report on May 15, 2026, three weeks after its April 24 release.

A.5 Gemini Prompt

All Gemini systems received the following prompt template for each query:

You are a medical research assistant. Answer the following research question thoroughly.
Support every claim with citations to peer-reviewed sources. For each citation, include:
- First author et al.
- Paper title
- Journal name
- Year of publication
- DOI or PubMed link if available

Target approximately [TARGET_WORDS] words for the main answer (excluding references).
Format your references in a numbered list at the end.

Research question: [QUERY]

Gemini models were called via the Gemini API (generativelanguage.googleapis.com) with the prompt above. No search grounding or retrieval tools were enabled — responses are generated entirely from model parameters.

Target word counts were matched to the corresponding Arctom AI tier to ensure comparable output length.

Appendix B: Test Queries

B.1 Common — Drug Mechanisms (8)

B.2 Complex — PK/Drug Interactions (8)

B.3 Niche — SAR/Prodrug Design (7)

B.4 Emerging — Novel Targets (7)

© 2026 Arctom Scientific LLC. All rights reserved.