SciReviewAI: LLM-RAG for Literature Review

A refined approach to customizing Large Language Models with Retrieval-Augmented Generation for multi-domain scientific literature review.

Biomedical AI/ML Engineering Physics Math

Retrieved Papers

42 results

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

arXiv 2020

Lewis et al. propose RAG models that combine pre-trained parametric and non-parametric memory for language generation.

NLP RAG Architecture
124 citations PDF available

BioBERT: a pre-trained biomedical language representation model for biomedical text mining

Bioinformatics 2020

Lee et al. present BioBERT, a domain-specific language representation model pre-trained on large-scale biomedical corpora.

Biomedical BERT Text Mining
892 citations PDF available

Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

ACM 2021

Gu et al. investigate different strategies for pretraining language models in the biomedical domain and evaluate them on biomedical NLP tasks.

Biomedical Pretraining Evaluation
256 citations PDF available

Efficient Retrieval-Augmented Generation for Scientific Literature Review

Nature Methods 2022

Zhang et al. propose a hybrid retrieval-generation framework specifically optimized for scientific literature review tasks.

Literature Review Retrieval Generation
78 citations PDF available

Multi-Domain Adaptation for Scientific Literature Summarization

AAAI 2023

Wang et al. present a novel approach for adapting summarization models across multiple scientific domains with minimal retraining.

Multi-Domain Adaptation Summarization
42 citations PDF available

AI-Generated Summary

Generated by our fine-tuned BioBERT-RoBERTa hybrid model with RAG

The literature review reveals several key approaches to customizing LLMs for scientific literature review:

(Lewis et al., 2020) introduced Retrieval-Augmented Generation (RAG) models that combine parametric memory (a pre-trained seq2seq model) with non-parametric memory (a dense vector index of Wikipedia). This approach shows promise for knowledge-intensive tasks.

For biomedical applications, (Lee et al., 2020) demonstrated the effectiveness of domain-specific pretraining with BioBERT, achieving state-of-the-art results on biomedical NLP tasks. Their work highlights the importance of domain adaptation.

(Zhang et al., 2022) optimized RAG specifically for scientific literature review, implementing a hybrid retrieval-generation framework with improved efficiency for large document collections.

Recent work by (Wang et al., 2023) addresses multi-domain challenges, proposing adaptation techniques that enable a single model to handle diverse scientific literature with minimal retraining.

Confidence:
85%

Citation Analysis

Citation Accuracy

92% verified 8% potential issues

Citation Distribution

PubMed 12
arXiv 5
IEEE 3

Potential Issues

Citation to "Zhang et al., 2022" may refer to multiple papers - verify context.

"Wang et al., 2023" preprint - consider peer-reviewed alternatives.

Provide Feedback

Help improve our model by rating this summary

Methodology Overview

Data Collection

  • PubMed for biomedical research
  • arXiv for CS/physics/math preprints
  • IEEE Xplore for engineering
  • Google Scholar for benchmarking

Model Architecture

  • BioBERT for biomedical domains
  • RoBERTa for general science
  • DPR with FAISS indexing
  • End-to-end RAG training

Evaluation Metrics

  • ROUGE-L, BLEU for summaries
  • Citation accuracy percentage
  • User satisfaction surveys
  • Comparison with Elicit/Scispace

RAG Training Pipeline

1
Data Preprocessing

GROBID for PDF parsing, text normalization, SciBERT tokenization

2
Supervised Fine-Tuning

Train on S2ORC for summary generation with embedded citations

3
RLHF Refinement

Researcher feedback on citation relevance and clarity

4
End-to-End Training

Contrastive loss to align retriever-generator outputs

Made with DeepSite LogoDeepSite - 🧬 Remix