scholargraph

Academic literature intelligence toolkit for multi-source paper search, analysis, and knowledge graph building with AI assistance.

# ScholarGraph - Academic Literature Intelligence Toolkit

## Overview

ScholarGraph is a comprehensive academic literature intelligence toolkit that helps researchers efficiently search, analyze, and manage academic papers using AI-powered tools. Features 11 academic search sources with intelligent domain-based source selection and PDF download capabilities.

## Security & Privacy

This skill operates with the following permissions:

- **Network Access**: Queries academic APIs (arXiv, Semantic Scholar, OpenAlex, PubMed, CrossRef, DBLP, IEEE, CORE, Google Scholar, Unpaywall) and web search services
- **File System**: Reads/writes configuration files, downloads PDFs, stores knowledge graphs in SQLite database (`data/knowledge-graphs.db`)
- **LLM Integration**: Sends custom system prompts to AI providers for structured JSON output (concept extraction, paper analysis, etc.)
- **Optional Python**: PDF figure extraction (pymupdf) and PPT export (python-pptx) require Python 3.8+

**Data Storage**: All data is stored locally. No telemetry or analytics are collected.

**API Keys**: Optional API keys are only used for their respective services and are never transmitted elsewhere.

**Source Code**: Open source under MIT license at https://github.com/Josephyb97/ScholarGraph

## Features

### Core Modules (6)

1. **Literature Search** - Multi-source academic paper discovery (11 sources)
   - **Free sources**: arXiv, Semantic Scholar, OpenAlex (250M+), PubMed (biomedical), CrossRef (150M+ DOI), DBLP (CS), Web Search
   - **API-key sources**: IEEE Xplore, CORE, Google Scholar (SerpAPI), Unpaywall (OA PDF)
   - Adapter-based plugin architecture for easy extension
   - Complementary search strategy with auto domain detection (biomedical/cs/engineering/physics)
   - Priority-based source selection per domain
   - Query expansion for better search results
   - PDF download with multi-strategy URL resolution

2. **Concept Learner** - Rapid knowledge framework construction
   - Generate structured learning cards
   - Include code examples and related papers
   - Support beginner/intermediate/advanced depth levels

3. **Knowledge Gap Detector** - Proactive blind spot identification
   - Analyze knowledge coverage in specific domains
   - Identify critical, recommended, and optional gaps
   - Provide learning recommendations and time estimates

4. **Progress Tracker** - Real-time field monitoring
   - Track research topics and keywords
   - Generate daily/weekly/monthly reports
   - Monitor trending papers and topics

5. **Paper Analyzer** - Deep paper analysis
   - Extract key contributions and insights
   - Support quick/standard/deep analysis modes
   - Generate structured analysis reports

6. **Knowledge Graph Builder** - Concept relationship visualization
   - Build interactive knowledge graphs
   - Support Mermaid and JSON output formats
   - Find learning paths between concepts
   - SQLite-based persistent storage
   - Bidirectional concept-paper indexing

### Advanced Features (9)

7. **Review Detector** - Automatic review paper identification
   - Multi-dimensional scoring (title 30% + citations 25% + abstract 25% + AI 20%)
   - Chinese and English keyword support
   - Confidence-based filtering with user confirmation

8. **Concept Extractor** - Extract concepts from review papers
   - AI-powered extraction of 15-30 core concepts
   - Four-level categorization (foundation/core/advanced/application)
   - Importance scoring and relationship identification
   - Cross-review deduplication and merging

9. **Review-to-Graph Workflow** - End-to-end pipeline
   - Search reviews -> Detect -> Confirm -> Analyze -> Extract concepts
   - Build knowledge graph -> Enrich with key papers -> Index -> Store
   - Interactive or automatic confirmation mode

10. **Knowledge Graph Query** - Bidirectional literature indexing
    - Concept -> papers: find papers related to a concept
    - Paper -> concepts: find concepts covered by a paper
    - Paper recommendations based on multiple concepts
    - SQLite-optimized high-performance queries

11. **Compare Concepts** - Compare two concepts
    - Identify similarities and differences
    - Provide use case recommendations

12. **Compare Papers** - Compare multiple papers
    - Find common themes and differences
    - Generate synthesis analysis

13. **Critique** - Critical paper analysis
    - Identify strengths and weaknesses
    - Find research gaps and improvement suggestions
    - Support custom focus areas

14. **Learning Path** - Find optimal learning paths
    - Discover paths between concepts
    - Generate topological learning order
    - Visualize with Mermaid diagrams

15. **Graph Management** - Manage persistent knowledge graphs
    - List all saved graphs
    - View graph statistics
    - Export graphs to JSON
    - Visualize with Mermaid

16. **Paper Visualization** - Interactive paper presentation
    - Convert paper analysis to HTML slide presentations
    - Academic dark/light themes with responsive typography
    - Keyboard/touch/scroll navigation, edit mode (E key)
    - PDF figure extraction (pymupdf) and PPT export (python-pptx)
    - 8+ slides: title, abstract, key points, methodology, experiments, contributions, limitations, references

17. **Interactive Knowledge Graph** - D3.js force-directed visualization
    - Convert knowledge graphs to interactive HTML with D3.js v7
    - Node size reflects paper count, edge thickness reflects concept tightness
    - Zoom/pan, node dragging, click-to-detail panel, search, legend
    - Paper preview bridge: click "View Presentation" to open paper slides in new tab
    - Category colors: foundation=#4FC3F7, core=#FFB74D, advanced=#CE93D8, application=#81C784

## Technical Features

- **11 Academic Search Sources**: arXiv, Semantic Scholar, OpenAlex, PubMed, CrossRef, DBLP, IEEE Xplore, CORE, Google Scholar, Unpaywall, Web Search
- **Complementary Search Strategy**: Auto-detects query domain and selects optimal source combination
- **Adapter Pattern**: Plugin-based search source architecture for easy extension
- **PDF Download**: Multi-strategy URL resolution (direct, Unpaywall, OpenAlex OA, CORE)
- **Multi-AI Provider Support**: 15+ AI providers including OpenAI, Anthropic, DeepSeek, Qwen, Zhipu AI, etc.
- **SQLite Persistence**: Knowledge graphs stored in SQLite database via bun:sqlite
- **Bidirectional Indexing**: Concept-paper and paper-concept bidirectional query support
- **Rate Limiting**: Per-source rate limiting with automatic retry and delay
- **Interactive HTML Output**: Paper slide presentations, D3.js knowledge graph visualizations
- **Multiple Output Formats**: Markdown, JSON, Mermaid, HTML, PPTX
- **TypeScript + Bun**: Fast and type-safe runtime
- **CLI + API**: Both command-line and programmatic interfaces

## Installation

```bash
# Clone repository
git clone https://github.com/Josephyb97/ScholarGraph.git
cd ScholarGraph

# Install dependencies
bun install

# Initialize configuration
bun run cli.ts config init
```

## Configuration

Set up your AI provider:

```bash
# Using OpenAI
export AI_PROVIDER=openai
export OPENAI_API_KEY="your-api-key"

# Using DeepSeek
export AI_PROVIDER=deepseek
export DEEPSEEK_API_KEY="your-api-key"

# Using Qwen (通义千问)
export AI_PROVIDER=qwen
export QWEN_API_KEY="your-api-key"
```

### Academic Source API Keys (optional, expand search coverage)

```bash
export NCBI_API_KEY="your-key"           # PubMed high-speed access (10 req/s)
export IEEE_API_KEY="your-key"           # IEEE Xplore engineering papers
export CORE_API_KEY="your-key"           # CORE open access full text
export UNPAYWALL_EMAIL="your@email.com"  # Unpaywall OA PDF resolver
export CROSSREF_MAILTO="your@email.com"  # CrossRef polite pool (higher rate)
export SERPAPI_KEY="your-key"            # Google Scholar (via SerpAPI)
export SERPER_API_KEY="your-key"         # Web search via Serper
```

## Usage Examples

### Search Literature
```bash
# Auto-select best sources ba