@fragments-sdk/context

@fragments-sdk/context

Source-available. This package is distributed publicly on npm under FSL-1.1-MIT; source stays private. Contribute by filing an issue at https://github.com/fragments-sdk/fragments/issues.

Code intelligence and RAG primitives for codebase indexing and search.

Modules

Chunking (@fragments-sdk/context/chunking)

Split source files into semantically meaningful chunks for embedding.

• chunkFile(content, filePath, language) — Auto-selects AST or line-based chunking
• chunkByAST(content, filePath, language, grammar) — Tree-sitter AST-aware chunking (preserves functions, classes, etc.)
• chunkByLines(content, filePath, language) — Sliding-window fallback

AST chunking supports: TypeScript, TSX, JavaScript, Python, Go, Rust, Java.

import { chunkFile } from "@fragments-sdk/context/chunking";

const chunks = await chunkFile(code, "src/auth.ts", "typescript");
// Each chunk has: content, filePath, startLine, endLine, language, symbolName?, scopeChain?

Embeddings (@fragments-sdk/context/embeddings)

Generate code-optimized embeddings via Voyage AI.

• generateEmbeddings(texts, options) — Batch embed with voyage-code-3
• rerankResults(query, documents, options) — Cross-encoder reranking with rerank-2.5

import { generateEmbeddings } from "@fragments-sdk/context/embeddings";

const vectors = await generateEmbeddings(chunks.map(c => c.content), {
  apiKey: process.env.VOYAGE_API_KEY!,
  inputType: "document",
});

Search (@fragments-sdk/context/search)

Post-retrieval utilities for combining and deduplicating results.

• reciprocalRankFusion(resultSets) — Merge multiple ranked lists (e.g. vector + keyword)
• deduplicateChunks(chunks) — Remove near-duplicate results

Indexing (@fragments-sdk/context/indexing)

File filtering, change detection, and language detection.

• shouldIndexFile(path, size) — Filter by extension, size, ignored dirs
• detectLanguage(path) — Map file extension to language name
• resolveChanges(oldFiles, newFiles) — Diff two file trees for incremental re-indexing
• hashContent(content) — Content hashing for change detection

Install

pnpm add @fragments-sdk/context

Requires Node.js 18+. Tree-sitter WASM grammars are included as dependencies.