How to give Claude “Long Term Memory” of your local files (No Docker required)

Posted on by — No Comments ↓

The Problem: RAG is too hard

I have a folder of 50+ PDFs—technical specs, old logs, and some… interesting declassified government documents.

I want to open Claude and ask: “What happened in the 1952 sightings?” and have it answer using my files.

The standard advice for this is a nightmare:

  1. Spin up a Docker container (Chroma/Qdrant).
  2. Write a Python script to parse PDFs (LangChain/LlamaIndex).
  3. Set up an API server to bridge it to the LLM.

People turned “Ctrl+F with semantics” into a distributed systems architecture.

The Solution: local-faiss-mcp

I got tired of the bloat, so I built local-faiss-mcp. It’s a CLI tool that runs 100% locally.

  • No Docker: Just a Python script.
  • No API Keys: Uses local embeddings (all-MiniLM-L6-v2) and FAISS.
  • MCP Native: Connects directly to Claude Desktop.

Here is how to set it up in 5 minutes.

Step 1: Get some “Cool” Data

To test a RAG system, you need real-world data. Let’s use the ODNI Preliminary Assessment on Unidentified Aerial Phenomena. It’s declassified, text-heavy, and full of weird military jargon—perfect for testing semantic search.

  1. Create a folder: mkdir ~/ufo_docs
  2. Download this PDF: ODNI Preliminary Assessment 2021 (PDF)
  3. (Optional) Throw in the CIA Simple Sabotage Manual just for chaos.

Step 2: Install the Tool

It’s a standard Python package. 

pip install local-faiss-mcp

Step 3: Index the Docs (The Magic Part)

We used to have to write ingestion scripts. Now, we just use the CLI.

Run this command to recursively index your new UFO folder: 

local-faiss index "~/ufo_docs/**/*.pdf"

You’ll see a progress bar as it:

  1. Reads the PDFs (using pypdf).
  2. Chunks the text.
  3. Calculates embeddings on your CPU.
  4. Saves a .index file to your disk.

Done. You now have a vector database.

Step 4: Connect to Claude

Open your Claude Desktop config file:

  • Mac: ~/Library/Application Support/Claude/claude_desktop_config.json
  • Windows: %APPDATA%Claudeclaude_desktop_config.json

Add this block: 

{
  "mcpServers": {
    "local-faiss": {
      "command": "local-faiss-mcp",
      "args": ["--index-dir", "~/ufo_docs"]
    }
  }
}

Restart Claude. You should see a little “plug” icon indicate two tools are loaded: ingest_document and query_rag_store.

Or use claude in the terminal directly.

Step 5: Ask Questions

Now for the fun part. Open Claude and ask:

“Using your local faiss knowledge: Does the Pentagon’s refusal to answer questions in the UAP Hearing count as ‘General Interference with Organizations’?”

Claude will:

  1. Call query_rag_store with your question.
  2. The CLI will search your FAISS index and return the top N chunks.
  3. Claude will synthesize the answer with citations.

Advanced: Enabling “God Mode” (Re-ranking)

If your dataset is huge, standard vector search can sometimes be “fuzzy.” You might ask about “Project Mogul” and get results about “Project Grudge.”

In v0.2.0, I added Re-ranking. This uses a second, smarter model (CrossEncoder) to double-check the results before giving them to Claude.

Update your config to enable it: 

"args": ["--index-dir", "~/ufo_docs", "--rerank"]

Now the accuracy is much better, running entirely on your laptop.

Summary

We don’t need Kubernetes to chat with a PDF. We just need a good CLI.

If you try this out with your own docs (or the UFO files!), let me know how the re-ranking performs for you.

🔗 Repo: github.com/nonatofabio/local_faiss_mcp

Leave a Reply