🚀 Scaling Client-Side Search: 100,000 Users, Skills, and Real-Time Filtering in React

Posted on by — No Comments ↓

Today, I came across a post on dev.to from Fatemeh Paghar on using useTransition and react-window. I decided to push a typical React search/filter UI to the extreme by scaling it up to handle 100,000 users — not a typo — one hundred thousand records client-side.

Instead of simple emails, each user now has:

  • A name
  • A skillset (e.g., React, Python, AWS)

To make it even more interesting, I added a second search field — but with a twist:

You must first search by name before the skill search field becomes active.

🛠 How It Works

  • First input: Search by name (e.g., “Alice”, “Bob”)
  • Second input: Search by skill (e.g., “React”, “Rust”), but it remains disabled until a name is entered.

Once a name is typed, the skill search input unlocks and users can further refine their search.

⚡ First: The Basic Naive Version (No Optimization)

Let’s look at a simple implementation first — without react-window and without useMemo. 

"use client"
import React, { useState, useTransition } from "react";

const generateUsers = (count: number) => {
  const names = ["Alice", "Bob", "Charlie", "David", "Eve", "Frank", "Grace", "Helen"];
  const skills = ["React", "Python", "Node.js", "Django", "Rust", "Go", "Flutter", "AWS"];

  return Array.from({ length: count }, (_, i) => ({
    id: i,
    name: `${names[i % names.length]} ${i}`,
    skill: skills[i % skills.length],
  }));
};

const usersData = generateUsers(100000);

export default function BasicMassiveSearch() {
  const [nameQuery, setNameQuery] = useState("");
  const [skillQuery, setSkillQuery] = useState("");
  const [isPending, startTransition] = useTransition();

  const handleNameSearch = (e: React.ChangeEvent<HTMLInputElement>) => {
    const value = e.target.value;
    startTransition(() => setNameQuery(value));
  };

  const handleSkillSearch = (e: React.ChangeEvent<HTMLInputElement>) => {
    const value = e.target.value;
    startTransition(() => setSkillQuery(value));
  };

  const filteredUsers = usersData.filter((user) => {
    const matchesName = user.name.toLowerCase().includes(nameQuery.toLowerCase());
    if (!matchesName) return false;
    if (skillQuery) {
      return user.skill.toLowerCase().includes(skillQuery.toLowerCase());
    }
    return true;
  });

  return (
    <div className="p-6">
      <h1 className="text-2xl font-bold mb-4">Simple Skill Search</h1>

      <input
        type="text"
        value={nameQuery}
        onChange={handleNameSearch}
        placeholder="Search by name..."
        className="border p-2 mb-4 block w-full"
      />

      <input
        type="text"
        value={skillQuery}
        onChange={handleSkillSearch}
        disabled={!nameQuery}
        placeholder="Search by skill..."
        className="border p-2 mb-4 block w-full"
      />

      {isPending && <p className="italic text-sm">Filtering...</p>}

      <div className="mt-4">
        {filteredUsers.length === 0 ? (
          <p className="italic text-gray-500">No users found.</p>
        ) : (
          filteredUsers.map((user) => (
            <div key={user.id} className="p-2 border-b">
              <p className="font-bold">{user.name}</p>
              <p className="text-sm text-gray-600">{user.skill}</p>
            </div>
          ))
        )}
      </div>
    </div>
  );
}

😵 Problems With This Naive Version

  • No Virtualization:
    Every single result (up to 100,000 DOM elements) renders at once.
    ➔ Browser lags, page freezes, and scrolling becomes painful.
  • No Memoization:
    Every re-render recomputes .filter() unnecessarily.
    ➔ Causes typing delays as the dataset grows.
  • Performance collapse:
    Mobile users and low-end laptops will suffer first.

✅ Now: The Optimized Version (Virtualized + Memoized)

"use client"
import React, { useState, useMemo, useTransition } from "react";
import { FixedSizeList as List } from "react-window";

const generateUsers = (count: number) => {
  const names = ["Alice", "Bob", "Charlie", "David", "Eve", "Frank", "Grace", "Helen"];
  const skills = ["React", "Python", "Node.js", "Django", "Rust", "Go", "Flutter", "AWS"];

  return Array.from({ length: count }, (_, i) => ({
    id: i,
    name: `${names[i % names.length]} ${i}`,
    skill: skills[i % skills.length],
  }));
};

const usersData = generateUsers(100000);

export default function MassiveSkillSearch() {
  const [nameQuery, setNameQuery] = useState("");
  const [skillQuery, setSkillQuery] = useState("");
  const [isPending, startTransition] = useTransition();
  const [searchDuration, setSearchDuration] = useState<number>(0);

  const filteredUsers = useMemo(() => {
    const start = performance.now();

    const lowerName = nameQuery.toLowerCase();
    const lowerSkill = skillQuery.toLowerCase();

    const result = usersData.filter((user) => {
      const matchesName = user.name.toLowerCase().includes(lowerName);
      if (!matchesName) return false;
      if (lowerSkill) {
        return user.skill.toLowerCase().includes(lowerSkill);
      }
      return true;
    });

    const end = performance.now();
    setSearchDuration(end - start);
    return result;
  }, [nameQuery, skillQuery]);

  const handleNameSearch = (e: React.ChangeEvent<HTMLInputElement>) => {
    const value = e.target.value;
    startTransition(() => setNameQuery(value));
  };

  const handleSkillSearch = (e: React.ChangeEvent<HTMLInputElement>) => {
    const value = e.target.value;
    startTransition(() => setSkillQuery(value));
  };

  const Row = ({ index, style }: { index: number; style: React.CSSProperties }) => {
    const user = filteredUsers[index];
    return (
      <div style={style} key={user.id} className="p-3 border-b border-green-100 hover:bg-green-50">
        <p className="font-medium text-gray-800">{user.name}</p>
        <p className="text-sm text-gray-500">{user.skill}</p>
      </div>
    );
  };

  return (
    <div className="p-6">
      <h1 className="text-2xl font-bold mb-4">🧠 Skillset Search (Virtualized)</h1>

      <input
        type="text"
        value={nameQuery}
        onChange={handleNameSearch}
        placeholder="Search by name..."
        className="border p-2 mb-4 block w-full"
      />

      <input
        type="text"
        value={skillQuery}
        onChange={handleSkillSearch}
        disabled={!nameQuery}
        placeholder="Search by skill..."
        className="border p-2 mb-4 block w-full"
      />

      {isPending && <p className="italic text-sm">Filtering...</p>}

      {filteredUsers.length === 0 ? (
        <p className="italic text-gray-500">No users found.</p>
      ) : (
        <List height={500} itemCount={filteredUsers.length} itemSize={70} width="100%">
          {Row}
        </List>
      )}

      <div className="mt-6 text-sm text-gray-600 text-center">
        <p>Total users: {usersData.length}</p>
        <p>Filtered users: {filteredUsers.length}</p>
        <p>Last search took: {searchDuration.toFixed(2)} ms</p>
        <p>{isPending ? "Searching..." : "Idle"}</p>
      </div>
    </div>
  );
}

🧠 What Changed — Why It Matters

Feature Basic Version Optimized Version
Rendering 100,000 real DOM nodes Only ~20 visible DOM nodes
Filtering Recalculates .filter() every render Memorized with useMemo, recalculates only when query changes
Scrolling Laggy and crash-prone Smooth and buttery
Typing Freezes momentarily on big datasets Remains fluid
CPU Usage Very high Low and efficient

🏁 Final Verdict

✅ If you are just prototyping or handling small datasets (few hundred items),

you can survive without virtualization and memoization.

❗ But the moment you deal with tens of thousands of records (or more),

you must:

  • Virtualize large lists
  • Use useMemo smartly
  • Possibly move search server-side

Otherwise, your app will become unusable — fast.

🎯 Closing Note

React gives you amazing flexibility, but scale tests your architecture.

Always think about memory, CPU, and user experience as you grow datasets!

Leave a Reply