C++ File Search Engine Project
Index every text file in a folder, then search it like Google — instantly.
What you will build
Point the engine at a folder: it walks every file, builds an inverted index (word → files containing it), and answers searches in milliseconds, ranked by relevance. This is the architecture behind desktop search and a miniature of how real search engines work.
What you will learn
- Recursive directory traversal with <filesystem>
- Building an inverted index
- Tokenising and normalising text
- Ranking results by term frequency
- Persisting an index to disk
Feature checklist
- Indexes all .txt/.md/.cpp files in a tree
- Full-text search over contents
- Results ranked by relevance
- AND queries: every word must appear
- Shows matching line snippets
- Re-index command; saved index loads fast
Starter code
#include <filesystem>
#include <fstream>
#include <iostream>
#include <map>
#include <set>
#include <string>
namespace fs = std::filesystem;
using Index = std::map<std::string, std::map<std::string, int>>; // word -> file -> count
void indexFile(const fs::path& p, Index& idx) {
std::ifstream in(p);
std::string word;
while (in >> word) {
for (auto& c : word) c = std::tolower(c);
word.erase(std::remove_if(word.begin(), word.end(),
[](char c) { return !std::isalnum(c); }), word.end());
if (!word.empty()) idx[word][p.string()]++;
}
}
int main() {
Index idx;
for (auto& e : fs::recursive_directory_iterator("."))
if (e.is_regular_file()) indexFile(e.path(), idx);
std::string query;
std::cout << "Search: ";
std::cin >> query;
auto it = idx.find(query);
if (it != idx.end())
for (auto& [file, count] : it->second)
std::cout << count << "x " << file << "\n";
}Full walkthrough
Step 1: Walk the tree with <filesystem>
std::filesystem::recursive_directory_iterator visits every file under a root folder. Filter by extension — .txt, .md, .cpp — and skip files you cannot open. Indexing is I/O-bound drudgery; the iterator turns what used to be OS-specific directory code into five portable lines.
Step 2: indexFile(): build the inverted index
The core data structure is std::map<std::string, std::set<int>> index — word → set of document IDs (each file gets an int ID, with a side table ID → path). Read a file line by line; split each line into lowercase words (letters/digits accumulate, anything else is a separator); for every word, insert the doc ID into its set AND remember which lines contain it for snippets. 'Inverted' because documents point to words in a normal index; here words point to documents.
void indexFile(int docId, const std::string& path) {
std::ifstream in(path);
std::string line; int lineNo = 0;
while (std::getline(in, line)) {
++lineNo;
for (auto& w : wordsOf(line))
index[w].insert(docId); // + store lineNo for snippets
}
}Step 3: AND queries are set intersections
A query 'socket thread' splits into words; the answer is the INTERSECTION of their doc sets — std::set_intersection from <algorithm>, or iterate the smallest set and test membership in the others. AND semantics (every word must appear) is what the checklist specifies and what makes multi-word queries actually narrow results.
Step 4: Rank by relevance, honestly
Score each matching document by total occurrences of the query words (count during indexing or re-scan the matching files), sort descending. It is crude — real engines weigh rare words more (TF-IDF) — but it obeys the one law of ranking: a file mentioning 'socket' nine times outranks one mentioning it once, and your users can predict why.
Step 5: Show snippets, not just filenames
For each result, print the first matching line (truncated to ~100 chars) with the file path and line number. A search result you cannot judge without opening the file is half a feature — the snippet is what turns a filename list into a search engine.
Step 6: Persist the index for fast reloads
Serialise the index (word, then its doc IDs; plus the doc table and snippet lines) to an index file; on startup, offer 'load index' vs 're-index'. The saved form loads in a blink versus a full tree walk — measure both and print the timings, because the difference is the entire justification.
Step 7: Test with a planted needle
Create a folder of files, plant the phrase 'purple elephant' in exactly one buried file, index, and search 'purple AND elephant' — the one file, with its snippet, or your indexer has a tokenising bug (usually case: queries must be lowercased with the same wordsOf() the indexer used, or 'Purple' finds nothing).
Common mistakes
- Using different tokenising for indexing and querying (case, punctuation), so existing words report as missing.
- Implementing multi-word search as OR, returning everything that mentions either word — the result list explodes and the ranking cannot save it.
- Re-indexing on every startup and calling it done; the checklist's saved-index load is where the speed lesson lives.
Compile and run
g++ -std=c++17 -o search_engine search_engine.cppUses <filesystem> (C++17). With GCC 8 or older you may also need -lstdc++fs; GCC 9+ and current Clang/MSVC need nothing extra.
Sample run
$ ./search_engine
Index folder:
> ./notes
Indexing... 128 files (.txt/.md/.cpp)
Index built: 9,412 distinct words · saved to index.dat
Search (AND query):
> socket thread
3 results in 12 ms:
1. chat_server.cpp (score 14)
line 87: std::thread recvThread(receiveLoop, sock);
2. networking.md (score 9)
line 12: A socket is one endpoint of a two-way link...
3. thread_notes.txt (score 5)
line 3: Every thread needs its own socket buffer...
Search (AND query):
> purple elephant
1 result in 9 ms:
1. ideas/story_draft.txt (score 2)
line 41: Nobody expected the purple elephant at the market.
Search (AND query):
> quitTry it in your browser
No browser demo is included for this project. Build the C++ version below — that is the real program.