Deduplication and LSH

Exact duplicates fall out of hashing the normalized content, but near-duplicates (the same record with a typo, a reordered address, boilerplate that repeats across documents) need similarity, not equality. MinHash estimates Jaccard similarity cheaply, and Locality-Sensitive Hashing buckets similar signatures so you only compare likely pairs instead of all O(n^2). This is a constant reflex when merging messy enterprise data and when curating training corpora.