x in my_list slow for millions of items?A dedup step checks if text in seen: for 2 million examples, where seen is a list. What is the cost of each check, and the fix?
Answer: B. O(n) per check; make seen a set for O(1) average lookups
A list is scanned item by item, so each check is O(n) and the job is O(n²). A set hashes the item and jumps to one slot: O(1) on average.