I came across a post about the quickest way to remove duplicates from a python list. This particular problem seems to crop up often, and I've noticed a huge slow down when doing something like:
I never took the time to benchmark different techniques like Peter did, though. It's interesting to see that:
new_list =  for i in old_list: if i not in new_list: new_list.append(i)
is the fastest method, but only a hair faster than the conceptually simpler:
Most of the time, I use
list(set(seq)) or just leave the sequence as a set without turning it back into a list. Also, the results present a strong argument against preserving order: even the fastest order-preserving method took 2.5 times as long as the quickest non-order-preserving method.
good to know.