Hey Andrew, many thanks for the question! You raised a good point, and first, I also thought that using the same method for both creation and detection could lead to overfitting, bias, or blind spots. But there are other remarks in the paper:

"AI-generated text is similar to human text in terms of unigram distribution, length, stopword usage, and adherence to Zipf’s rank-frequency law. Ippolito et al. (2020) argued that improved model decoding strategies, particularly top-k sampling, are better at fooling humans but simultaneously introduce statistical abnormalities, such as different unigram distributions, that make generated text easier to detect."

Also

"we evaluated the performance of our model without access to ada and/or davinci, or without access to any of the handcrafted features, finding that a model which uses only n-gram and ada features (i.e., the without davinci condition) nearly matched the performance of our full model."

Hope this helps!