Adversarial Validation

Sydney suspected that her training and test data didn't come from the same distribution.

To prove it, she mixed all the data, removed the target variable, and added a new binary target that contained a value of 1 for each training sample and a value of 0 for each test sample.

She then trained a new binary classification model using this new dataset to see whether she could separate the samples based on whether they belonged to the training or test sets.

Sydney used a ROC curve to evaluate the results of the model. She looked at the area under the curve to make her final determination.

Based on Sydney's strategy, which of the following is the correct way to evaluate her model?

If the area under the curve is close to 1.0, the training and test samples come from the same distribution, and if it's close to 0.5, they come from different distributions

If the area under the curve is close to 0.5, the training and test samples come from the same distribution, and if it's close to 1.0, they come from different distributions.

If the area under the curve is close to 1.0, the training and test samples come from the same distribution, and if it's close to 0.0, they come from different distributions.

The area under the curve is not a good metric to determine whether this new model performs accurately.