core are parallelized over the permutations. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. random_state : int, RandomState instance or None, default=0 Pass an int for reproducible output for permutation of ``y`` values among samples. See :term:`Glossary `. verbose : int, default=0 The verbosity level. scoring : str or callable, default=None A single str (see :ref:`scoring_parameter`) or a callable (see :ref:`scoring_callable`) to evaluate the predictions on the test set. If `None` the estimator's score method is used. fit_params : dict, default=None Parameters to pass to the fit method of the estimator. .. deprecated:: 1.6 This parameter is deprecated and will be removed in version 1.6. Use ``params`` instead. params : dict, default=None Parameters to pass to the `fit` method of the estimator, the scorer and the cv splitter. - If `enable_metadata_routing=False` (default): Parameters directly passed to the `fit` method of the estimator. - If `enable_metadata_routing=True`: Parameters safely routed to the `fit` method of the estimator, `cv` object and `scorer`. See :ref:`Metadata Routing User Guide ` for more details. .. versionadded:: 1.6 Returns ------- score : float The true score without permuting targets. permutation_scores : array of shape (n_permutations,) The scores obtained for each permutations. pvalue : float The p-value, which approximates the probability that the score would be obtained by chance. This is calculated as: `(C + 1) / (n_permutations + 1)` Where C is the number of permutations whose score >= the true score. The best possible p-value is 1/(n_permutations + 1), the worst is 1.0. Notes ----- This function implements Test 1 in: Ojala and Garriga. `Permutation Tests for Studying Classifier Performance `_. The Journal of Machine Learning Research (2010) vol. 11 Examples -------- >>> from sklearn.datasets import make_classification >>> from sklearn.linear_model import LogisticRegression >>> from sklearn.model_selection import permutation_test_score >>> X, y = make_classification(random_state=0) >>> estimator = LogisticRegression() >>> score, permutation_scores, pvalue = permutation_test_score( ... estimator, X, y, random_state=0 ... ) >>> print(f"Original Score: {score:.3f}") Original Score: 0.810 >>> print( ... f"Permutation Scores: {permutation_scores.mean():.3f} +/- " ... f"{permutation_scores.std():.3f}" ... ) Permutation Scores: 0.505 +/- 0.057 >>> print(f"P-value: {pvalue:.3f}") P-value: 0.010 ú