ndidates with `X` and `y` of increasing sizes. Parameters ---------- evaluate_candidates : callable This callback accepts: - a list of candidates, where each candidate is a dict of parameter settings. - an optional `cv` parameter which can be used to e.g. evaluate candidates on different dataset splits, or evaluate candidates on subsampled data (as done in the SucessiveHaling estimators). By default, the original `cv` parameter is used, and it is available as a private `_checked_cv_orig` attribute. - an optional `more_results` dict. Each key will be added to the `cv_results_` attribute. Values should be lists of length `n_candidates` It returns a dict of all results so far, formatted like ``cv_results_``. Important note (relevant whether the default cv is used or not): in randomized splitters, and unless the random_state parameter of cv was set to an int, calling cv.split() multiple times will yield different splits. Since cv.split() is called in evaluate_candidates, this means that candidates will be evaluated on different splits each time evaluate_candidates is called. This might be a methodological issue depending on the search strategy that you're implementing. To prevent randomized splitters from being used, you may use _split._yields_constant_splits() Examples -------- :: def _run_search(self, evaluate_candidates): 'Try C=0.1 only if C=1 is better than C=10' all_results = evaluate_candidates([{'C': 1}, {'C': 10}]) score = all_results['mean_test_score'] if score[0] < score[1]: evaluate_candidates([{'C': 0.1}]) z