Skip to content

Documentation for SMACOptimizer

SMAC Wrapper for PHOTONAI.

SMAC (sequential model-based algorithm configuration) is a versatile tool for optimizing algorithm parameters. The main core consists of Bayesian Optimization in combination with an aggressive racing mechanism to efficiently decide which of two configurations performs better.

SMAC usage and implementation details here.

References:

1
2
3
Hutter, F. and Hoos, H. H. and Leyton-Brown, K.
Sequential Model-Based Optimization for General Algorithm Configuration
In: Proceedings of the conference on Learning and Intelligent OptimizatioN (LION 5)

Examples:

1
2
3
4
5
6
my_pipe = Hyperpipe('smac_example',
                    optimizer='smac',
                    optimizer_params={"facade": "SMAC4BO",
                                      "wallclock_limit": 60.0*10,  # seconds
                                      "ta_run_limit": 100},  # limit of configurations
                    ...)

__init__(self, facade='SMAC4HPO', run_obj='quality', deterministic='true', wallclock_limit=60.0, intensifier_kwargs=None, rng=42, **kwargs) special

Initialize the object.

Parameters:

Name Type Description Default
facade

Choice of the SMAC backend strategy, [SMAC4BO, SMAC4HPO, SMAC4AC, BOHB4HPO].

'SMAC4HPO'
run_obj str

Defines the optimization metric. When optimizing runtime, cutoff_time is required as well.

'quality'
wallclock_limit float

Maximum amount of wallclock-time used for optimization.

60.0
deterministic str

If true, SMAC assumes that the target function or algorithm is deterministic (the same static seed of 0 is always passed to the function/algorithm). If false, different random seeds are passed to the target function/algorithm.

'true'
intensifier_kwargs dict

Dict for intensifier settings.

None
rng int

Random seed of SMAC.facade.

42
**kwargs

All initial kwargs are passed to SMACs scenario. List of all a vailable parameters.

{}
Source code in photonai/optimization/smac/smac.py
def __init__(self, facade='SMAC4HPO',
             run_obj: str = "quality",
             deterministic: str = "true",
             wallclock_limit: float = 60.0,
             intensifier_kwargs: dict = None,
             rng: int = 42, **kwargs):
    """
    Initialize the object.

    Parameters:
        facade:
            Choice of the SMAC backend strategy, [SMAC4BO, SMAC4HPO, SMAC4AC, BOHB4HPO].

        run_obj:
            Defines the optimization metric.
            When optimizing runtime, cutoff_time is required as well.

        wallclock_limit:
            Maximum amount of wallclock-time used for optimization.

        deterministic:
            If true, SMAC assumes that the target function or algorithm
            is deterministic (the same static seed of 0
            is always passed to the function/algorithm).
            If false, different random seeds are passed
            to the target function/algorithm.

        intensifier_kwargs:
            Dict for intensifier settings.

        rng:
            Random seed of SMAC.facade.

        **kwargs:
            All initial kwargs are passed to SMACs scenario.
            [List of all a vailable parameters](
            https://automl.github.io/SMAC3/master/options.html#scenario).

    """
    super(SMACOptimizer, self).__init__()

    if not __found__:
        msg = "Module smac not found or not installed as expected. " \
              "Please install the smac/requirements.txt PHOTONAI provides."
        logger.error(msg)
        raise ModuleNotFoundError(msg)

    self.run_obj = run_obj
    self.deterministic = deterministic
    self.wallclock_limit = wallclock_limit
    self.kwargs = kwargs

    if facade in ["SMAC4BO", SMAC4BO, "SMAC4AC", SMAC4AC, "SMAC4HPO", SMAC4HPO, "BOHB4HPO", BOHB4HPO]:
        if type(facade) == str:

            self.facade = eval(facade)
        else:
            self.facade = facade
    else:
        msg = "SMAC.facade {} not known. Please use one of ['SMAC4BO', 'SMAC4AC', 'SMAC4HPO']."
        logger.error(msg.format(str(facade)))
        raise ValueError(msg.format(str(facade)))

    self.rng = rng
    if not intensifier_kwargs:
        self.intensifier_kwargs = {}
    else:
        self.intensifier_kwargs = intensifier_kwargs

    self.cspace = ConfigurationSpace()  # hyperparameter space for SMAC
    self.switch_optiones = {}
    self.hyperparameters = []

    self.maximize_metric = False
    self.constant_dictionary = {}

optimize(self)

Start optimization process.

Source code in photonai/optimization/smac/smac.py
def optimize(self):
    """Start optimization process."""
    self.smac.optimize()

prepare(self, pipeline_elements, maximize_metric, objective_function)

Initializes the SMAC Optimizer.

Parameters:

Name Type Description Default
pipeline_elements list

List of all PipelineElements to create hyperparameter space.

required
maximize_metric bool

Boolean to distinguish between score and error.

required
objective_function Callable

The cost or objective function.

required
Source code in photonai/optimization/smac/smac.py
def prepare(self, pipeline_elements: list, maximize_metric: bool, objective_function: Callable):
    """
    Initializes the SMAC Optimizer.

    Parameters:
        pipeline_elements:
            List of all PipelineElements to create hyperparameter space.

        maximize_metric:
            Boolean to distinguish between score and error.

        objective_function:
            The cost or objective function.

    """
    self.cspace = ConfigurationSpace()  # build space
    self._build_smac_space(pipeline_elements)
    if self.constant_dictionary:
        msg = "PHOTONAI has detected some one-valued params in your hyperparameters. Pleas use the kwargs for " \
              "constant values. This run ignores following settings: " + str(self.constant_dictionary.keys())
        logger.warning(msg)
        warnings.warn(msg)
    self.maximize_metric = maximize_metric

    scenario_dict = self.kwargs
    scenario_dict.update({"run_obj":self.run_obj,
                          "deterministic":self.deterministic,
                          "wallclock_limit":self.wallclock_limit,
                          "cs":self.cspace,
                          "limit_resources": False})

    scenario = Scenario(scenario_dict)

    def smac_objective_function(current_config):
        current_config = {k: current_config[k] for k in current_config if (current_config[k] and 'algos' not in k)}
        return objective_function(current_config)

    self.smac = self.facade(scenario=scenario,
                            intensifier_kwargs=self.intensifier_kwargs,
                            rng=self.rng,
                            tae_runner=smac_objective_function)