Hyperparameter tuning is the process of finding the model configuration — learning rate, tree depth, regularisation strength, number of layers — that produces the best generalisation performance on held-out data. Every machine learning model has hyperparameters that are not learned from data but must be set before training, and their values have a large impact on performance. Choosing them well separates a mediocre model from a competitive one.
Manual tuning is unreliable and inefficient: the space of reasonable hyperparameter combinations is high-dimensional, the relationships between parameters are non-linear and interactive, and human intuition is a poor guide in this space. Grid search is exhaustive but scales exponentially with the number of parameters. Random search is better but still wastes evaluations on clearly poor configurations. Bayesian optimisation — the approach used by Optuna — builds a probabilistic model of the performance landscape and directs evaluations toward promising regions, finding good configurations with far fewer model fits than grid or random search. Optuna is the leading Python library for Bayesian hyperparameter optimisation, combining algorithmic sophistication with a clean, flexible API.