In the 4th session, we will focus on practical experiences and practises on hyperparameter optimization.

Long before machine learning ventured outside of academia, hyperparameters had a different name. “Magic Numbers”. These were unexplained constants appearing in code, and their values completely (and destructively) controlled the behavior of programs since the 1960s.
One day, someone (probably a physicist) realized that some of their hardcoded constants could be altered, thereby generating completely new behavior in the software they were working on - without changing anything else. The constants were now parameters as well, and the hyperparameter was invented.
This has made a lot of people very angry and been widely regarded as a bad move.
Fast forward more than half a century into the future, these “list of constants” are no longer tweaked by the shaky hands of weary grad-students, but are methodically swept to find “performance-enhancing” configurations. This is done by very fancy algorithms that are designed to find these “lucky” combinations using the least resources possible.
Such sweeping mechanisms are fairly easy to mismanage and may result in a huge waste of money, time, and energy. Nevertheless, every now and then comes a rare combination of an efficient optimization mechanism for the search - and a robust and easy way to perform it.
In this webinar, we will gawk at the amazing wealth of this realm and use open- source tools to move from a simple grid search - to algorithms verging on Bistromathics, all by keeping a neat interface and controllable execution.

All sessions of the series: