03·nn.Module, nn.Parameter, layers and their registration

Introduction

nn.Module is the base class of all PyTorch models and layers. It defines a parameter and sub-module registration protocol via __setattr__ magic: when you assign an attribute of type nn.Parameter or nn.Module it is automatically indexed in the internal _parameters and _modules dicts. This lesson covers the module lifecycle, the difference between an attribute and a registered parameter, the role of nn.ModuleList / nn.Sequential, the state_dict and hook mechanism, and the train/eval mode and its effect on BatchNorm/Dropout.