Stochastic optimization

Datasets from experiments spanning many hours, or models that couple together many neurons, can produce more data than fits in memory. In that case we can split the data into chunks called batches and loop over them, updating the model parameters one batch at a time. Fitting this way – on batches, or in the extreme case on individual data points – is called stochastic optimization.

NeMoS provides some functionality to make fitting GLMs this way easier:

• GLM.stochastic_fit as a simple interface.

• Built-in data loaders for common data sources.

• Monitoring and convergence criteria using callbacks.

• Straightforward customization through custom data loaders and callbacks.

• Stochastic optimization basics
  • Load data from NWB
  • Build the data loader
  • Callback for logging
  • Model configuration
  • Running the optimization
  • Continuing fitting

• Creating a custom DataLoader
  • The DataLoader interface
  • A Simple Example: PynappleDataLoader
  • Handling Disjoint Recording Intervals

• Stopping on convergence and custom callbacks
  • Create a custom callback for early stopping
  • Run the optimization until convergence
  • Evaluate result

• Manual update loop for more control
  • Set up the data and model
  • Manual update loop calling GLM.update