Learning Rate Scheduler¶

Learning Rate Scheduler¶

class kospeech.optim.lr_scheduler.lr_scheduler.LearningRateScheduler(optimizer, init_lr)[source]¶: Provides inteface of learning rate scheduler.

Note

Do not use this class directly, use one of the sub classes.

Tri Stage LR Scheduler¶

class kospeech.optim.lr_scheduler.tri_stage_lr_scheduler.TriStageLRScheduler(optimizer, init_lr, peak_lr, final_lr, init_lr_scale, final_lr_scale, warmup_steps, total_steps)[source]¶: Tri-Stage Learning Rate Scheduler Implement the learning rate scheduler in “SpecAugment”

Transformer LR Scheduler¶

class kospeech.optim.lr_scheduler.transformer_lr_scheduler.TransformerLRScheduler(optimizer, peak_lr, final_lr, final_lr_scale, warmup_steps, decay_steps)[source]¶: Transformer Learning Rate Scheduler proposed in “Attention Is All You Need”