Speaker:Jacob Pratt

Hopf Bifurcation as an Activation Function for Complex Recurrent Neural Networks: A Feasibility Study

Complex recurrent network performance is dependent on the choice of activation function. However, the difficulty in differentiating complex-valued functions, limit their functional use in complex recurrent neural networks. Exploring different approaches to complex activation functions has the potential to produce better performing complex recurrent networks. This work presents a new type of activation function that utilizes the normal form of the Hopf bifurcation as a complex-valued activation function. The underlying theoretical properties to make the Hopf bifurcation a feasible activation function is explored. The proposed activation function is evaluated over the Mackey Glass and Copy Memory datasets commonly used in analyzing recurrent networks. Two experimental setups are used to evaluate the optimal configurations for the activation function and compare with five established complex activations. All experiments are compared to a Base model which uses no activation function. Results show the Hopf bifurcation has potential to be used as a complex activation function, performing equivalently or better than the compared common complex activations, but is ultimately limited by training time.

Committee: Chris Quinn (major professor), Wallapak Tavanapong and Alexander Stoytchev