Chris Fourie#

Medical doctor turned health data scientist and machine learning engineer, trying to use the powers of AI for social good.

Curriculum vitae#

Research#

Fundamental research interests:

Applied research interests:

Communiities and academic affliations#

Industry#

Independent contractor for LifeQ

Independent digital health consultant

Selected master's work#

Research report#

[Full report]

Impact of Noise on Learned Value Functions at Depth in CoAgent Networks for Neural Network Credit Assignment

CoAgent networks (CoANs), networks of reinforcement learning agents, have been shown to be a biologically plausible alternative to backpropagation for solving the neural network structural credit assignment problem [Gupta et al. 2021]. This is accomplished where many agents, each as a neuron in a stochastic neural network, use only their local policy gradient and a global reward. Noise is an important consideration in the learning dynamics of any stochastic neural network [Schoenholz et al. 2017]. We investigate the impact of noise on learnt value functions for baselines and Actor-Critic methods in CoANs at depth. We demonstrate that with additional layers, CoANs using REINFORCE, REINFORCE with a baseline or Actor-Critic methods perform significantly worse. However unbiased variance reduction methods are effective at alleviating this to a moderate extent. For CoANs of increasing depth and width using Actor-Critic methods we show that learned value functions are more sensitive to noise. We show as well that large bootstrapping bias impacts Actor-Critic CoAN methods significantly.

Projects#

Machine Learning#

Reinforcement Learning#

Computer Vision#

High Performance Computing#

Algorithm Analysis#

Robotics#