Neuroevolution

Jevo.jl is primarily designed for neuroevolution using Flux.jl, and is heavily inspired by the following papers:

• Evolution Strategies as a Scalable Alternative to Reinforcement Learning (Salimans et. al, 2017
• Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning (Such et. al, 2017)
• Deep Neuroevolution of Recurrent and Discrete World Models (Risi & Stanley, 2019)
• A coevolutionary approach to deep multi-agent reinforcement learning (Klijn & Eiben, 2021)

Jevo.jl contains the following features designed for state-of-the-art neuroevolution:

• Differences between parents and children can be represented as a collection of random seeds and mutation rates for efficient evolutionary operations and fast transmission between workers. See 1, 2 for more information.
• Support for decoder-only transformers using a fork of Transformers.jl
• Weight primitives designed for evolutionary flexibility:
  • WeightsCollection allows different subarrays of weights to evolve separately within a layer
  • FactorWeight produces a weight matrix which is a product of two smaller factor matricies.
  • CompositeWeight adds multiple weights with the same dimensions to be added together

Genotypes

In Jevo.jl, a neural network's genotype is represented as a subtype of AbstractLayer which contains various weights/biases represented by subtypes of AbstractWeights, like Weights, WeightsCollection, FactorWeights, and CompositeWeights. Layers can contain other layers, and work much like you'd expect from a traditional deep learning librarly like PyTorch.

Phenotypes

A Jevo genotype needs to be developed into a phenotype, i.e. the explicit Float32 arrays that make up the neural network and the various Flux.jl structures associated with them. This is done on the worker, once the genotype has been transmitted from the main process. Once a phenotype has been created on the worker, the phenotype is evaluated and the worker sends its score back to the main process.