It used to be good enough just to copy others. Now, with AI in the hands of billions, there’s little value in copying.

For instance, take programming. Five years ago, building apps, websites, or games required a non-trivial amount of skill, and getting your first project off the ground was an accomplishment. Now, AI can generate most starter projects in hours, if not minutes. I think this decimates the reward, both internal and external, of actually completing the first few projects.

I’ve been an author on a few papers before, but I recently published the first research project where I was responsible for most of the work and direction. It’s in the first 2024 issue of the journal Artificial Life, which you can find here. You can find a non-paywalled version here Below, I tell the chronology of the project and summarize our findings.

We explore the conditions under which trade can emerge between four deep reinforcement learning agents that pick up and put down resources in a 2D foraging environment. Agents are rewarded for having both resources once, but the resources are distributed far apart from each other. To maximize reward, agents need to split up the work - agent 1 goes to resource A, agent 2 goes to resource B, etc, and then they meet to exchange resources, since meeting halfway can get them the most of each resource in the shortest amount of time.

An ever-expanding list of concepts in the field of AI to give myself and others an easy reference. Each item in the list contains a short, rudimentary definition I’ve written, as well as a link to a resource that can explain it better.

Ablation Study: Removing some parts of a machine learning model to measure impact on performance

Advantage Function: The difference between a Q-value for a state-action pair and a value for the state. Useful to determine how good an action is relative to its state.

Here’s the problem with advertising AI-based technology that doesn’t exist:

You cannot promise anything about your product.

We’ve all seen AI advertised to the masses that doesn’t work as advertised, just look at any voice-to-text system. When I got my Apple Watch, I hoped to use it to respond to messages without getting distracted by my phone. I quickly realized that wasn’t a viable solution: I had to repeat my message multiple times per text in order to get the correct dictation.