If you want to do world-class research, temperament matters more than talent.

One PhD later, I'm giving my secrets away for free.

There used to be many possible candidates for how humans might build intelligent systems. Now there's only one.

On the Platonic Representation Hypothesis, decoding whale speech, and what language model embeddings have to say about universality.

A roadmap for RL-ing LLMs on the entire Internet.

There are only two ways to learn — supervised learning and reinforcement learning. Which one will give us superintelligence?

Why AI researchers should dream bigger and publish less.

LLMs were invented in four major developments — all of which were datasets.

Our employers have all the data they need to train AI models to replace us. How long will it be until this actually happens?

It’s become very popular over the last few years to speculate how close society might be to Artificial General Intelligence (AGI). What AGI actually means is murky, and often-debated, but mentioning AGI is usually a good jumping-off point for discussions...

Approachable post about my research into inverting representations from text embedders and language models.

A high-level overview of the field of Text Retrieval (as it stood in 2022)

A taxonomy for thinking about research papers that say they use contrastive learning.

How a collection of Twitter people, Discord chats, and Colab notebooks is changing the art world.

Explanation of the Noise Contrastive Estimation algorithm and clarification of terminology. Also covers InfoNCE, a popular technique in contrastive learning.

A primer on variational autoencoders (VAEs) culminating in a PyTorch implementation of a VAE with discrete latents.

Opinionated introduction to adversarial examples within the context of natural language.

Comparison of two transformer-based NLP models, one big, one small.

Demonstration of the integration between two open-source packages, HuggingFace Transformers and Weights & Biases, and how to use them together to train NLP models.

Are split-brain experiments strong evidence that humans have multiple consciousnesses? Essay about Nagel, Parfit, and theories of consciousness.

Can artificial neural networks create semantics from syntax? Essay about connectionism, the Chinese Room, and artificial neural networks.

A fun number theory problem: Given all of the integers from 1 to N, can you arrange all elements such that each adjacent pair sums to a square number?

A juvenile rant about set theory terminology.

A visualization of sorting algorithms, sorting randomized colors by hue. It's since been adapted into several top posts on Reddit.

An interactive illustration of distortions in the Washington, D.C. metro map.

Quantitative analysis showing how to get the most likes on a popular social network.

Using CSS and single-point perspective to render the iconic Star Wars opening text crawl in the browser. (Warning: after some changes to CSS syntax in the past few years, it doesn't really work anymore.)

Discussion of NP-hard problems, a cool color-based stippling method, and generating art by solving the Traveling Salesman problem.

Animation showing the world's largest cities through the ages.

Implementation of K-D trees in Javascript and an animation showing the effect of log(n) nearest-neighbor search on performance.

A new metric for quantatitatively estimating the perfect salary for each NBA player. Who's overpaid, and who's due for a bigger contract?