Andrej Karpathy (@karpathy) 's Twitter Profile
Andrej Karpathy

@karpathy

Building @EurekaLabsAI. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥

ID: 33836629

linkhttps://karpathy.ai calendar_today21-04-2009 06:49:15

9,9K Tweet

1,0M Followers

931 Following

Andrej Karpathy (@karpathy) 's Twitter Profile Photo

Actually this was really good - a tour from one transistor to a small CPU (Scott CPU, to be precise). The YouTube playlist: youtube.com/watch?v=HaBMAD… I also haven't yet come across the "But How Do It Know" by Scott, which this is based on, and which looks great:

Actually this was really good - a tour from one transistor to a small CPU (Scott CPU, to be precise).

The YouTube playlist:
youtube.com/watch?v=HaBMAD…

I also haven't yet come across the "But How Do It Know" by Scott, which this is based on, and which looks great:
Andrej Karpathy (@karpathy) 's Twitter Profile Photo

Very exciting! Congrats Robin and the Black Forest Labs team (of Stable Diffusion fame) on the launch! The open sourced FLUX.1 image gen model looks very strong, main page with examples: blackforestlabs.ai Clean/readable (inference) code on GitHub: github.com/black-forest-l…

Andrej Karpathy (@karpathy) 's Twitter Profile Photo

August 1, 2024: The Music Video Fun hack just stitching up gen AI tools :), in this case to create a music video for today. - copy paste the entire WSJ front page into Claude - ask it to generate multiple scenes and give visual descriptions for them - copy paste scene

Andrej Karpathy (@karpathy) 's Twitter Profile Photo

So cool! farm.bot (FarmBot) FarmBot is a bit like solar panels for food. I love the idea that automation could help us reclaim control over our food production and move it from farms back into our own backyards. (Also - food Factorio!) youtube.com/watch?v=qwSbWy…

So cool! farm.bot (<a href="/farmbotio/">FarmBot</a>)
FarmBot is a bit like solar panels for food. I love the idea that automation could help us reclaim control over our food production and move it from farms back into our own backyards. (Also - food Factorio!)

youtube.com/watch?v=qwSbWy…
Andrej Karpathy (@karpathy) 's Twitter Profile Photo

# RLHF is just barely RL Reinforcement Learning from Human Feedback (RLHF) is the third (and last) major stage of training an LLM, after pretraining and supervised finetuning (SFT). My rant on RLHF is that it is just barely RL, in a way that I think is not too widely

# RLHF is just barely RL

Reinforcement Learning from Human Feedback (RLHF) is the third (and last) major stage of training an LLM, after pretraining and supervised finetuning (SFT). My rant on RLHF is that it is just barely RL, in a way that I think is not too widely
Andrej Karpathy (@karpathy) 's Twitter Profile Photo

SQL injection-like attack on LLMs with special tokens The decision by LLM tokenizers to parse special tokens in the input string (<s>, <|endoftext|>, etc.), while convenient looking, leads to footguns at best and LLM security vulnerabilities at worst, equivalent to SQL injection

SQL injection-like attack on LLMs with special tokens

The decision by LLM tokenizers to parse special tokens in the input string (&lt;s&gt;, &lt;|endoftext|&gt;, etc.), while convenient looking, leads to footguns at best and LLM security vulnerabilities at worst, equivalent to SQL injection
Andrej Karpathy (@karpathy) 's Twitter Profile Photo

Actually I was reading the book "A Poison Like No Other: How Microplastics Corrupted Our Planet and Our Bodies" just last week. I didn't realize the extent to which plastics have come to permeate and mess with our entire environment. It's not just about the polymer granules of

Andrej Karpathy (@karpathy) 's Twitter Profile Photo

Haha we've all been there. I stumbled by this tweet earlier today and tried to write a little utility that auto-generates git commit message based on the git diff of staged changes. Gist: gist.github.com/karpathy/1dd02… So just typing `gcm` (short for git commit -m) auto-generates a

Andrej Karpathy (@karpathy) 's Twitter Profile Photo

This was a cool listen. I think Cloud+AI is increasingly making the @levelsio -style model of a scrappy solo serial micro-entrepreneur viable, allowing one person to spin up and run a number of companies that generate income, possibly well into billion-dollar valuations.

Andrej Karpathy (@karpathy) 's Twitter Profile Photo

I feel like a large amount of GDP is locked up because it is difficult for person A to very conveniently pay 5 cents to person B. Current high fixed costs per transaction force each of them to be of high enough amounts, which results in business models with purchase bundles,

Andrej Karpathy (@karpathy) 's Twitter Profile Photo

Very cool, place well under “feel the AGI” category. As mentioned in the post, making actual apps is a lot more than code, you have to set up the entire environment, deploy it, etc. Automating all of this other infra will allow anyone to quickly build and deploy entire web apps.