What a Time for Language Models
New capabilities, new products, new possible futures
Hi, I’m Jay from the Illustrated series of blog posts about language models. You’re getting this because you’ve subscribed to my mailing list. I have recently moved it to Substack and we’re due for an update.
Language Models are All The Rage Now
It’s truly fascinating how quickly language models have been developing. Their commercial potential is now bleeding into the mainstream tech industry, and lots of people are starting to pay attention. Hundreds of millions of people are now asking themselves “Now what? What does this mean for my job, my company, my economy?”.
If you’re on this mailing list, you’ve likely been pondering these questions before everybody else. But with each surprising new capability that these models develop, those questions arise again and again.
My latest articles highlight a few of my observations on these developments:
What's the big deal with Generative AI? Is it the future or the present?
1- Recent AI developments are awe-inspiring and promise to change the world. But when?
2- Make a distinction between impressive 🍒 cherry-picked demos, and reliable use cases that are ready for the marketplace
3- Think of models as components of intelligent systems, not minds
4- Generative AI alone is only the tip of the iceberg
AI is Eating The World (Part 2)
5) Maps and Landscapes of AI Technology and Value Stacks
Don’t forget about the business moats:
6) Enterprises: Plan Not for One, but Thousands of AI Touchpoints in Your Systems
7) Account for the Many Descendants and Iterations of a Foundation Model
The data development loop is one of the most valuable areas in this new regime:
8) Model Usage Datasets Allow Collective Exploration of a Model’s Generative Space
Pockets of Moats for Application Layer Players:
The Illustrated Stable Diffusion
Image generation models also went through surprising and rapid developments in 2022. I made this article and video to be the most accessible intro to how diffusion models work and how they’re trained. If you’ve been curious about what goes on under the hood, this shall be a great place to start building the intuitions about their major components.
Talking Language AI YouTube Series
In the last few months, I got to learn from some of the leading creators of NLP tools in a YouTube series called Talking Language AI. Five episodes are now out. They cover tools like BERTopic, Sentence Transformer, TalkToModel, Human Learn, and many more. In episode 4 I also share my experience at attending the EMNLP conference and some of my major takeaways.
More 2022 Videos
I’m also glad to have been able to make a few more videos in 2022 on my YouTube channel. They cover some of the main topics I’ve found fascinating. From Retrieval-augmented Transformers to multimodal models, and to the future data sources of language models after web text.
What are you thinking of?
I’m curious about what you are thinking of with all these recent developments. What questions are you tackling at the moment? Let me know on Twitter.