Reading Time: 4 minutes

We’ve seen language models before: enter a question and an A.I. provides the “answer”. Nowadays, A.I. has gotten so good that humans cannot differentiate between an A.I.-derived answer and a human-given answer.

Language models have been called the greatest A.I. breakthrough in the past five years. And there’s no denying their powers. But what exactly are these powers and how can we use them day-to-day?

This week, I’ll mention two of the latest language models, explain what they are, and show a few things that they’re capable of. Perhaps there’s something here that can benefit your business!

Language Models

Language models are statistical models that incorporate an understanding of how words “fit together”, based largely on the probabilities of how words fit together in giant (really!) collections of text. The models are trained on vast collections of text, hopefully with lots of diversity, such as Wikipedia.

The two most powerful and largest language models right now are Google’s model and OpenAI’s GPT-3.

Google’s “Switch Transformer” model has a trillion parameters (it definitely does not fit on your laptop), putting it nearly six times the size of the earlier GPT-3 (link). This is important because models with more parameters are generally more performant, as these large models incorporate many more textual intricacies that we humans take for granted when reading/hearing and writing/speaking.

Let’s have a look at a few things these models can do!

Text Generation and Curing Writer’s Block:

Start a sentence and the model writes the rest for you. Need help with a research paper? Language models can now write convincing sentences about lots of topics, and humans generally cannot tell a machine was involved.

Question and Answers:

Ask any question and get a reasonable answer. Google Assistant, Siri, and Alexa do this millions of times per day right now.

Generate and Explain Computer Code:

Having been trained on lots of well-written computer code, language models allow us to:

  1. Use speaking language to have the A.I. generate text that does what the “programmer” asks for.
  2. Select computer code and have the A.I. report exactly what the code does in plain language.

Generate Website Mock-Ups:

Need a landing page for your new app? Copywriting can be very challenging. Language models make it a snap. For example, a GPT-3 × Figma plugin generates the following design when given the text: “A website like Stripe that is about a chat app”, (here’s the video):

Bennett Data Science Tech Tuesdays Language Machine Learning Website

Niche Recommendations:

Imagine finding a book in a niche that very few people know about. I’m reminded of a book I used in industry called “The Acoustic Bubble” by Timothy Leighton. How would I ever find a companion to that esoteric book about the acoustic interactions of bubbles? Or let’s say I have a collection of such books and I’m looking for more. In these cases, simply using a language model can sift through trillions of written words to find similar books, relevant to either case.

These models can do a lot more, the list is non-exhaustive and continues to expand with novel innovations. If you’re interested in more examples, have a look at this article.

And if you’d like to start using one of these magical language models, GPT-3 now has an API.

All of this text was written by a human 😉

Be healthy and have a great week!


Of Interest

How Google Knows you Better Than you Do
We’ve all been there: surfing the web and bumping into ads with an uncanny flavor. How did they know I was thinking about joining a gym? Or changing careers? You might wonder if Google can read your mind. This is not the case, of course, but it can read your search history and tracks a lot of your web browsing, too. The company runs a vast profiling machine, fitting people into categories that say who they are, what they’re worth, and how they’re expected to act. Google isn’t just organizing the world’s information; it’s sorting the world’s populations. Read more here.

Why the Best Business Writers Avoid Certain Words
Stephen King hates adverbs. And if the goal of writing is to get results, you should too. “Adverbs, like the passive voice” King claims in his classic book On Writing, “seem to have been created with the timid writer in mind. … with adverbs, the writer usually tells us he or she is afraid he/she isn’t expressing himself/herself clearly, that he or she is not getting the point or the picture across.” Now, data science also reveals why the best business writers avoid certain words. Read more here.

Kaggle’s State of Machine Learning and Data Science Report
Kaggle has published a report on the State of Machine Learning and Data Science for 2020. The report is based on survey responses from over two thousand users currently employed as data scientists. The report notes that the “vast majority” of data scientists are under 35 years of age, two-thirds have a graduate degree, and most have less than 10 years coding experience. Around 55% have less than three years of experience with machine learning. Read more here.