Episode 101

The AI Battle Royale

Watch on YouTube

About this episode

In this episode of the Untangling Web3 podcast, we break down the latest developments in the rapidly evolving world of AI, focusing on the rising competition between major players like OpenAI, Google's Gemini, and China's groundbreaking DeepSeek.

With performance, cost, and openness on the line, this episode unpacks the global battle to dominate the future of artificial intelligence.

Key highlights

  • Origins and Missions of the Major Players. The episode begins by tracing the roots of leading AI companies. From OpenAI's nonprofit beginnings to its dominance with GPT-4.1, and Google's Gemini suite developed from DeepMind and Brain, we compare their approaches to safety, openness, and commercialization. The team also discusses Anthropic’s Claude and the unique positioning of models like Mistral and Cohere’s Command R, each targeting different market segments and use cases—from transparency to enterprise knowledge management.
  • The Rise of DeepSeek and the Global Shake-Up. The hosts unpack the massive impact of DeepSeek, China’s open-source AI contender that rocked global tech markets in early 2025. They explain why DeepSeek's performance-to-cost ratio, open accessibility, and groundbreaking efficiency innovations like batch tokenization sent shockwaves through Western tech stocks. With its free and highly capable models, DeepSeek challenges the dominance of U.S.-based platforms and marks a shift in global AI power dynamics.
  • Performance, Pricing, and Future Trends. Diving into technical metrics, the hosts compare context windows, output limits, and per-token costs of today’s top models. They highlight how GPT-4.1, Claude, Gemini, and DeepSeek serve different user needs—from coding and storytelling to academic research and cost-efficient startups. The episode also explores how integration strategies like Model Context Protocol (MCP) and services like AWS Bedrock are reshaping how AI is deployed and scaled in real-world applications.

As the AI battle heats up between OpenAI, Gemini, DeepSeek, and others, users are the real winners—gaining access to faster, cheaper, and more specialized tools. But with geopolitical dynamics, training data transparency, and business models evolving fast, staying informed is key.

Whether you're an entrepreneur, developer, or tech enthusiast, this episode is your guide to navigating the increasingly competitive and globalized landscape of artificial intelligence.

This episode is sponsored by the VeChain foundation. Learn more about VeBetterDAO here: https://vebetterdao.org/

Transcript

Auto-generated from the episode audio.

Welcome to another episode of the Untangling Web 3 podcast. So, take your minds back. Two years ago, Chat GPT kicked open the AI floodgates and we've been swimming in large language models ever since. Powerful starting statement. Now, the space is packed with powerful players each race into redefine what AI can do. So, in this episode, we're going to trace the evolution of the models that are currently available from humble beginnings to today's turbocharged tools and explain who's shaping the future of AI or artificial intelligence. Are you ready for this one, Jack?

Uh, I'm going to say no, just because it seems like within a matter of hours or days, you can be out of date. As we were just discussing, Alec, uh, you know, since we planned this episode a few days ago, you take a few days off and then something's changed again. So yeah, I never feel completely 100% ready to talk about AI, but I always want to. So yeah, let's let's go for it. Well, one more thing we can say, we can speak to events that have already happened.

We can speak to the history of it. Maybe we're not going to have the most upto-date version, but we can speak about things that have already happened. OpenAI chat GPT the kind of you know large language models and chat GPT the word is synonymous now whereas I actually hear people using the vert GPT it which is just shows that it was chat GPT open AI that kind of initiated this incredibly fastm moving space that we now find ourselves in and to set our minds back so open AI was founded in 2015 by Sam Alman as most people know but also Elon Musk which is quite a funny tur of events considering everything that's been happening and it was founded as a nonprofit research lab aiming to ensure that artificial general intelligence AI which is the singularity benefits all of humanity and obviously we now find ourselves in the circumstance where Sam Alman and Elon Musk are no longer friends and open AI has actually transitioned to a capp profit model to scale safely apparently.

Yeah, it it always surprises me how long Open AI has been around because I first came across uh OpenAI and chat GPT. I think it was sometime kind of mid the first COVID lockdown in 2020 and I saw uh GPT3 trending and it was completely new to me and it was just like pictures of the paper that they published. Um and yeah, to know that they've been around for since 2015 and only had this kind of major success in the last couple of years always baffles me.

um they seem to have been in the background. A lot of people don't realize, especially given what's been happening recently with the potential buying offers and things, who was involved originally and there's been lots of turbulence as well that's documented with the uh the founders and the key the key players, board members, chief executives, etc. at OpenAI. But yeah, there's still I think am I safe to say right now they're still the dominant player at least in terms of the the kind of household name recognition and the number of users but we have seen in the last year or two some really big players coming on board you know likes of Google uh Microsoft as we'll get into who are really challenging them as well as challenging the way that open AI does AI as well right so not just being completely closed source but as we'll see some some kind of open alternatives too so there's a what's happened in the space recently.

Yeah, you are right. Like it's a bit mad to think that they were around seven years before it kind of everything kicked off and they released chat GPT 3.5 in November 2022 and that's the one that everyone knows is kind of the the single prompt and it can do many many things and give you incredibly complex and incredibly interesting insightful answers. So yeah, it was 2022 when that model released and people really started to understand the value of LLMs for everyday life which was an absolute gamecher like it really did change the entire space.

Everyone's attention and all the headlines were then on OpenAI and LLMs and it hasn't really changed since it's one of the main things that we've been talking about this podcast for the past two years. But like you mentioned there, there are competitors and there are other people that have slowly been growing in the space. I think it's worth just taking a moment to go back to some of the the main big names and how they started and what their mission was. And one thing I'm going to say is all of these companies, they all say the same thing.

They all started with the core mission of developing helpful, honest, and you know, harmless technology. I think that's just what they always say because no one wants to read and say, "Yeah, we're going to be the real bad guy. We want AI that's going to, you know, take people's jobs and do really nasty things. So, they all start like that. But let's go into some of the the first ones. I think one that most people will be familiar with is Claude. So, this was Anthropic's um kind of baby.

This was co-founded in 2021 by someone called Dario and Daniela. Uh former OpenAI executives actually. And Anthropic was created as a public benefit corporation with a a mission rooted in AI safety. They basically wanted to be kind of OpenAI's more ethical challenger. Whether that's going to be the the endgame, who's to say, but that's why they left OpenAI and that's why they started Claude and Anthropic. Yeah. And that's interesting because as I said, like there's been turbulence at OpenAI. A lot of that has been to do with them as a kind of transitioning between not for profofit and forprofit and how that affects what they do and and how much is disclosed about what they do and how the models are trained and all that.

Um, so yeah, it makes sense that you you would have competitors like Anthropic springing up to kind of fill that gap. Um, what that they maybe were leaving behind. Um, and yeah, I I I haven't actually used Claude myself. Um, but it's one that I think a lot of it's what a lot of the people's favorite if you ask them kind of like, you know, whenever I go online, you see people arguing about which one is better to use and everyone kind of has their baby.

I think a lot of people are using multiple now, but it's not one I've had experience with. But again, it's been on my radar for pretty much as long as OpenAI has as well now. Yeah. So, I've used almost all of the models, just played around with most of them. Um, Claude, we'll go into like the current like kind of the current best model and how they all stack up and compare against each other. But I think I although I do say that like all of these kind of core missions of all the companies I'm going to mention is to be, you know, good, helpful, positive for humanity, not bad, all this stuff.

They always say that. You can kind of actually tell that from interacting with Claude and how it gives answers compared with say open AAI like even the way in which you can set it up like they kind of had privacy preservation quite early on. They kind of had closed models that wouldn't learn anything from your interactions before open AI did. It was much more say user controlled and user defined whereas open AI was like really great to use the most in my mind like the the superior kind of answer but they were taking and harvesting all that information.

So if you just care about the utility and the output and don't care about what information is kind of happening behind the scenes or being exchanged behind the scenes then maybe you don't care about you know using open AAI but claude is definitely the more user centric model in my mind. Yeah and I think kind of tracks with um my experience of chat GPT being a bit more uh ask for forgiveness not permission. You know they added in a lot of these things after the fact whereas yeah I think anthropic took a much more uh progressive approach to a lot of those things.

Um, similarly, one that people will probably have heard of is Gemini from Google. Um, and I've actually, again, I've known about Google's AI capabilities for way longer than OpenAI. Like, I've heard Google Deep Mind for a very, very long time. They've always had a big uh AI research arm and it kind of in I think 2023, they merged their two main research arms. So, Deep Mind and Google Brain, and that kind of came became the same uh research function. And then we finally saw in December 23 they launched Gemini which is their competitor.

Um and this one had you know probably I I would say kind of less a successful launch because it had some some negative press associated with it because they were also taking I guess if you look at Google people see them as the the good face of the internet in a lot of ways. you know people that that's something that people have a lot of brand trust associated with Google right they trust Google to go online you know my my grandparents would use Google to to find that information so it's kind of it's got that appeal um and they clearly baked in lots of checks and balances maybe political correctness into their model and there was there's some quite funny uh discussion online about how it would hallucinate a little bit and give you uh incorrect answers if you were asking for historical things for example So I'm pretty sure didn't they pull it for a short while as well and they started working on it right that they were they had a bit of

a botch launch but I think since then has become much much more performant much more reliable than it was you know just a year and a half ago say yeah yeah I remember seeing some of the a lot of people kicking off about say historical depictions of race and all this kind of stuff and it was like a big thing on the internet like they were trying to be politically correct and some people said maybe they were too politically correct to get but verging on kind of incorrect Corct.

Um, but yeah, who's to say like where the line should stand on that stuff? And we're a tech podcast so we're not going to touch that right now. The one thing like say you are right though, Google has a big foot in the door and obviously a big vested interest in this kind of whole space, right? They're the search engine everyone uses and like we're kind of maybe people that are using LLMs today understand that more and more we're actually using say an LLM as a search engine.

So they have a vested interest in kind of doing this right and make sureing they aren't replaced by say an open AI. I actually had a few friends that worked at Google brain and Google deep mind just from robotics background. So they were doing a lot of the kind of text um interpretation video interpretation for some kind of useful output. So like imagine robots going around a space understanding what the surroundings are being able to interpret that and interact with that. So they were doing this kind of stuff like four or five years ago and that was one of the big things around Gemini when they launched.

They launched a bit later than say everyone else. It was like 2023 but they were kind of known it was kind of known for being a bit more how to describe this like deep understanding. So they had this I think they one of the first companies had this multimodal understanding where say you know you could put it into the LLM and it would interpret text, images, audio, video code and kind of do all of those right out the gates. where say some of the earlier models from OpenAI and Claude were very specific.

We're either very text focused, very image focused and couldn't really interpret across the board whereas Gemini could do that probably because it started a bit later but also because they had this quite rich background of being very heavy on the audio text video stuff. Um yeah, it was it's pretty impressive even to this day. I use Gemini quite regularly. Yeah, and you're right like it makes sense that they'd be taking a very multimodal approach like Google, you know, owning YouTube. They've owned effectively image search for a long long time.

They've mapped the world. You know, Google has done a lot of impressive things with different types of media. Um, and also, yeah, the point you mentioned about the search engines is fascinating because there's a huge debate going on right now about the best way to get upto-date and live information into LLMs, into the responses to improve, you know, the the the the level of actual factual context you can give to an LLM. You've got the whole retrieval augmented generation stuff going on now. Model context protocol which we'll talk to.

But another big thing is just which search engine you allow your LLM to use and other other uh competitors to to Google aren't using Google, they're using things like Bing instead. So yeah, they've got a they've got a job on their hands to make sure they don't get replaced as a search engine by their competitors, which is just super fascinating. Exactly. I think another interesting thing about um Gemini is obviously it's attached to Google like we said whereas when you think about open AAI and anthropic you kind of think of them as pure AI companies whereas Google has these arms so like all of these kind of AI models that they're working on Gemini pro ultra nano they fit into the Google package of like Google search obviously Google workspace Android phones like all this stuff there is like an immediate use case an immediate like access to retail users where obviously open AI had to do themselves.

Anthropic and Claude had to do that themselves. They had to go to retail users and say this is a new interface you've never seen before. Trust us, use us. So they they're kind of their products had to be a lot better in a way to get the traction. Whereas Google we're using it anyway. So if you could just fit things underneath the surface like I have an Android device. Gemini is now in there and it's like helping me and the normal interactions I'd be doing anyway.

Gemini is working beneath the surface to kind of make those more efficient, more productive, whatever. So that's a big big advantage that Google has is that they're already hitting billions of users. Yeah. And uh similar to what we said in our episode 100 about meta and threads, right? Like threads being able to use this existing user base to get traction on threads, even though I don't think it's that popular in in day-to-day use, but they could get acquire users very easily. Same is true of Google.

Now, if you go on Gmail, like most people do every day, you'll see Gemini popping up asking you to to, you know, if they can help you and stuff. And so you you kind of have that platform locket that helps them get traction as well. So yeah, it's going to be it's going to be interesting to see if if they can improve performance and and uh quality of that model over time. Maybe that could just from the existing user base end up taking over uh the kind of market share.

But yeah, it's definitely one to watch. And then the the two ones that I want to mention are definitely not household names by any means, but I think they're quite important to mention purely because they do have good user bases. people are using these but also they kind of offer something different like the three models that we just spoke about they are US focused companies everyone knows that the US is very dominant or well they were very dominant in all things LLM and the next one I want to speak about is Mistl so this is actually a startup from Paris in France um and this is gaining recognition because it's a lot more transparent and well they're saying more performant than the other models the transparency is probably the more interesting thing so they kind of show the weights they have for answers and how like say answers were derived and how the models were trained and all this kind of stuff and that kind of makes sense right Europe is much more say kind

of heavy-handed on the regulation so it's much more there's a much more kind of requirement for companies to be transparent whereas say that the generalization is you know US companies are a lot more um liberal let's say and we'll just go with the thing and try and like you say not ask permission and just go for it yeah it's a good point like any how is anyone going to compete in Europe generally from building a better AI AI model and training it better if you have to comply with GDPR because it's it's strictly more difficult, right, to to build something than just this kind of free open uh system of kind of a free-for-all that you have with the the chat GPTs and open AIS.

But on the flip side, I think that maybe also has informed their decision to be more transparent, right? They're like, well, if we're hamstrung by the the laws anyway, then we may as well contribute to this openw weight kind of model that you're talking about this transparency. So yeah, it's really interesting. Again, not one I've used um because it's again pretty niche. Um but I it's again one of I'm very aware of and I think it's one we should definitely keep an eye on because that open model is you know part of what prompted this as we'll talk about soon is deepseek and that is also playing in that space but also for like say um training and say when you want explanability so like a big thing with open AI is it's like okay I ask a question it gives me a crazy answer I don't know how it came to that crazy answer and actually it's really hard for open AI to know how it came to that answer as well the way

in which it's built but say something like Mistl that we're talking about um because they have this need to be more transparent to prove waitings to prove prove where data came from that actually helps a lot with the explanability. if it did come to some crazy answer and you're an enterprise that's deployed this, it's much less of a black box and there are ways to say fine-tune answers out or work out say how an answer was derived and that has a lot of appeal as well.

Like the answer is probably not going to be as accurate across the board as say something like open AI but the fact that you can open the box and work out how answers are actually came about is very appealing say big institutions big enterprises that want to trust these models and aren't just going to say black box here we go let's see and hope it works. Yeah. Yeah, and it uses this a lot of this I mean there's lots of there's been lots of not just developments recently in the models or the competitors but the techniques being used as well.

So um Mistl is using the mixture of experts technique which is really interesting for how it compartmentalizes the neural networks it uses makes it more efficient by not using the whole network for specific tasks or specific questions and it because it can narrow things down earlier. That's kind of a big theme of how LLMs have changed in the last few years. The other one we've seen is the development of reasoning models. So just not not just traditional transformers, but reasoning models that have this this chain of thought that you can follow through.

And I think that's again contributing to this how you explain uh AI models and what what what outputs they come to, right? That's something you you definitely need. And you know, we talked about a lot in the early days of the podcast about uh the use case of things like well when things go wrong like self-driving cars, you need to be able to see what the the reasoning was. It's not a direct analogy to uh how LLM work, but it's the same kind of thing, right?

You need to have some level of accountability baked in there. Um if these are going to be used for like mission critical things in the future. Yeah. And on that, the final one that we'll mention is Commander. This is Kahir's enterprise LLM. very targeted towards enterprise kind of similar to the last one in in what they're doing here but they use a different kind of training approach which is called retrieval augmented generation rag and one of the things here is about accuracy like you mentioned earlier on the hallucination that's effectively like these models can sometimes get it wrong and because they're kind of just like probabilistic machines working out what the next word should be it can sometimes get a bit weird you can ask a question it can give a wrong answer because it's pulling from somewhere else and anyway that's quite hard to prove and quite to fact check and they often get it wrong, right?

They're not say factual machines whereas like command R it's doing it in a slightly different way. So it's it's much more related to real documents. So they kind of have this real-time document um database retrieval system where it's like it's good for enterprises. So, one of the examples they like to push is that I have a I want to create an entity that represents my knowledge base, my entire company's history, all the files, everything in there, every single document, every single person, whatever. And then when someone asks a question to be like, what information do we have on Jack and I get all the information summarized, whatever, I know that that's related to one my specific database.

So, it's not asking, you know, about Jack from, you know, the other country, Jack from France or something like this. it's the jack in its database, but also it only pulls out real factual information from that database and it's far less likely to hallucinate. The downside is that is its reasoning model is much worse. So, because it's so like kind of linked to the factual information it has, it doesn't have other information. It's not constantly improving from the millions of other users outside your company that are using it.

But say for some companies that just want accurate document-based information, effectively like a, you know, a search prompt on steroids, then this might be the thing you want. Yeah, I'm glad you brought this one up because it speaks to the direction or one of the two directions I think the big directions that LLMs are going in. One is the kind of big public open usage one like the Mistro we mentioned or Gemini open AI uh there where everyone's kind of using it as like a bit of a Google right or as something to use that is trained on everything.

The other big branch is for that private enterprise type of use where you have some valuable data that is proprietary. So you don't it's not in the public domain. You don't want it to be in the public domain, but you want to have some value ad on top of it that's based on that proprietary information. And that's where this kind of rag system retrieval augmented generation can work, right? You can you can effectively train an AI to use that proprietary information to give better answers or tailored answers that no other company potentially can give, right?

or no one no other entity could give. So I think that's really interesting and lots of companies are moving that way and like how do we build our own rag based LLM or or rag or rag based LLM product essentially that is going to you know provide value that no one else can provide. So I think it's really interesting that you have some big players already like providing some of this stuff out of the box that people will just you know be plugging into very quickly.

Today's episode is sponsored by the Vchain Foundation. Now, we often talk about how emerging technologies like blockchain have the potential to revolutionize sustainability and Vchain is at the forefront of this green movement with initiatives like Vet Dao. Let's take a closer look. Imagine tracking small actions like recycling at low cost making them valuable assets. These verified records build a massive collective impact benefiting people through their own actions. Sound a bit complex, but Vchain made it fun. Explore our apps at veter.com. Vchain's goal is to power a new era of sustainable solutions.

So, if you've got an app idea or just want to earn rewards for small sustainable actions every day, go check out vb better.com to find out more. And now back to the show. And now we have the big contender. So, this is like the kind of the thing that rocked the stock markets in early 2025. All the models we've mentioned so far have been fairly basically western, right? US dominant, one French one in there. It's almost entirely US typically and some in Europe. Now when we talk about Deep Seek, this was one of the first LLMs to come well the first mainstream LLM to come out of China.

It came out in early 2025 and it led to an unprecedented kind of hit to the stock market and the reason for that was it was so very performant. A lot of say the stock market have been propped up by investment in AI companies in tech companies based on AI basically on the promise of productivity on the promise of you know you can use a model that can basically do the job of 10 people and that's going to lead to more productivity more output so basically more capital going into the markets and more money and this was kind of you know leading to a bit of a buzz in the stock markets over the last two years everyone was using more LLMs and LLMs becoming mainstream the problem was all of a sudden deep sea came out in early 2025 and there were two main things.

one was okay all these US dominated tech stocks they took a hit because all of a sudden they have a competitor from you know China so maybe the investment going to the US stock markets will not be the dominant monopoly that they expected to be and then the other one was the fact that this was an open- source model so all of these almost all of these models we've talked about so far are proprietary are closed are paid for whereas Deepseek was open anyone can use it for free and you know if you're a kind capitalist economy and lots of money's gone into it based on profit accumulation over time and all of a sudden there's a competitor that's as good as you doing it for free.

There's no profit in that and there's no money and the the global reaction was immense. Yeah, it it really took everyone by storm. It was like all of a sudden there was a new name in in the AI game that no one had thought of and it was kind of it it felt like it came out of nowhere but actually they've been around for a little while and they have been producing models but it was their first kind of big reasoning model that came out that kind of blew everyone away.

Um and just to give you some kind of scale for how performant it is and and why it's so cheap to use because that was you know the big headline as you said it's so performant at such low cost. um the the the precursor to the one that got all the headlines. Um the one that got all the headlines was this re reasoning model, Deepseek R, but the precursor to that Deepseek V3, which is just a traditional kind of transformer LLM. So that DeepS model took around 2,000 GPUs to train.

That was the estimate from from the Deepseek company. Um the equivalent model at the time was Meta's Llama 4. That would take 100,000 GPUs to train it. Right? already there. You have this kind of 50x improvement in cost for one of those. And if you combine that with the the approach DeepSeek as an organization took, which was very, as you said, transparent, putting the models out in the public and going down a very innovationheavy approach. They weren't just taking what existed and trying to see how far they could push it with hardware and with training.

they were actually making lots of new innovations in how the the neural networks themselves are constructed, different techniques um that we've mentioned so far actually they were making proper innovations there that kind of blew away lots of people in the west basically and one of the things I saw and this might be just Sam Alman saying it to save his own hide was actually they're like okay well deepse had to be trained using some of the earlier versions of OpenAI they wouldn't have been able to achieve this from scratch and acknowledge okay yes some of the improvements they've They basically did like batch tokens whereas like a typical model would think of say a request in terms of each individual word occupying a bit of space occupying a token.

Um deepse would batch sentences or batch like parts of sentences together to be more efficient and kind of more insightful. But he basically said they've been using open AI they've kind of used proprietary information. Whether that's true who knows but yeah there's definitely some crazy well some big strong innovations that came from that. But what is quite clear from the deepseek stuff is it marks like a a declaration by China, right? That AI would not be this closed competition dominated by, you know, three US tech giants, but it's going to be an open playing field and China's going to be one of the dominant players in that.

Yeah. And I guess it's no surprise, right? Like anything in the high technology sector has has had a big competitor or or role player somewhere in the supply chain coming from China, right? So that it makes sense, but I think no one could have predicted the impact that it would again like like a chat GPT moment. It basically hit everyone else out of nowhere. Um and they must have been very confident that this was going to uh going to have a huge impact. Um yeah, I think it's it's interesting because it's also a bit more compact.

It's something that you can use in a kind of more individual level as well. So you can kind of potentially fork it and do things with it that you can't with a traditional closed source model. Again, by the fact that it's closed source, you can't just uh do what you might do with a GitHub repo and and fork it and make your own version with chat GBT, but you have a something closer to that because these weights are published because the techniques themselves are published in much more detail uh with Deepseek.

So yeah, that question remaining about about the the training and the tokenization is interesting. But um this this again similar to Mistral is completely challenging the way that everyone else was doing at the time. I'm glad you actually I'm glad you emphasized that point because it is important. It's like the reason so much money was going into you know open AI into anthropic into Google DeepMind was because if I'm a startup and I I don't know I had a what's it called let's call it a healthcare startup and all of a sudden like every other startup I'm now an AI enabled healthcare startup and I want to use AI beneath the surface even if it's just a simple chatbot to say I've got these symptoms what do you think it is open AI the way you do that is you factually build your system on top of open AI but you don't really know how it works you kind of you basically do an API call to OpenAI's somewhat closed box and they would give

you the response back. So you have no control over that and you're paying per every single call and you're paying quite a lot in the grand scheme of things. Whereas all of a sudden DeepC completely changed that game. They gave you the model like kind of all the pre-trained stuff in a lot of ways. You would have to host it on your end. You could retrain it, optimize it, and effectively run it at cost. run it at the hardware requirements to run that model, your private model on the side.

And like I said, this just completely undermines any company that's trying to go for the first approach where you're the service provider, they come to you, they ask for something, you give it them back, and they pay every single time they do that because they can now run their own version on DeepSeek and basically run it at a fraction of the cost. Like it really did blow open the kind of the whole business model to a lot of these companies. Yeah. And I think this is also maybe a reminder of why I think AI is such a related topic to web 3.

And when we talk about web 3, we always refer back to, you know, the differences between web 1, web two, and web 3. And the fact that the web was originally built on open protocols, open standards. They tend to win over the long run, over the long time horizon. So I think that's part of the natural evolution of the LLM space that we're seeing is that you would have very large performant models eventually released that are open source that can be worked on more quickly that can be localized by individuals and enterprises more quickly.

Um it just it just makes sense, right? Because otherwise we just have the same gatekeeper problem, the same platform lock in that we've we've always complained about with web two, right? So I think that's it in in many ways this is the step that people in web 3 should want, right? they should see this as a as as with their etho with their ethos going forward which is you know I'm not sure everyone would necessarily agree but I think um yeah for me this is probably a positive thing because again it's more competition in the space as well which which can never be a bad thing really.

Yeah I agree. So let's go to the battle of the models. We kind of talked about the founding companies but like I say these guys are releasing models every few months. We put this the kind of like the idea of this episode out last week and already it's outdated because the we were looking at the stats for four, you know, GPT4 turbo and now I'm looking at the stats for GPT 4.1 which is completely different and is mind-blowing compared to the original. So yeah, this stuff is going to be outdated.

We're just going to try and give you basically the kind of the the highlights and the most important bits per those companies for the most recent models. Um, one thing that I do want to mention is the idea of tokens. So, tokens is kind of like the metric with which you compare how much one of these models can process. Not how well, but how much. So, it's I, you know, if I say a thousand words to one model, that would potentially be worth like a thousand tokens.

If it can understand all a thousand words that I just said, let's say that's equivalent to a thousand tokens. If I say a sentence of seven words and that's the max it can understand, then it's seven seven words. Now, like I say, these LLMs, they they're kind of the tokens are pieces of text or modal reads then generates. And like it's kind of somewhat tricky to compare these because each model might interpret a token differently. And like I said, DeepS may say one token is actually a whole sentence rather than a word, but we're just going to go with the tokens to just find how kind of smart these things are, how much information they can understand in one go.

Does that make sense? Yeah. Yeah, I think very it's a very different type of tokenization to what we normally talk about, but um yeah, I think essentially it's just the way I think about it is it's the way that your LLM breaks down the input that you give it, right? So, how does it tokenize it into discrete chunks that it can deal with and manipulate and run through rather than giving it the whole uh you know 100 line input that you may have given it in a very difficult prompt then and the bigger it can deal with the better because then you know the more tailored it's going to be.

But um yeah, that that's one of the key metrics you have for all LLMs. Okay, so let's go with like the context window. How much can these LLMs understand in one go? So when we did this like Friday, so a few days ago, GPT4 Turbo could understand 130,000 tokens in one go. The model that was released a few days ago, 4.1, can now understand over a million tokens in one go. So that's basically 10 times the amount of context window. It can understand 10 times the amount of information in one go which just shows how quickly these things are moving.

Like it really is crazy. Yeah. Like just to give you like the the other other models like the highlights, Claude can understand 200,000 tokens. So almost like five times less than the current GPT model. Gemini 2.0 can understand 2 million tokens in one go. And Deepseek around 100,000 tokens. Now I've used all four of these and quite regularly and that actually makes sense in the way I use them. So when I go with Gemini like I can ask a really detailed technical question, give it like lots of different information and it really breaks it down and it takes it really slow will give me all the kind of the ways in which it's thinking about the problem and it'll take a long time to produce the output because it is digesting so much information in one go.

Whereas the other models they're much speedier. They kind of they output fairly quickly. They're good for punchy little outputs but you know it kind of the way you use these actually does quite relate to the context window they have. Yeah, it's a classic tech trade-off, isn't it? It's like the larger the context window, the better the likely the output, but the slower and the more expensive it's going to be to do it. It kind of tracks for me that Google would have the biggest context window available right now, two million tokens because they're the big data specialists at the end of the day.

They do the biggest data of anyone really. Um, so yeah, that makes a lot of sense. Uh, and I was going to highlight one one thing as well when I was looking at this kind of summary we put together. uh the fact that it has been you know well over a year now for the update for the main uh GBT best performing model I was like well that's pretty slow for the industry like everything else is moving quickly and then they drop one over the weekend so yeah that's that again goes to show you can't you can't rest for a minute with this stuff and I think even on these context windows it's not just about the request so if I'm like create a research document for I don't know quantum resistant technology it's not just okay Alex said 10 words there That's what it can understand.

It's when it's creating the computation and it's trying to answer that question. How many different things can it keep track of when answering it. Yeah. So what it will first be like what's the request? Then it be like okay how many different things do I need to pull in to really do this properly? Like what is quantum? How does it apply to practical use cases? Then going into okay what kind of solutions break that down. So, it's kind of how many things can it keep track of when it's answering these things, which like I say with Gemini, Gemini is really targeted towards like research, technical, academic, when you want really detailed, complicated reports, but it's much slower, maybe more expensive in a lot of ways, but it can handle 2 million tokens, which is like double the nearest next, which is now CHBT 4.1.

Yeah, if you've used CHBT and you've seen it saying like it's updating its memory, like that's what it's doing. It's kind of like the memory of your your LLM as well as what it's processing. Yeah, it's a good point. the random access memory that you have available. So yeah, bigger is always better. So that's that's a positive. Childish. Okay, let's go with the outputs. So this is like how much it can say in one go. Kind of related to the answer, but slightly different. It's not what it's keeping track of.

It's how it answers actually produces the output. So like I say, the chap GPT 4.1 is 32,000 tokens. That's quite a lot. Um other ones, the biggest by far is Claude, which is 130,000 tokens. Again, this really relates to how you use these cuz like Claude is kind of known as the most romantic, the most personable. It can create fullyfledged stories and that's how a lot of people use it. A lot of kind of like say story writers, script writers, whatever will use Claude because it can basically output an entire script, an entire story, an entire book in one go because it's got such a maximum output tokens compared with all the other models.

And like I say, that's again it kind of relates to how these are being used and the use cases they're being targeted towards that everyone's kind of around the 10,000 tokens mark 32,000 with uh 4.1 which is quite good and then Claude blows them all out the windows on this with 130,000 tokens. So it can output huge scripts of information in one go. Yeah. And this kind of again speaks to the diversity you have in the models now. You're not just having models to compete to be the kind of the let's say the Google of models, right?

the chatbt of models. They're competing to be the best at certain things like different models are competing to be the best for coding now, right? Like being trained on certain data to do certain tasks to be the best at providing agency uh to AI, right? They're very they're becoming more specialized in some ways because there's different markets for each of those. So yeah, I'm glad you pointed that out about the uh the Claude model. Okay, so the next one is cost. when we talked about like how we're building on top of these models like as a someone that makes products or makes software or something like that and you want to be the AI enabled insert word whatever it's like when you build on top of these models it's how much did it cost to actually use them whereas Jack sends a request to my healthcare app and it's like I've got these symptoms I'm like oh well Jack don't want to know that information anyway I pass it on to the model I put

in the symptoms and it comes up with a response so like how much does it cost to actually use them so open AI is by far the most expensive. So, it's about well, it was previously per million tokens $10 on the input and $30 on the output. With the newest model, it's actually gone down to $2 and $8. So, we're seeing basically one of the cost and onethird of the cost. So, it's really come down and is actually, you know, cheaper than Claude. Now, one of the biggest concerns with OpenAI was how costly it was to use on the API side, but now it's actually cheaper than Claude and has gone to 20% of the previous price with this new model, which just shows like they're not doing it out of the goodness of their hearts.

They're doing it because they've now got a huge efficiency saving with this newest model that's come out. But by far, Gemini is much much cheaper. It's 10 cents per million tokens. So, that's like what is it? one 100th of the price of the other models on both counts. Um, but obviously the cheapest by far is uh DeepSeek because it's free, it's open source, like there's no need to API core. Like it's just it's a game changer. It doesn't compete. It's Yeah, it's crazy. Yeah. I think at the end of the day, that's going to be the bottom line for a lot of people.

And as with any technology, it be it's not just about the quality of the technology, it's about how well it scales. because if if you're going to make money out of any of these, you have to overcome those those crazy costs associated with training and the infrastructure typically. Um, so they're now all competing. The ones who want to stay closed like Open AI are competing to scale and drive down those costs. And the ones that are open, well, they don't h they don't have quite the same problem.

The other problem they have is how do they monetize effectively, you know, so they're going to be coming at the problem from a bit of a different angle. Um, but yeah, it's it's good to see that the average cost is being driven down pretty rapidly now because for some people it would have been, you know, pricing them out before probably as well. Yeah, exactly. So, I don't want to spend too long on these. I have used most of them and like the point is it's changing constantly, but how you use these like by far the best in terms of like coverage for answers, the highest quality answers across the board is this new model of GPT.

Like the previous model was still the best for answer quality, but the newer model is is really a gamecher. I think that like if you want something that can do as many things as possible, like is super versatile, super customizable, then GPT 4.1 is the one for you. Like it is just incredibly accurate, incredibly insightful, can answer most things very well. The problem is is it's quite expensive compared to other models. So like if you're building something on top of it, you are going to be paying paying a premium.

But like we just said, the cost has come down by 80% with this most recent model, but it's still much more expensive than an open model. Um, for Claude, if you like Claude is kind of best for style, personality, like scripts, marketing, it really does adopt that quite well. Like can learn how you like to write, what preferences you have, and it can output, say, marketing materials, I don't know, blurs, whatever you want, like even entire stories to your style extremely well. And then Gemini is the best for research because it's got such a huge context window.

It can answer really, really complex questions and pull in so many different sources of information and site them so well because it's got the largest context windows by far. And then I would say finally, Deepseek. If you want to do it free and cheap and easy, Deep Seek's the one for you. Like it's pretty good. Like it's very decent in answering and you run it for free. So like why not take a punt on something free? Yeah. And I think for the only thing I'd like to add really is that for anyone who thinks we're making very subjective statements here about how good they are, you should know there are there are pretty standardized ways to measure performance of LLMs now, right?

There are a bunch of LLM benchmarks that every new model is passed through and it tests them rigorously to see how well they perform on a bunch of different tasks. Um, and basically, you know, DeepSeek is is pretty good. It's not the highest performing people ever seen, but it's pretty close. And for that low cost, that's why it's it's such a gamecher. And again, you'll have different models. There's kind of these big graphs you can see where, you know, you have different axes for different kind of types of scores against different activities.

Um, so it's again a question of horses for courses. Some models be better for some tasks and we'll score higher on these. But yeah, there are a bunch of benchmarks. So I would encourage you to check them out so you can verify that we're not just we're not just hopping on about our favorite ones here. It is a kind of a science to it as well. I mean the thing is by the time people have checked these out the models are going to change and the metrics are going to change.

Like it's something someone pointed out when I was saying oh we're going to do this episode and this stuff. It's like by the time you air that it won't make sense because all the models will change. They'll all be able to do different things. One of them might disappear. But yeah that's what currently they are being used for. And I think like it's kind of like the it's kind of like the chicken and egg style type thing or self-filling prophecy where because the use case is this right now we're actually seeing that a lot of the features a lot of the kind of the research is tailored towards fitting that and they kind of are focusing more and more to solve the use cases that I just mentioned where Gemini is kind of like the allpurpose research technical tool and open AAI is like can do almost everything like versatile across the board very performant.

Deepseek is for the startups because they can do it cheap and claude is for more I don't know personality marketing style stuff that kind of thing. So we are kind of seeing that the narrative is self-fulfilling because they've got their target audience and the audience is requesting these features. So they're going down that rabbit hole but these things are changing constantly. So this is just to tell you where we're at right now. I think like some things we should talk about I think in terms like the future stuff is you mentioned it a little bit there Jack like one thing previously is like you mentioned was the idea of generaliz generalization versus specialization we are seeing some models more and more are going for very specific use cases they're like this is the thing we do legal documents and that is it and it's not about being super crazy performed it's about being very specific um being very dialed not exactly being correctness and like provability, explainability and then you got things like open which

is like super general can do as many use cases you can imagine pretty well but may hallucinate Yeah, I think uh there will I actually anticipate there will be someone who goes for the fully generalized version. I think it'll be you know open a or open I think it'll be open AI or Google most likely because you know Google are already playing in that space and open AI have the first mover advantage so whichever because there is an advantage to being general is that you have the widest potential market right so I think they're always going to going to be someone going for that but as we've discussed there are so many now that that much more specialized already for various different reasons so I think um I think we're the more we use agents too they're going to be hyper specialized, so they'll probably have preferences for which LLMs they use as the underlying LLM for that agent.

So, I think yeah, that's going to drive specialization even further. Definitely. Another thing is like the global dynamics and the sovereignty of the models coming into play. Like we'd seen we hadn't I hadn't noticed how much bias there was in answers because all of the models up until 2025 or the big models were western bias. They were US companies. They all answered in a certain way with the political bias. You could say some of the things about political correctness maybe showed that there were some differences in how they were trained and how PC some things were like but with like chi with Chinese models coming out now they answer questions in a very different way.

If you ask about one political event between a US-based model and say um deepseek like they answer it in a completely different way. I think the idea of bias coming into this and global dynamics having a role in these models is going to be is going to become more apparent I think as that we kind of get the proliferation of these models. Yeah. I think and also when you have the biases coming into models whether it be from the training or from the the sub subset of people on the planet using them they're they're kind of self-reinforcing right because the more they get used the more the biases reinforce themselves because people like those answers because they put the bias in themselves effectively when they're asking the questions for example so yeah I think I think globally diversifying the pool of uh models we use and the data that's training them is is only a good thing really uh so long as long as there's not too many nefarious actors out there trying to to

to input biases themselves that we don't know about. But yeah, that's that's one for another day, I think. Yeah, and that kind of relates to the point that you raised earlier is like the open and closed debate as well. What are these what are they being trained on? How are they used? Like what information is going into it? And seeing the success of the open models like Mistral and Deep Seek is incredible. like the fact that it's hard to know what these companies kind of business model is based on um because like how are they making money if everything's open source versus say proprietary models like GPT and Gemini like it's going to be really interesting to see how these things compete because you know chat GPT can't go to zero like they have to charge something for them so they have to really push the envelope in being more performant being more accurate offering more features otherwise everyone's slowly going to move to the free models yeah maybe Deep Seat's model was just shorting

US tech stocks so far, but that's the only one I can think of. Yeah. Yeah, that would not surprise me at all. And then the last thing that I think is quite important to speak about is the integration like how these things are deployed into workflows, into our phones, into everything we're doing already. Because it's not enough like to be an LLM where I have to go out of what I'm doing right now, go into a chat prompt, type these things, and then get an answer, then go back into my workflow, start to action it.

And you mentioned AI agents. that's going to be a big part of it. Like having AI agents that are integrated into workflows or having these tools like you know you're going to do a Google search but it's already got an LLM built into it and how that fits into it. Like that's one of the most important things and we're seeing partnerships being announced for every big LLM constantly because they know the big real game changer is getting these models built into already existing retail focused products.

Yeah. Yeah, and I think integration is such a good point to kind of end this on because that's been the biggest news or theme in LLM's this year other than deepseek or since deepseek is the you know MCP right model context protocol has completely gone off the charts in the last few months it's everywhere uh with good reason you know so this was something that was essentially a standard developed by anthropic and then and almost immediately picked up by open AI right which is already a big step showing you know corporation essentially on standards between these big competitors.

Um, and essentially this is a way of going a bit further than retrieve augmented generation that we talked about and giving you the ability to have kind of an endless open way of injecting data and context into your your LLM process. And the the most interesting thing I find about MCP is that it actually involves LLMs in the retrieval process. So it's like you can define a bunch of data sources like Wikipedia and and Google search or p private repository of documents that you want to be used as context for your your prompts and then the LLM itself will go and check what's available and then decide what's useful and then it retrieves the data and then it finally gives you your answer.

So I think that's really models already all the way. Yeah, exactly. But I think like one of the things that I've seen coming out of this as well is that we're seeing loads of tools that are actually built just to give people the option to use whatever model they want. So a really big one is AWS. So I when if you're building like any kind of I don't know AI service or whatever, I recommend having a look at AWS Bedrock. It's effectively like a a tool for building your own workflow with AI.

And one of the things that they've really pushed to the forefront is the configurability. You can switch between models like whenever you want. So if you want to go super low cost, you go deepseeek. If you want to do something extremely accurate, you go Gemini. And they know that these models are changing constantly. And they've created this setup just so that people can flick between them whenever they want. And like say where data is retrieved, where the user inputs, that's all isolated from the model itself.

And it just gets a very dialed input. But they're building for the future because they know that, you know, you can't bet on one horse in this race because it is constantly evolving. Yeah, it's a smart play not to not to back any one horse at this point. Um, but yeah, it's a good time to be a consumer of AI, I think, because you have cost going down, you have competition doing what competition does in the market and you have no shortage of choice. So, yeah, I think it's a think it's a good time and I hope that was a a quick overview of the snapshot of LLMs that will be valid for the next 12 hours or so.

Um, and yeah, maybe we'll have to do another one in the future to see what's changed. 100%. I think um, we had Dan on the podcast talking about this stuff and he explained it really well is that think of these like people. Think of them like assistants. They all have different skills. They all have different personalities. Use them for different tasks and that is constantly changing as well. So, use as many as you can and see which one you like and see which one works for you.

But, you know, this is a bit of an episode on the battle of the AI models. Like Jack said, it's already probably irrelevant by the time you're listening to it because this space is evolving so rapidly. But yeah, I hope you got something from it. Keep up to date with this space because it's moving so quickly. It's going to be so important for everything you're doing in the future. And thanks for listening wherever you may be. And join us next time to entangle a little more of web

Topics: AI & Robotics · Blockchain & Crypto

Related episodes

All episodes