Introduction and Hosts' Catch-up

[00:00:00] Jeff Roster: Well, hello everyone, and welcome back to another edition of This Week in Innovation. I'm your host, one of your co hosts, Jeff Roster, and I'm here with Brian, uh, who is a world traveler. Brian, where, where are you? Where, where have you been the last, uh, last two or three, no, four weeks, I think, right?

[00:00:15] Brian Sathianathan: I've been all around Jeff.

Brian's Global Travels and AI Discussions

[00:00:17] Brian Sathianathan: Um, so I think, um, in the early part of September, the late part of September, I was in Sri Lanka. I had a chance to speak, uh, in the United Nations, uh, Science Policy Council about the effects of AI. So did that for about three days. Um, and then I ended up, uh, going to Singapore and then meeting one of our customers in Singapore.

[00:00:41] Um, and then basically presenting, uh, with their, um, executive at the cloud conference. So cloud conference is a 23, 000 attended by 23, 000, uh, both tech and business executives, uh, in Asia pack and it's held in Singapore. Uh, so we presented, um, a, [00:01:00] our, you know, interplay AI solutions and also our partnership with one of our, uh, customers Fuji, uh, and then, uh, spend some time there.

[00:01:09] And then I get to. I get to go to then I get to go to Hong Kong. Um, uh, the following week, uh, spent a couple of days there and then finally came back to California. So it was a pretty long trip. I think I was out until that from the 26th to I think the, the. 19th or the 21st or something like that or the following month.

[00:01:34] Yeah. So

AI's Role in Environmental Sustainability

[00:01:34] Jeff Roster: let's go back and touch on the United Nations. So obviously AI has been, um, the front and center, uh, the white house has made some statements, um, on, on their council. You're at the UN. So can you, what, what can you share with us? Was it a public session? Was it under background? What, what

[00:01:58] Brian Sathianathan: can you will actually, uh, [00:02:00] See, uh, some of the reports, uh, from that session, um, on the internet, uh, right. Because what we spoke about there in that session went as a kind of all content got rolled into and went into the environmental council. of the main UN, then eventually that will help you make policy decisions and so on, right?

[00:02:17] So that given meeting was between the private sector in AsiaPAC and then the government sector, the public sector essentially, right? Uh, because what happens is, you know, there are, you know, the the world, the world is heating, you know, by by a couple of degrees, right? The one degree heating is going to happen, right?

[00:02:35] The problem with that Uh, here, the global warming and all the effects of all these things is that, you know, the, the, the, the, the, the, the deterioration is happening at a much, much faster pace, right from, you know, ice cap melting and all kinds of things. But what happens though, is in the Asia region, I think the top five or the top eight of the world's most polluted cities in these, [00:03:00] in these different countries in Asia.

[00:03:01] So, so they were thinking about how do we actually, you know, accelerate the environmental agenda. And how, you know, both the private sector and the public sector, you know, can help and, you know, all the way from the public policy to fund sustainability projects and environmental projects to how can the, the, the, the, the private sector get involved in it, both from a funding perspective, as well as from a technology, as well as from a, you know, um, expertise offering perspective, right?

[00:03:30] Uh, so my role was essentially to talk about like how artificial intelligence can help in this journey Uh, what happened was before that I had a a couple of companies. Um, that's locally based in these regions I had a couple I had a chat with a couple of those companies in terms of how well AI is helping, uh, in, you know, optimization of solar power production, you know, cold storage and efficiency usage in cold storage rate, [00:04:00] um, in efficiency gains in agriculture, because, you know, when even typically when in these countries, when people can.

[00:04:05] You know, plant and plant these paddy fields. There is water on top of the paddy field for at least like, you know, have like six inches thick, right? But all that water is wasted because, you know, the plants don't need that much water, but that's the process they've learned through, right? But by using AI and IOT sensors.

[00:04:23] You can, you can very precisely understand how much water to put, so you end up saving a lot of natural resources, right? As humans, you know, we end up using a lot of resources, you know, in an unnecessary way, right? And it was really well received, and in fact, some of the policy... Requirements or like some of those three takeaways I put in my, uh, in my speech, uh, in fact, was, uh, I think, uh, a part of some of the suggestions they made because I saw that in one of their documents.

[00:04:49] Right? So, which is great. That means, you know, people are listening to this message. And I think, you know, we want to, I think technology can change and it's a very hopeful message, right? Uh, but one of the things though that made me [00:05:00] realize while I started doing the research on how AI can help with sustainability and start, started to talk to startup companies in this region and try to gather actual KPIs and numbers, um, it became very evident in my mind, right?

AI's Unique Position in Technology Trends

[00:05:13] Brian Sathianathan: AI is unlike any technology trends that we've seen, you know, in the past. Blockchain, like augmented reality, you know, you name it, whatever, right? Whatever technology trends that we've seen in the past, AI, of course, is, is something that's like, it's, it's very, what I would call, like, almost like very, an omnipresent technology, right?

[00:05:33] What does that, what does that mean? It means that we have this technology that can be applied for every use case, right? Let's take, let's take blockchain. You can apply it in some banking cases. Fantastic. Well, you can apply in like, you know, pricing resources. I mean, it's good, but it's, but it's, it also has its limitation, right?

[00:05:52] The thing with AI is you can apply it across every industry, all the way from environment to sustainability, to medical science, health [00:06:00] science, insurance, it's everywhere, right? And it's fairly easy to apply it. And then you are seeing like amazing amounts of benefits very, very quickly because this type of, you know, intelligence is like really, really optimizing and looking at.

[00:06:14] Patterns that humans can't, you know, track on a day to day basis, right, which is amazing, right? So that's and then it's also, you know, good growing going across any given organization like let's say, you know You have a company in the retail sector now You know the the retail company can use it all the way from their consumer talking to their consumers And optimizing those conversations through AI to all the way to, you know, in supply chain and, you know, look optimizing, you know, payroll or like doing HR processes, hiring resources.

[00:06:45] So it's all, it's all across everywhere.

AI's Pervasiveness and Future Predictions

[00:06:47] Brian Sathianathan: So this AI is so, so really pervasive, it's going to be very pervasive and it's going to spread like wildfire. And, you know, you wake up in December 2024 and suddenly realize, you know. Every [00:07:00] analyst prediction has to be upped again and things are going to actually be, you know, the numbers may or may or not may or may not meet, you know, maybe one point, whatever the trillion dollar things that they're talking about, but you're going to wake up and see, you know, my God is AI is everywhere, right?

[00:07:17] Because companies already, I saw a report today in LinkedIn that saying that a lot of organizations have actively started both AI as well as generative AI projects, right? So it's top of mind for every board. It's top of mind for every C level leader, right? Um, and actually, especially in tech, tech companies, let's say like, you know, what penetration does generative AI specifically have in, uh, tech companies?

[00:07:42] I think it's about, like, probably north of 35, 40 percent penetration, right? I'll give you a classic example. Iterate on average across all of our customers and everything, every, and also our new product builds as well as Interplay. I mean, our, our flagship. revenue generating product. We [00:08:00] have typically on a given day, we have about 35 40 projects.

[00:08:04] This includes the build projects we are doing for customers, our existing Interplay apps versus, right, versus, uh, versus solutions, um, like Interplay itself, like the features inside Interplay, right? So if you look at that entire 35 40 projects, today, 16 of them, Uh, purely generative AI. That means it's, it's, it's like 45 percent for a company like Iterate, right?

[00:08:31] And it's true for every tech company. It's true for every height, larger high tech company too. So you're looking at 25 to 40 percent of a company's effort today, if you are in the tech business or in the business of selling tech has turned for within like last September till now into Into generative AI, right?

[00:08:50] But for retail or a traditional company, it's probably lower if you're a bank, right? I think today is probably like at the executive attention level. It's probably 30 percent [00:09:00] attention or 40 percent attention But from an actual project perspective today It's still under less than five percent based on some of the analysis that i've done, right?

[00:09:08] But that number is going to like rapidly change in 24 Because now we are in the financial cycle, financial planning for FI24, right? Because most companies, we are like where their financial year starts in, right? Uh, next year or, or next January or next April, they're all. Thinking about their, their, what their future budgeting looks like and every leader that I've spoken to has generative AI in their budget, both our customers as well

AI Budgeting and Investment in Companies

[00:09:33] Jeff Roster: as so Brian, what does that mean?

[00:09:35] It's AI in their budget. Um, is that a separate line item? Is that something that the analyst or community is going to be forecasting? What does that look like?

[00:09:45] Brian Sathianathan: It's both. I think what happens is generative AI is going to be generative AI specific. Thank you. Specific projects are going to be a separate line items, but then generative AI is also in marketing.

[00:09:57] So if you are a CMO in your budget, you, [00:10:00] you budgeted like, okay, now we are doing targeting, or we are doing like messaging on our website, or we are responding to social media comments, but we are just doing using whatever tool point solution we've got today. But how do we actually train generative AI to do that?

[00:10:16] So they put that as a part of an existing funded project. There is a, there is a generative AI sub item inside an existing funded project. So that's for some of the players. In fact, that's, that's true for most of the traditional companies because they already have existing projects they are planning for FI24, right?

[00:10:34] And they are putting generative AI as a sub item in there. But then, uh, companies that are also looking at building their own IP, right? Because the beautiful thing about this space, uh, Jeff, right? Is that, you know, like... If you look at all the big, the big tech, right? The big five, the big six, how many, how many bigs you want to add to, right?

[00:10:54] They all have a, like a Google, Amazon, Apple, Facebook, right? All these guys, they all have a search [00:11:00] engine. They all have a video website. They may or may, they all kind of have a phone or like a bunch of very, very popular super apps in the phone, right? Um, they all have all these end consumer interfaces. So they all want to know, you know, what the 300 million people in America are doing every day, right?

[00:11:19] They want to be a big part of your attention economy, right? So from the time you wake up to the time you go to bed, uh, these big companies want to be a part of that, right? But, but they are not always a part of your entire life in everything you do. I believe, at least my estimation, there is up to about a 20 percent of your life that big companies are not a part of it, right?

[00:11:41] That 20 percent is the time, you know, where you are at church or you are at, uh, you know, you're at, you're getting a haircut, right? Or you are in, or if you're a lady, you're in, you're in the beauty salon, right? Right. Or the time, you know, like, you know, you are at some local restaurant. So there is this time where like traditional [00:12:00] businesses do have.

[00:12:02] Your information like banks, right? Like and I'm like and I'm doing tax So I'm like logging into my bank and sending stuff to my accountant, right? So all these all these times where you know These big companies are not a part of your life and that part of that 20 percent of your life, right? At least 80 percent of that, so 20%, is basically owned by traditional companies, like beauty companies, like banks, right?

[00:12:28] Like places where you go pump gas, gas stations, places where you go buy, you know, grocery, supermarket, retailers. So this information is, this information is only owned and accessed by these traditional companies.

Building Intellectual Property with AI

[00:12:43] Brian Sathianathan: With generative AI, this is an opportunity for these companies, right, to actually take their proprietary data and train these generative AI large language models to, to give advantage.

[00:12:56] For their business, this is the time where they should not, you know, upload this to chat [00:13:00] GPT or give it away to these big fives, big tens. They should take it back. They should build IP modes around their business, right?

[00:13:07] Jeff Roster: So those are, those are, let me make sure I'm understanding that. So you're talking basically about building, um, uh, we used to call that, what do we used to call that, um, software in the old days?

[00:13:16] Um, on prem? I mean, you're not really, you're talking about an AI model that's built that is...

[00:13:25] Brian Sathianathan: That is trained on your proprietary data, right? Um, or, or you're using an AI model to answer information, like analyze your proprietary data, right? But that, that does not have to be on premise. It can actually be in cloud, but it'll be on your secure cloud.

[00:13:40] Or if you are running on AWS or Google, it'll be in your private segment. But you're not actually pumping all that into a, into a networked large language model like a chat GPT. So this is an opportunity for larger companies, especially traditional businesses, right? When I say [00:14:00]traditional businesses, I mean companies whose core is not tech, right?

[00:14:03] Today, every company is a tech company because, you know, tech is everywhere, right? But, but the thing that you're selling to customers is not tech. What you're selling to customers is essentially. You know, your, your retail product, or your banking services, or your insurance services, or your vehicles, whatever it is, right?

AI's Impact on Traditional Companies

[00:14:21] Brian Sathianathan: So this is the thing that I think they should look at, because now there is a part, there is a black box in every consumer's life that Big Tech does not have access to. And that black box is owned, and part, most part of that black box is owned by social organizations and traditional companies. And this is the opportunity for traditional companies to take the data they have.

[00:14:43] about you and use that to train large language models so they could serve you better and also build an IP mode for their own business, right? So this is something that I think we've been telling, we've been meeting with several board members and various leaders in several boards. We've been [00:15:00] talking about this because we want to help because we are in the business of helping traditional companies and leaders in this space.

[00:15:06] Right. So there is a lot to think about here. So now going back to the budget conversation, the budget, uh, FI24 budgets will probably have two major sub themes, two major themes. One theme will be a sub theme on existing project. How are you, how are you going to introduce AI or generative AI on existing projects, right?

[00:15:24] The other is how do you actually, the other second section is. Some companies, it's not true for every company, some companies that have a lot of access to data, especially banks, insurance companies and all, they will, they will basically think about, wait a minute, we have a lot of customer information, a lot of transactional data, a lot of really powerful data about usage and customer access and their interactions.

[00:15:48] Can we train large language models? So some of them will end up training. What would we call private LLMs or private large language models? When I say private, it doesn't mean it has to always run on premise. It could be in the cloud, but it's not. [00:16:00] It's not that they're using an existing third party like a chat GPT or whatever.

[00:16:03] It's their own model. They own the model, right? They're not talking because typically when you work in the, in the AI space, you have two options, right? You could train your own private models on certain types of data that you own the model, or you could go. And work with any number of Silicon Valley startups, like all the, which we call point solution, right?

[00:16:22] The, as you, as you know, like with generative AI, now, if you go to a Starbucks, every startup founder, right? Uh, who's worked on any idea, whatever idea, security, blockchain, whatever they worked in the past, ports, analytics, right? You give it a name, right? Uh, they are all transitioning themselves to become generative AI.

[00:16:41] So all these generative AI startup companies are knocking enterprise doors now, right? And say like, saying basically like, now we offer generative AI for this, generative AI for that. Basically generative AI is getting verticalized, right? Which is great. So what happens is if the data, if you are, say, if you're a large company, right?

[00:16:59] Like [00:17:00] large retailer, right? If you're a large bank, you're a large automotive retailer, you're a large Insurance company, right? You want to think about like, look at, look at my current projects. Which projects are core to my IP? Which projects can I build unique intellectual property out of? The data is unique for me, right?

[00:17:20] In those projects, you should not use point solutions or companies that are providing SaaS. Because when you work with a SaaS company, what they'll end up doing is all your data will go to them. Instead of giving it to open AI, you'll end up giving the data to them, but you won't get access to the model, nor will you own the model.

[00:17:37] They'll give you a solution, right? You can pay 9. 99 and get their solution a month, right? But what you really want to do is look at your, look at all your, across your projects and say, okay, these, these areas of my project, I'm going to build an IP mode, but these are non critical or non core for me. These projects, I'm just going to use a third party point solution, right?

[00:17:58] Um, I'll give you a classic example, [00:18:00] right? So today, you know, you, you know, you want to, you're a, you're a retailer. You want some of your blogs to be rewritten in your tone of voice or something. So there are so many SaaS companies that could write your, like, like using, like, and you click on it, put a little web plugin and it'll regenerate your product description page or whatever, right?

[00:18:17] Uh, or like, or like simple things, right? Like where you want to record a video and then they will change the background. Simple things like that you can actually use, or like generate some marketing images for you. Uh, solutions like that, you could actually easily use a point solution. But think about where you are doing customer analysis, or things, things about where you are, where you are actually doing loyalty on customers.

[00:18:39] Or you are doing like, or you are providing conversational, transactional commerce experiences with customers. Those are areas we are, we are deeply touching base or touching the customer. Or if you're a bank, you are doing loan processing, right? Or straight through processing, right? Those are core to your business.

[00:18:57] And you don't want to outsource them or use a SaaS [00:19:00] type point solution to it. That's why you want to use your own AI models. So that, that distinct, that, that, they need to distinguish in terms of where, where to build versus where to buy.

AI's Impact on IT Services and Staffing

[00:19:10] Jeff Roster: Jeff, any, any questions? The software we're using to edit this, um, this application or this, this podcast, um, just now went with, um, a really interesting AI addition. So all my notes will be run through AI and we'll actually be able to make some super easy links, which will be really awesome for the, the, the YouTube videos, um, using AI.

[00:19:34] That is built in the software. So if I buy, so I guess that that's what I'm trying to get at as an analyst, uh, is to understand how, where all this, all this shows up. Because you know, when I turn in my tax reports for, for, uh, for this week in innovation, it's, uh, it's just descript the software, but now has a lot of AI and we'll have more.

[00:19:56] And I suspect some of the applications you talked about really is just. [00:20:00] is AI built into existing applications. But then you also talk about, I mean, I, I'm here a lot of, a lot of, uh, IT services. I'm hearing a lot of, um, other components that are very uniquely AI, um, AI components. So when I'm looking at. If I was back in my old job of, of building an IT forecast, I'm trying to think of do I have a, a new line ai, um, certainly AI IT services without a doubt.

[00:20:27] And it sounds like maybe that whole large language model might become, I don't know, I guess that might be a unique AI component.

[00:20:35] Brian Sathianathan: A large language model will become sort of a, like, it won't be for every company. Right. I think there will be three types of, uh, three, three types of, uh, sales that will happen, right?

[00:20:45] One is existing apps, like the one like you are using here, right? Or like a Zoom or whatever, Microsoft Teams. Uh, they all, they all will end up using, right, you know, generative AI. Like, like, [00:21:00] like you're filing taxes. I'm sure quick. And one of these guys will eventually start using. I wish it was

[00:21:04] Jeff Roster: quick. And it's, it's, it's, it's a good old batch of CPA.

[00:21:07] So, yeah,

[00:21:08] Brian Sathianathan: yeah. So every, every app that you have will use generative AI. So that will be transparent to you. And that will mostly be a feature, right? Which will not, which will most likely will not even increase your budget. It'll be the same budget, right? Because now these providers are integrating it.

[00:21:23] What's already there and then they're just trying to create use generative AI as an inch. So that's method one, right? The second one is basically like if you want to use Generative AI for specific purposes for like blog writing, right? Or if you are using it for like, let's say like, you know Remessaging or translation or whatever Or like things that you're trying to use where it's not core.

[00:21:46] It's important to your business, but it's not very core Where it's not core for building IP Uh, you can actually use a dedicated generative AI startup solution There's a lot of point solutions that are out there that are doing that, right? Or like like for example, you want to [00:22:00] create some amazing marketing videos, right?

[00:22:02] There are a lot of like generative AI marketing, uh video software you can go in Click a bunch of things, pick the theme, like you can even put your, your bumpers, your starting screen, your ending screen. It'll just generate the whole thing and it'll put your company logo in the background, do all those cool things, right?

[00:22:17] Edit the volume, right? So all that stuff, you know, they're, they're not core. I mean, they're critical to the company's operation, but they're not a core IP building. So that type of stuff will also be fast, but from startups. So they're not existing solution. They may be just solutions specifically designed for those needs.

[00:22:33] Uh, uh, but might be generative AI point solutions. And then there might be the third option, which is where you need to actually build IP. Right. So those, those is, uh, you know, those, those are, those are the ones where you need to start, you know, looking at train, taking your data, preserving it, protecting it, and training them, training models by yourself and building private LLMs or large vector databases to that, that can basically, uh, [00:23:00] understand your business.

[00:23:01] So you can, you can create autonom. Autonomity and, you know, autonomous capabilities, uh, on your existing business, right? So

[00:23:09] Jeff Roster: let's let's take a 10 billion retailer, um, fairly aggressive adopter of technology. Um, five years ago. Probably with spending zero on AI. Um, let's take that person, that, that company to 2024, what dollar amount or percentage, I mean, do you have a dollar amount?

[00:23:29] Do you have a percentage of their it spend? Typically it spend is anywhere from one, one and a half a percent of rev to it. So a 10, a 10 billion retailer would have one and a half percent of that 10 billion of that budget. How big do you think those large language models? Are and the services wrapped around them to develop all this capability.

[00:23:49] And I think we're talking about the features and functions in existing software or even this new software you're talking about. I still think it's just going to probably being, um, lumped into marketing software or whatever, but the large [00:24:00] language models really does seem like that's like a unique new area of spend.

[00:24:04] How? I mean, what kind of number do you think? I mean, I haven't even begun to think about this. I've seen the numbers that are insane to me, honestly. So, um. It's just hard for me to wrap my mind around how big those numbers are.

[00:24:16] Brian Sathianathan: So I think in the ai, right, even before generative ai, right? The two, 2020 or the 2021 timeframe, early 2000, 20, 21 timeframe, right?

[00:24:27] Pre-check GPT. Even if you look at those markets, well, the, it, it budget is 1.5% or 2%, typically, right? Two to 4% depending on how you look at it, right? Um, but then. AI has been there in some degree, especially for data analysis and all those things, right? They've been using AI. So it's a very small, out of that one and a half or 2%, AI has always been very tiny, right?

[00:24:52] So that's another, another sub 5 percent of that one and a half, 2%. But AI has actually been, been used on the [00:25:00] marketing side more extensively because every, every retailer, even if you're a 10 billion retailer, you're probably spending. 12 to 16, 17 percent of your budget in marketing, right? That includes ad buying and, you know.

[00:25:13] Right.

[00:25:13] Jeff Roster: No, it's a, it's a huge number for sure. I know about 12%, but I know it's, it's a massive number compared to even the IT spend.

[00:25:20] Brian Sathianathan: Yeah, and it's a full funnel operation, right? What I mean to say is, depending on, you know, who you are and how your marketing strategy is, you know, there is a lot of, like, there are 200 components in a traditional marketing funnel, right?

[00:25:31] So, like, most of the funnels already had, you know, personalization, recommendation. Those were all marketing spins. They were not IT spins. Because they had a part of their, part of their commerce offerings or like, uh, so all those things rolled in. So, so if you look at an organization, like, like even, even post generative AI, uh, has always been like, probably if you roll everything, just pure AI budget for a retailer, probably if you take the full spin, it's probably half a percent [00:26:00] or 1 percent previously.

[00:26:01] Right. And the whole out of the entire retailer spend, uh, but now I think with generative AI, it probably will go higher. It'll, it'll probably double, right? And then, and then, uh, probably it'll be a percent of a 2%, depending on how you are doing it. I mean, retail, I think will be affected less by it because in the marketing, what will happen is the marketing folks are always campaign driven.

[00:26:25] So what they will do is they will go and buy a lot of point solutions for their campaign stuff And the it guys might do an investment Some marketing some cd especially in companies where you have cdos and you know ctos type companies those companies will actually Do a little bit build a little bit more ip because they would always think about Initiatives across their sort of building their own version of a GPT type thing.

[00:26:50] So those companies will probably spend a little bit more, right? Uh, but I think in banks the spend might be a little even higher because in in in banks The way these [00:27:00] operations go is that wherever you can justify Uh straight through processing that means from the time you deposit till you apply for a loan till the time the time the loan is Delivered how what percentage can you do straight through fully fully autonomous right without human?

[00:27:16] Those things can have significant savings because in the bank more than the, the, the, because in retail, it's always, it's always been like a, a lot of top line optimizations in terms of revenue increase, right? It doesn't mean we don't do bottom line, but it's essentially been a lot of top line, right? Uh, in terms of conversions, commerce, everything, uh, banks and all those guys, they focus a lot on like bottom line optimizations, right?

[00:27:40] So if you can like, you know, do straight through processing and cut like, you know, 20 percent of the staff. In straight through processing, that's a lot of money saving. I mean, it's significant amount of dollars, right? So AI will very quickly begin to influence all those things. And even in retail, you will begin to see this, especially in places like stores.

[00:27:57] Uh, generative AI will be one of those technologies that will [00:28:00] penetrate stores. Because a lot of other technologies have always, you know, tried to be in stores. Like, let's take, let's take computer, right? You can say, like, how many people are coming, people counting, all those things are there. But their penetration has always been somewhat limited.

[00:28:13] I think the generative AI, it will penetrate stores in a big way. And not only both stores and commerce, right? But this will be one of those technologies that will be everywhere. You'll wake up on, wake up in December 20, 2024 and, and see like this has made, this, this has grown faster than you thought it would.

[00:28:31] Where

[00:28:32] Jeff Roster: is this netting out? Are there new employees being hired in the IT shops to work on AI? Is it still a skunk works? What, what are you seeing, Brian? I mean, you're sort of in the middle of all this. I mean, is it new staffing? Is it redeployed staffing? What are the job descriptions for folks working in AI in retail right

[00:28:49] Brian Sathianathan: now?

[00:28:50] It's been all over the place. Right. Uh, some places where they are, they're hiring, you know, people like prompt engineers, people to interact with existing large language models and, but [00:29:00] issue the right, because you got to talk to these things in a certain way, right? So some places where they've been doing that.

[00:29:05] Uh, there has always been a head of AI or an AI head position that, uh, that, that in a lot of companies are, are hiring, right? And they're also trying to hire generative ai. The other is a lot, a lot of the data science team in, in traditional companies and retailers and other, because most retailers now have at least a top.

[00:29:23] The top billion dollars, they all have some kind of data science teams, right? So the data science team and the innovation teams are actually upskilling themselves to become generative AI experts, right? Especially using products like what we are offering Interplay and others where, where you can allow these folks to quickly, you know, learn this type of kits and begin to drag and drop these things.

[00:29:43] So I think there'll be a little bit of a upskilling of existing staff, a transitioning that happens, right? Uh, that's what will happen, but I think there might also be a little bit of, uh, core IT challenges that might happen too, because today if you look at, like, the IT staff, especially on the, on the in [00:30:00] store maintenance, so today most of the, like, in the, in the back office of all the, all the retail stores, right, they all have CPUs, right, typically computers, right, like traditional computers running in there.

[00:30:10] But with generative AI, now all those IT people have to begin to maintain GPUs. Thank you. Because the large language models won't really run on CPUs, right? Uh, we are working on large language models that can run on CPU with specially with CPU providers. So, so we are committed to that industry. But what's interesting is that's going to be an interesting shift on IT.

[00:30:30] So there'll be a lot more, lot more businesses, uh, in the future. IT folks, if on retail. Asking vendors for edge edge computing as well as CPU powered computing capabilities because they are not still, uh, they are not still comfortable with GPUs and building and maintaining and managing GPU systems is not very easy, especially on premise cloud is different because cloud.

[00:30:56] It's a little bit transparent to you because the cloud providers have made it easy, right? [00:31:00] But, but cost wise, even in the cloud, it's orders of magnitude high, right? It's not a simple, like we did some, like CPU was a GPU calculations, right? It's like 4X higher, four times higher, like 4X is a 400 percent increase, right?

[00:31:16] So that's not, that's very, you gotta be super careful. That's why I think a lot of retailers, a lot of these companies are going to start thinking about, you know. What, what the eventual cost of generative AI is going to be, but at the same time, that's not going to deter, right? That's the, that's the evolution in the step, right?

[00:31:33] Because when we invent great technology with great technology comes great, great pain and great responsibility. So this is a great thing. And at

[00:31:40] Jeff Roster: some point cost gets into it, which, which I'm starting to hear little rumblings about. It's not my, certainly not my area of expertise, but, um, Yeah.

Potential Winners in the AI Space

[00:31:48] Jeff Roster: Who, who are the big, who are the big winners in, in AI from a tech perspective?

[00:31:54] I probably have about seven names I can think of, but I'm just curious, who do you, who do you think are going to be the big winners [00:32:00] initially?

[00:32:00] Brian Sathianathan: From the tech side, I think you have to break it into who they are. So one is the, the chip at the very bottom is chip players, which is basically NVIDIA, Intel, Qualcomm, AMD, right?

[00:32:13] The chip players, right? And then there are all the other smaller chip players too, but these are kind of the main chip players, right? Then on top of the chip, there is the infrastructure players, AWS, Microsoft, Google, GCP, Google, right? And then the other ones, HPE, Oracle, right? Infrastructure players, right?

[00:32:29] Then, then you have basically the LLM makers or these LLM providers, right? Like for example, OpenAI, Anthropic, right? Um, companies like that, that who are building these, uh, Octable, uh, Diffusion, Stable AI, Stability AI, all these guys who are providing the, the, the, the, the software, the LLM software, right?

[00:32:52] Then on top of that, you have these application companies, application, application, uh, building like application [00:33:00] software companies that are allowing you to build generative AI, right? Like Interplay, like Iterate AI, or like, you know, like it's like LangChain, right? Any of these guys who are actually allowing you to build AI applications.

[00:33:12] There are a number of startups that allow you to do that. And

[00:33:14] Jeff Roster: those are the low code players that are, that are using low code to be able to play. Yeah. Okay. Differentiating from the big package software.

[00:33:21] Brian Sathianathan: Yeah, package software. Because they are, right? They're allowing low code to play, right? And on top of that, at the very tip of it is all the point solutions.

[00:33:27] Point solutions means SAS, pure SAS. So the SaaS is actually what their SaaS providers are playing, vertical SaaS and you know, all that stuff. So now the question is who will be the winners? So at the very bottom, right on, on the chipset stack or the very, very bottom of the base, I think NVIDIA, as you know, is already emerged as the winner, right?

[00:33:48] Which, which is kind of, you know. We all know, right. And then, and that's gonna grow and grow and grow. GPU needs gonna grow. Right? Uh, but there might be a, another winner

[00:33:56] Jeff Roster: we're at

[00:33:56] Brian Sathianathan: the very, very bottom of the stack, uh, of the, [00:34:00] of the core chip set. Right. I think as we know. NVIDIA has already emerged as a winner in terms of GPUs, right? Because they are the leading GPU providers. So I think they are going to continue that, that, that path.

[00:34:12] Because what happens is in this generative AI, AI in general, it's an 80 20 game, right? What I mean by that is 20 percent of all AI operations will involve GPU. That means whenever you train, you need a GPU. It's hard to train on a CPU, right? But then... When you are running these models, uh, several companies and including us and others are doing investigations on how to effectively run a lot of these, lot of these models.

[00:34:42] So I think in this journey, both Qualcomm and Intel might emerge as winners because Qualcomm will emerge as a winner specifically in the edge category where you are deploying it on the very edge in the store where like traditional computers are not used, right? It's like specialized boards and things like that, right?

[00:34:58] Uh, because that that case is across [00:35:00] everywhere. It's in retail stores or like, you know. In a retail point of sale, you have that case in a, in a QSR. You have this case in military, right? So this case is everywhere. So, so, so they will actually get a lot of traction specifically for inference. Like that means these models running doing predictions with the models.

[00:35:18] Uh, Intel, I think will also emerge as a winner, I think, in their, in their specifically wherever traditional CPU is used. Because most IT companies, right, uh, uh, and most retailers, right, Uh, they use traditional, you know, like they use traditional computers, right? Like they work with an HP or a Dell as a provider, and they provide computers with traditional CPUs, right?

[00:35:40] So there will be a, there will be a plethora of more powerful CPUs that will get deployed. Uh, and, and I think that will be a good opportunity for players like Intel and others to play there. I think, so in that stack, I think those three will, I think, will do really well, right? Uh, uh, uh, Nvidia. [00:36:00] Intel and Qualcomm, right?

[00:36:01] Uh, the one level of the upper stack, if you go one level higher, now you have the, the infrastructure players, right? Infrastructure players are going to benefit the most, right? Um, it's going to be like that saying, right? Uh, what is that? And it's a win or lose, we boost kind of concept. I don't know what they say here, but that's kind of very, you know, like, especially when you are going into sports games and all, you know, supporting a team, you know.

[00:36:27] You don't care which team wins. You know, eventually you party. Well, you're talking

[00:36:30] Jeff Roster: about the casinos. Um, they don't care, they don't care who wins or loses because they're taking cut off both

[00:36:35] Brian Sathianathan: sides. So, yeah, so I think the infrastructure players are going to, by

[00:36:40] Jeff Roster: infrastructure, you mean, you mean basically AWS, um, Google cloud and, um, And Microsoft.

[00:36:51] Of course, yeah. Right.

[00:36:54] Brian Sathianathan: And they're gonna win. And they're gonna win. Oracle also could be, because I think Oracle is catching up as well [00:37:00] specifically in generative AI. So all these four guys can probably do very well. Whatever happens, they're gonna be a part of the game. Right. Um, and then on top of it, application builders, uh, who are providing tool sets, like, you know, in a play and all, all those things, uh, the, the verticalized tool sets for application building, uh, in certain verticals, I think, you know, players like us and a few others will win because, because end of the day, large companies want to build and, uh, and customize IP for themselves.

[00:37:31] Not everybody wants to just switch to a SAS. Right. So I think that when that, that, that market, I think that that's a more stable business built there, uh, specifically with larger companies, companies that are whose market caps are revenues are greater than a billion dollars, right? They will all go towards, uh, building and securing their data.

[00:37:52] And they, and they are going to be very careful about how they use AI. And that's, and also like security, compliance, all those things will kick in, right? [00:38:00] The other is for big, large traditional companies, AI does not operate without any data. So integration into legacy is a big part of it, right? So those, that stack, I think players like us will win.

[00:38:10] Then on top of that, now you have the, the, the, the, the, the, the, the, the SAS tech, right? The SAS tech, you are going to have a lot of companies, lots and lots of companies, right? That's one of the things.

[00:38:22] Jeff Roster: I'm not sure I know what you mean by that. What do you mean by, by. Are you talking about the application providers?

[00:38:28] Yeah, I'm

[00:38:29] Brian Sathianathan: talking about SaaS providers. Okay, got it. So that's everyone. I'm talking about, you know, like, I'm not talking about the traditional Salesforce or so forth, because for them, generative AI is a part of an existing offering. I'm not talking about SaaS. I'm not talking about SAP. Right. I'm not talking about those players.

[00:38:46] I'm talking about like the company companies that come out of Y Combinator, the ones that get funded in entries by entries. Oh, okay. So, I

[00:38:53] Jeff Roster: mean, the startup, the startup community at this point, I mean, coming with new functionality.

[00:38:58] Brian Sathianathan: Yeah, there is, because [00:39:00] I don't want to just completely blanketly stay a startup community because there is a.

[00:39:04] A group of startup companies that are actually working very carefully on what the infrastructure is going to be. But the ones that simply say, okay, now you have a generative AI based block software, or you're going to have a generative AI based, you know, legal software, generative AI based document analysis software, whatever, right?

[00:39:20] The ones who pick a vertical and do a SAS, right? Generative AI based dating matching profile software, right? Whatever the thing is, right? And then, and then things like those companies, there's going to be like a million of them. And then very soon all the VCs are going to ask, like, you know, where's my money, right?

[00:39:37] Where's my contract, right? And then all these guys are going to start knocking on doors, uh, on sales, right? And eventually there's going to be mega consolidation. So you're going to see like, like now you're seeing amazing funding in generative AI in the SaaS space. You're probably like next August, September, you're going to, some of them will emerge as winners and unicorns.

[00:39:58] And most of them are going to die. [00:40:00] The same thing that played out in every other industry will play in SaaS, I

[00:40:03] Jeff Roster: think. Yeah. So I mean, I mean, the, the, the failure rate is, is, is in the AI community is going to be the same as it has traditionally been. I don't know if there's an official number, but it seems like it's nine out of 10 startups go, you know, either absorbed or.

[00:40:17] Brian Sathianathan: Yeah. Yeah. So those are things I think there might also be a lot of early acquisitions too, especially there'll be a lot of simple aqua hires, whatever type stuff that will happen. I think, I think the thing is now they are all in the overvalued funding cycle, but then what will happen is, um, reality will probably hit next October, uh, next year.

[00:40:35] Q3, Q4, for summer, you'll begin to see, you'll begin to see.

[00:40:40] Jeff Roster: I'm assuming the IT services guys are going to do quite well in this whole space. Yeah.

[00:40:45] Brian Sathianathan: I did not cover them. I think they will do very well. I think they will do very well. I would

[00:40:49] Jeff Roster: assume every one of them is going to have a practice. I mean, it's, it sounds like it's probably.

[00:40:53] Brian Sathianathan: Yeah. For them, it's a, what I call a sandwich, right? What I mean by the sandwich is at the very top of digital, because if [00:41:00] you look at the way the IT service. Services companies, they sell their offering, right? Most of them, even if you are a tech IT service, most of the companies have a strategy consulting on to it, right?

[00:41:11] So the, so it's like a, the strategy consulting is going to be started doing these board presentations, capturing the, the executive excitement from the very top, right? And then on the other side, uh, their delivery teams are going to deliver, right? Uh, but I'm sure they will also have failures, lots of failures too.

[00:41:29] Because, because the skill set in generative AI is, is much more complicated than it, it's, it's, I'm sure you, you know, it's not, I mean, you can do generative AI apps by, you know, reading medium and learning all those things. But getting LLMs up and running and, you know, like productionalizing them is not very easy, right?

[00:41:47] So it's like, but eventually they'll learn. I mean, because these companies have deployed, you know, high scale stuff, so they'll, they'll eventually catch up. But I think they will do well because they got, they got the sandwich because they got the very, very front and they got the very [00:42:00]back. So I think they will do well, but they need to upscale really quickly.

[00:42:03] And they also might use a partnership strategy where they'll work with other type of. Software companies like us and others, where they, you know, the skill set they can quickly procure very quickly begin to go to market very quickly, right? So that's kind of where those will be do. Then those will be, but there's one other segment I forgot is the ones who make large language models, right?

[00:42:23] Let's take companies like Anthropic, uh, companies like OpenAI, right? All these research labs that are. They're on large language models, right? A stable, stable, stability, AI, right? Uh, how many,

[00:42:35] Jeff Roster: how many, how many are in that? That's a slide. I think you, you showed me once. I mean, there's how many, I mean, like 30 or 40 in there, in that space.

[00:42:45] Brian Sathianathan: Yeah, there is, there is at least about, uh, 30 plus companies in that space, but the ones that have sort of broken the. Sort of broken the glass ceiling and kind of, you know, got into, you know, some level of traction and, and, and built models that are [00:43:00] useful, uh, probably around eight to 10. Right. Uh, but the thing for them is, you know, today we are all beneficiaries for the hard work, you know, they did.

[00:43:09] So I also want to be very respectful to them because, you know, if they haven't done what they did and, you know, built all these things. No, it's fine. It's fine. I think they've done a great job. But one thing I want to point out though is training a large language model is super, super expensive. Right. Uh, so typically these companies have raised hundreds of millions of dollars, right?

[00:43:30] And also have burnt 20, 30, 40, 50 million dollars training these models, right? And to some degree, like they are, they are in an arms race, which requires a lot of GPUs. It requires a lot of algorithmic inventions, right? Because algorithmic, it's great that GPUs, you know, are becoming faster and more capable and powerful and power efficient and everything, but without the right algorithmic inventions.

[00:43:56] Uh, you are not going to make breakthroughs. So those, [00:44:00] those algorithmic inventions have to happen. So they have a lot of investments, but then they have to monetize them, right? I think companies like I think like ChatGPT or others are like still yet to prove the level of monetization, right? I mean, they're building great models, but the question is Can they, can they monetize that valuations and other things that they're going after?

[00:44:20] Right. And it's, it's yet to be proven. I mean, it can, I mean, it can, but I think it's, it's a little bit, there's a lot to be played out in that industry. Right. Uh, because it's a very capital intensive, very intensive industry because everybody else will sort of take a quick free ride, right? So Brian, thanks to open source, I think open source is going to be the biggest winner.

[00:44:43] Jeff Roster: Okay. So Brian, uh, man, you know, obviously a lot of conversation around AI, um, right up to the white house, um, and, and their AI task force, which I just saw some notifications on, I haven't really dug into that yet. I'm not sure they're ready to really present to the public, but, um, you know, obviously you're at the UN, [00:45:00] uh, so white house, UN, every, you know, every.

[00:45:03] Every consultancy on the planet's talking about AI. What should our, our listeners, what should retailers be doing? Let's say it's Saturday, you know, we're recording this Saturday night, um, on Monday morning with, uh, what's, what's their strategy

Closing Thoughts and Next Steps for Retailers

[00:45:17] Brian Sathianathan: ?

[00:45:17] See, I think what, if you're a senior retail leader, and I would say the same thing as of what I said previously, is that

First is look at your organization and look at use cases, where can you actually apply it?

[00:45:28] So that would be the first one, right?

Second is. Out of these use cases, where can I build IP or an IP mode? So separate those use cases where you can build an IP mode and make. Large investments in those places where you can build an IP model, right? Where you can train your own large language models or like not you don't need to build your model from scratch But you can fine tune and build models with your data.

[00:45:52] So that figure out where where is the IP? Where do I use to point where do I need to use point solution right make that determination, right? [00:46:00] Third I think what you should look at is that how do we How do we, um, how do we deploy and scale some of these things and how do we actually, uh, position this, uh, to, to our user community?

[00:46:15] Because there's a lot of stuff happening in AI, so the internal messaging within your organization as well as external messaging to your users and to your employees have to be thought through really well. So focus a little bit, work with your teams to focus on what the messaging positioning on this is.

[00:46:30] So those are the kind of the three things I would look at, right? And maybe the fourth thing, I think, along with not maybe the third thing along with the messaging is to do enough experiments very quickly and try to get some initial wins and some KPIs. Because in the retail world, uh, on even the banking world, I think KPIs are everything.

[00:46:47] You want to be able to show some level of numbers. Uh, because, because I think unlike other technology trends that have passed through, AI is here to stay. It's not passing through, uh, and, and if you don't get involved with it, you know, [00:47:00] you're going to be behind and significantly behind. Right. Um, so that's, that's what I would think about it.

[00:47:06] All right.

[00:47:07] Jeff Roster: Sounds good. Um, lots more around AI, I think going forward, certainly as we, as we begin to build into, uh, NRF and, and beyond. So, uh, look forward to ongoing conversations. Welcome back home, Brian. Uh, our, uh, our little hometown here was, it missed you being on the road so much. So I'm looking forward to really getting back into a rhythm and, um, unpacking what's happening in the, in the startup community and the innovation space in the retail industry.

[00:47:32] Brian Sathianathan: Thank you, Jeff. . Take care. Bye.