AI Sprawl Is Real | Ep. 4 with Priya Udeshi

Dave Goyal: How can I use AI to make my life easier, both personally and professionally? And how can I just improve my own productivity? From conceptual idea to product, AI was a partner. Welcome to the Think AI podcast. Each week, we talk about the most exciting AI research, tools, case studies, and more. I'm your host, Dev Goyal, and I've been working behind the scene in data and AI for over 30 years. Whether you are an AI expert, skeptic, or something in between, this podcast is for you. Today, I have a guest I'm really excited about, Priya Odeshi. She's the chief of staff to the CIO, head of the IT PMO at MoGoDB, one of the most important data companies in the world. And she sits at the intersection of AI strategy, IT transformation, and enterprise execution. She's not just watching AI happen. She's in the room where it gets deployed, governed, scaled across the global organization. Priya, welcome to the show. Thank you very much, Dave. I am very happy to be here and excited to talk about a topic that is very top of mind for pretty much everyone in the tech industry. Great. So let's start with your story. How did you get involved with AI? And how has the evolution helped you, what you're doing today at MongoDB? Yeah, no, for sure. I'll give a little bit about just sort of my background. So I've been in the tech industry for about 17 years. I've kind of grew up through the... the channel of project program and portfolio management. So I think being at that, like you said, the intersection of strategy, execution, operational discipline, it's really cornerstone to driving PMO leadership, driving modern IT portfolio leadership. So that's been my primary space. I've been at Mongo now for three years, so I joined leading a technical program management function under our CIO's office. Again, driving technology enabled, AI powered strategic delivery across the enterprise. About seven months ago, officially assumed the chief of staff role. So again, kind of getting that inner corner office of the CIO vantage point for all things AI. You know, I would say that my AI journey, you this word AI has actually been in practice in our tech evolution for many, many years, LLM era. And so I would say, you know, prior to this sort of an official launch, you know, kind of pre-GBT era, AI was really a ⁓ wrapper for, I would say all things predictive analytics, machine learning, robotics process automation, right? So like RPA, you know, capabilities, any type of automation that you could put some predictive algorithmic, you know, a boundary around was considered AI at the time. And so, you know, a lot of the project initiatives that I had led or had folks on my team leading were really in that space. ChatGBT was launched just before I joined Mongo. Just as I was entering Mongo, we did, I would say, what most tech companies do. We started on our chat bot era. It was really focused on how do we... We didn't want to just roll out ChatGBT at scale across the enterprise. You have your... security, all of those things are very important. So we don't want to start pumping Mongo specific information or internal company data, of course, into those public LLMs. So we went on the journey of building our rag infrastructure, connecting a GBT style LLM to our data sources. We ended up building something called Mongo GBT, which is actually still in existence today and it is used across the enterprise. So that was really, I would say the learning phase of kind of the AI evolution. Fast forward to the agentic ⁓ phase, which is what I think we're in now, we kind of have two paths, at least at Mongo, enterprise agents and enterprise scale level agent agented workflows definitely is cornerstone to our AI strategy, looking at how we can improve the lives of sellers, right? Across our go-to-market teams, like how can we build a GTM, a kind of go-to-market agents to help sellers get everything they need to know about their accounts, their customers, right? At their fingertips ⁓ on the people. team side, how can we build agents to address employee cases, submitting simple task workflows. So there's definitely a good portion of our AI strategy that is focused on infusing AI into the daily workflows across all of our customers across the business, which again, working in IT, our customers are every employee of the company. But then we also are looking at how can we actually democratize the use of AI for everyone in the company? mean, we talked about this when we first met Dave. You and I should be able to create our own agents. You and I can create our own agents today, right? It doesn't, not, I mean, I'm an engineer by trade, but I don't sit behind a computer and code, you know, in my role today. It's a very different, you know, kind of vantage point, but how can I use AI? We're all thinking that. We're all asking that question every single day. How can I use AI to make my life easier, both personally and professionally? And how could I, you know, just improve my own productivity? So, you know, we, we, we do have, and that's a real thing that I think a lot of companies are facing is, you know, what we don't want happening is. similar to every, and I would say being intact, we've, you we talked about this the other day too, like the notion of shadow IT, right? There's, you know, when an enterprise solution is maybe not as quick or as fast or coming into your fingertips as quickly as you want, what do you end up doing? You go and buy it yourself or go and build a solution yourself or purchase a solution yourself. And we... will and are seeing that happen with AI as well, right? So if we don't keep up with the pace of innovation and speed that is necessary to get these agentic workflows and these AI infused into the day-to-day of every single employee, you'll see people creating their own personal accounts. you know, creating their own ⁓ agentic capabilities outside of maybe your company enterprise, you know, secured environments. And that's exactly what is counter to what we want. So those two streams, would say enterprise scale is definitely a stream. And then also low code, no code, you know, democratized agent creation is another avenue that we're looking at as well. No, that's really good to hear. And that's a pretty powerful story that you have. I want to follow up on one thing. I think the other day we talked about it. Before that, I want to ask you, I think I know the answer. I'm looking at three different buckets today. One is AI curious, who are just seeing chat GPT as everything. Then there are AI enthusiasts who are actually seeing the chat GPT. They are seeing the other world like Claude and so many other open source model, Kimi 2.5 and so on. And then they are AI skeptic where they think, AI touches anything, it's gonna, you know. bring our world to the knees. Where do you put yourself and why? Yeah, I mean, I, course, the majority of my, my, would say where I'm at is in the enthusiast bucket, but even being in the enthusiast bucket, there's a ladder, right? And there are so many runs on this ladder and I by no means will say I'm at the top run. I'm probably at the lower to mid run. Of course I am, you know, exploring, think just again, having that vantage point within Mongo and seeing how, you know, we are bringing to life a lot of these, you know, AI infused capabilities. But then on the personal side, had mentioned to you the other day, I got the itch to try to figure out how can I start building my own stuff. And I built a lovable website and some AI powered digital products. And I connected to a GitHub repo and I used Claude to help me do all of that, which I really was starting pretty fresh from being able to do that. So I think there's a ton of power and a ton of opportunity ahead. I do feel that even with the skeptics and maybe the folks that are still using chat GBT today, I believe that the evolution of where we're going to be with AI even a month from now is going to look vastly different. Like I have seen the timelines around the burst of this emerging technology is so much far more compressed than any other emerging tech that at least I've been privy to in the 15 plus years. We saw the cloud boom and then the SaaS kind of boom and all of those still. They took time. People are still on their cloud journey today, of migrating into ⁓ cloud-based environments. But with AI, what the landscape looked like, even with cloud, would say, what the landscape looked like just a month or a few months ago was vastly different than it is now. So yeah, I think I would put myself in the enthusiast bucket, but I still have a lot more to learn in that space. So I'm just excited to keep learning. No, you're humble. And I see more humble people will say, I'm learning. And that's a good thing. If ⁓ learning stops, because everything in your life, you have to learn, you know, even if you're a musician, you are an artist, you are a player, you're always picking something up from someone else. And that's the great thinking and the value that you have. So I appreciate that. I will also pick up on one more thing. I think the other day we talked about, and I could be wrong, Glean's agent framework that you mentioned. What is it and how do you see it fit? in what you're going to do. So Mongo has adopted Glean as our enterprise search platform. And that's actually the other, you we talked about enterprise scale agentic solutions. We talked about democratized low code, no code agentic solutions. Enterprise search was the other. I'll just give a little bit of a story before I kind of talk about the Glean agent platform. But I, having grown up in this PMO space in many of my parts of my career, part of my remit was to implement a project portfolio plan of record, right? Like, let's implement a PPM tool, a PPM system. We'll have one central, quote-unquote, database for project and program tracking assets. Like, I always think projects are data objects, and just like any data object, they have attributes associated with them, risks, issues, stakeholders, resources, all the things that can inform the, quote-unquote, health of a project or program. in many experiences of my tenure, I have had this remit of implement a system of record, implement a plan of record where we can pump all of our project data and run analytics and run data-driven insights, obtain data-driven insights off of them. And those have been successful. I've implemented multiple PPM tools. They've been successful. But even as a PMO leader, even with leading a team of whether they're project program, program managers, TPMs, as much as you can really drive to say that ⁓ this solution is to help you do your job. better, the solution is to help you make life easier. There's always this administrative component. In the day-to-day of running a technical program, in full transparency, the TPMs and the project teams are not living in a PPM tool. They're living in docs and sheets and unstructured data sources. That is how they're making decisions. That is how they're documenting driving decisions forward. They're not going and updating a PPM tool. What I found in any of those implementations is it becomes a little bit of an afterthought. Even when you do all the things to try to not make that the case, to try to have that be the forefront of driving program delivery, it still tends to be an afterthought. So what I see with Glean or with Enterprise Search capabilities is it solves that problem. Meet people where they are, right? People, and you and I, both have different ways that we're using tools to help us do our job. And yes, we should have Enterprise standard solutions, and we don't want to talk the other day about things like sprawl, I'm also, get the gift of leading our SaaS sprawl initiative also. And so yes, that is a real thing. You don't want to have this kind of hodgepodge of a bunch of different tools and different things that people are using. But what enterprise search does is it doesn't try to make this one size fits all of whether it's, know, I gave the use case about project data. There's so many other use cases for that. I'm able to just, Hey, hey, give me the latest on project one, two, three. And it's searching the network for all of the different data sources that contribute to that. And it's translating it using AI power and summarization to translate that into a meaningful output that I can then, again, whether I'm writing a QBR deck or giving a strategic update to our senior leadership team, that's the power of enterprise search. So that's where we started with Glean. ⁓ We have since kind of moved away from mandating the pumping of project information, at least from my team standpoint, into a singular tool. The next stage of what we really want to do with Glean, and there is a ⁓ dedicated team to focusing on this, is how can we release an agent moderation framework across the enterprise so that we can build, again, build our own agents directly on the Glean platform. So that is sort of our next phase of the evolution. That is beautiful. And it just reminded me of one of my own stories. I worked with a large healthcare organization, and they had several operating companies across the globe. And we were managing different programs. And every year we used to evaluate one of the PPM tools. So we had to do PMP and all that, but that's fine. But then we are looking at PPM tools and guess what? None of the tools were really solving the purpose. Like you mentioned, you don't live in the tool and all the PMS hated it. They didn't want it to go into the tool because they still have to see if you're entering the data there, somehow that should made into the people who are looking at it. They don't look at it either. So then you're spending time in meetings and then saying the same thing that you entered over there. And then they had a legit reason that, I don't want to update it because I'm already providing it in so many other ways. And I don't blame them. We started doing it in our organization from that point onwards. So in our consulting business, we are also using consultants and employees in India and other places. And those, you know, they have different cultures, as you already know, different time zones and things. So we started using different methods, you know, maybe WhatsApp, Teams chat, and started collecting it and started putting back into the system in the BPM tool itself. But it's more on the... No, just imagine if you had agents to do all that for you. Yeah. And those are happening as we speak, right? So they created their own unified box. I have my own unified box and those are guiding us what to do. Not that we don't want to connect human to human. But that connection should be about solving problems rather than providing updates. that's one of the things I personally hated most, that you're just sitting in a meeting where 10 people are providing updates, which you could probably read off and start taking actions. You are preaching to the choir. And I think about this a lot too. mean, again, being in the PMO space, that is, this is the evolution from outputs to outcome-driven leadership to value-driven leadership to, what is that focus on business outcomes and how that really underpins everything that we do. You know, I've always said the outputs, again, coming from the vantage point of program delivery, the outputs are important because they inform the outcomes. You can't drive and of course an outcome is like, you I don't want to say it's a, you you have a strategic priority and intended impact on the business that you want to have. There's a set of work that has to be done, bringing that down to earth and decomposing that, that strategic, you know, pie in the sky, you know, intended value outcome. You have to do a body of things and set up, yes, there's tasks and activities and things that have to be done to realize that outcome. But the value is in driving towards that outcome. If you can automate and build agents to focus on the outputs. To your point, then the shift in the conversation becomes very different. You're not talking about administrative tasks and activities and tracking progress against those because in an ideal world, you have agents doing that part for you. You're really the shift in the, whether it's a meeting, a report out, ⁓ an update from one of these reporting tools, you're able to look at a data-driven report that gives you decision velocity, right? That gives you what do I need to do next to actually drive progress towards this outcome? And that's where I think AI is going to really change how many roles, as we already know, how many roles operate today. And I think with that intended shift into focusing on the outcomes and accelerating decision velocity across the business. So Priya, thanks for that great insight. One of the things we also like to hear from our guest is that how what you do in your job and your passion helps and solve the real world problem. Some of the things we already talked about. We personally work in manufacturing and healthcare space, so we are focusing a lot more there. What's your experience looks like? What customers at MongoDB ask for in terms of solving their own original problems and how you and AI is helping them? Yeah, no, thanks for the question. So yeah, as I shared earlier, like my... passion and I think why you and I had such a great call even earlier this week to, you we're problem solvers. ⁓ And so I think the crux of like the basis of how I grew my career where I kind of evolved my passion for this space of IT strategy, IT transformation, program and portfolio leadership is rooted in that fundamental core principle of loving to solve problems, loving to help optimize. My husband kind of jokes and says I'm too type A for my own good. I can relate to that. have someone at home also, same thing here. It's ⁓ the relentless and ruthless focus on optimizing everything down from my own personal day-to-day to how we serve the business, again, working in IT, how we're driving solutions for our customers who are every employee of the business. How can we ensure that we are delivering in the most efficient and effective way with, again, that relentless and ruthless focus on driving strategic value for, for, ⁓ and customers. And I jokingly say like, you know, it's, it, it tracks that I grew up in this space of project and program management. am again, I'm type A, I'm a Virgo, whatever, you know, you want to say, ⁓ you know, kind of contributes to that particular persona, but it is, it's a persona for, I would say people that really thrive in driving efficiency. And I said, he says I'm too type A for my own good. also says I'm almost to a fault. if it's not done in the most efficient way and it's like anything even like cleaning the house or you know packing for it you know trip and things like that like I try to just optimize and make you know as efficient as possible anything that I do. So again and how that translates in from a professional standpoint again that that's where you know when I say I'm an AI enthusiast it's because when you talk about driving efficiency and optimizing how you do things like that is what AI is for. You know, so I am going to continue to be a forever learner about how can I use the tools and the resources available to me. Of course, security is very important. So I think, again, just being in, in IT, you know, I don't think I'm the person that's going to just throw my likeness all over a bunch of different tools and see which sticks. Like I do want to be thoughtful about what information I'm putting out into what tools that I'm using and how I'm using those tools and how my information is being used within those tools. So again, I think taking a thoughtful approach to just again, both professionally and personally, how can I use AI to make me better at my own job, at everything that I'm doing, at everything that I'm focusing on, and how can that translate into actual value that I'm driving, both for the enterprise as well as our customers. Now, that's a great talk. And one of the things you talked about being Taipei, ⁓ I'll it to a better place. What's the difference between the leadership think AI does versus what you actually do it on the ground because sometimes, and this is a good thing with type A people, sometimes they are driving it in the way it needs to be versus what is needed to be on the ground. So how do you use your personality type there to build better leadership within yourself and within your teams also? How I'm using AI specifically or? Yeah, you and AI both. Ultimately your AI will be your clone, right? I mean, at the end of it, It will be sort of an emotionless clone of what you do, but it will do it in a better way. So your personality would always reflect in what you do in AI. Absolutely. And I think we talked about this earlier this week too, even with AI is exactly that. It is a partner. It can be even on this personal project that I tried really just with the intention of professional development, learning more about AI so that I could be better in my day-to-day job. From idea... From conceptual idea to product, AI was a partner. The AI was in the passenger seat. I was in the driver's seat. And I think that always has to be the core. Yes, an AI is not going to be a replacement for your ideas. Your ideas are your own. There's a book that I read to my kids about similar. You want your ideas to be your own. That should never go away. An AI will never replace that. What AI can do is help polish your ideas, sharpen your thinking. give you a vantage point that maybe you didn't think about. But you also have to be mindful that that vantage point is based on whatever information could be scraped from the internet and surfaced into the LLM. So take it for what it is, but always feel grounded in your own ⁓ intuition, if that makes sense, in terms of what your approach to leadership is, what your approach to human empathy. Nothing replaces human experience. And that will always be the case. And I think having been in the industry now for almost 20 years, there's human experience that is valuable to me. It is part of my value. It is part of the absolute value of the value that I drive. And I will always use that to really influence what my approach to leadership is. AI is just going to help me sharpen that. It will never replace ⁓ me and my thinking. But what I will say is at the same time, coin or like at the same time, I do believe that, you know, a few years from now in every professional ⁓ role, AI will in some shape or form, even for the skeptics, be infused in their day to day into their workflows. And what, you know, whatever that shape or form could be will look different. But I do truly believe that with the accelerated, you know, timelines that we're seeing with the evolution of this emerging, ⁓ you know, capability, it's going to be in the hands and in the workflows of every single professional across the world, I think. You said it well, and you just mentioned skeptics. think everyone, whether they don't know it or do not know it today, AI is already there. YouTube is listening, and then it's showing you the videos that you like. You go to Insta, do a room scrolling, and then you see everything that you have done, and you're wondering why it shows, because you actually clicked on something. and is building the traces of it. So you are using AI or you'll, you are the consumer of AI without knowing it. So it's going to happen. One of the good stats I found, I forgot where I found it. So call it a conspiracy. We cannot define a factor, but 3.3 months, AI is doubling its capabilities and capacities both, right? Which means, and I did the math, it will be about a million times better than what it is today by 2030. And that's a very short amount of time, you know, just because how it is multiplying today. So I can only imagine what are the goods and what are the bads it can bring in. ⁓ On that note, what surprised you the most with AI? I mean, we've been both have been learning about it, implementing in our lives and what you didn't expect when you started implementing AI, meaning, you know, when you heard Chet GPT, I mean, we both have worked. ⁓ I started AI in 1996, 97, doing algorithms and things like that. A lot of things had to pre-plan. You had to have powerful computers. You have to figure out these machine learning algorithms and get a lot of testing data and everything. But now, that all has been gone, or at least people think it's gone. So what was your expectation and unexpected thing that you saw with AI today? Yeah. So I mean, compute, you already kind of, the compute aspect and how we can keep up and keep up with the pace of the needs of AI from a compute angle, I think is something that every infrastructure and technology company is having to grapple with. But another one that I would say, and I would say I wasn't, it's not that I am surprised that it happened, but maybe I wasn't, but I'll just share what it was. ⁓ You know, when we talked about like the chat GBT coming out and the mong, you we created our own Mongo GBT and then, you know, of course, Gemini, we have Gemini in our Google suite. have, you know, enterprise search. have this, this notion of sprawl, right? Agent sprawl is a real thing. And again, this is where I say, I can't say I didn't expect it because we saw it happen with fast sprawl. We saw it happen with tech sprawl. We, with process sprawl, with data sprawl. I mean, these are words that Every scaling technology company has to figure out. And again, working in this space of program management and portfolio leadership, I have across all of these emerging techs, have had to balance this. I've always had to play this balancing act and me and my team and the value that we deliver. I always say project management is a change management process. It was actually born from an IT GC. Like it's a controls process, right? It's how do we ensure the controlled delivery? of technology in a controlled fashion and a risk managed fashion. That was sort of at least my introduction into the space 15, 17 years ago. Project management was really kind of anchored on that being a very risk managed, risk focused ⁓ delivery approach. But as this evolution has happened, we talk about outputs and outcomes. You have to balance process and risk and governance, dare I say the word governance with speed, agility, value, know, value acceleration, innovation. So how do you balance those two things? That's always been something that, you ⁓ know, it depends on the company that you're in, the risk appetite, there's pace of innovation, there's pace of growth, but that has always been a fundamental balancing act that PMO leaders have had to play. And so I think it's no different, that kind of... That balancing act is table stakes for the nature of my role, but how it shows up, right? What level of process, what level of governance versus how quickly are we trying to deliver this thing? that's where I think it varies based on what you're trying to deliver. And I would say with AI, because it's moving so quickly, like the old frameworks for governance don't really. shape up anymore, right? Like I'll give you an example. In the PMO space, we build intake and prioritization frameworks where we're looking at value and complexity and how long is it going to take to deliver this thing. Just the time it takes to ask those five questions, you can spin up a solution with AI. So, you know, I will say the AI agent sprawl, AI sprawl is a real thing. There is such a thing as AI governance. Now, is it going to look the same way as traditional program governance that we've typically overlaid on some of those historical technology solutions that we've delivered? No. But I think that was, again, not an unexpected hurdle, just something that we are living the reality of that. And if we don't put some baseline guardrails in place for what is considered low code, no code democratization creation, that you don't need to run through some intake and prioritization. effort and you have the capability, those don't need to go to some central tracker, central, you know, rule up versus what is this other body of enterprise scale solutions, AI powered solutions that we're building that do need to follow because there is security implications. There is, you know, a mass change management effort, whatever the cases may be, you know, I think thinking of those two buckets in that way and thinking about AI governance as, you know, how can we ensure again, we're delivering in the most efficient and effective way, but not losing sight of whatever. the balance is needed for, know, based on the company and risk appetite and growth pace that we have to, you know, drive towards. Now, this is great conversation and it pointed me to a few things. I'm looking at unexpected hurdles, adapting AI, and that happens to every new technology. When you go to, let's say, an EV-based, you know, vehicles, then you have to have a charging issues. The blockchain came. and lot of fraudulent events started to happen, right? So every technology innovation will bring that risk and you have to bypass or surpass that hurdle. Now going back to governance and I'm kind of putting all big companies, tech companies on spot. Back then, you know, in 54, we came from a background where the software needs to be really stable and then only you start using it. And then we see Google's and Microsoft's and I want to pick one company here. ⁓ They started releasing beta versions. And if you relate our PMP head or project management head there, that's not a good solution because you are having your customer test it out. But it goes in both ways because some customers are hungry, so they want to get their hands ⁓ on it. But most solutions today are released in beta versions. They are not governed. So I would argue on AI side the same way. If those softwares can be released in beta, AI can be released in beta. Now, I understand the threat. Threat is much bigger. That's why some of the organizations are going into responsible AI. What's your take on that? And that relates to tech sprawl that you saying before. Yeah. I mean, I think you're 100 % correct. A beta is a, whether it's a POC, whether it's a pilot, right? Like, I mean, even internally, we roll out solutions to a beta group, right? And that is the group that it's testing from the standpoint of the user experience as opposed to the stability of the platform. So I do think those two things, beta doesn't mean put something out into a production use case that hasn't been tested, right? What beta means is it's beta, right? We're not GA yet. haven't, you general purpose this solution. We want to get user, it's really grounded in the user feedback and the user customer journeys. that are utilizing that specific solution, how is it actually helping them solve their problem? So I do think maybe thinking about beta from that standpoint is important. With AI, like code generation, that's what everybody wants. Everyone wants to get their hands on Cloud Code and Augment and Cursor, like all these big tools that are out there. And they're very powerful. Again, as a non-coder, I've dabbled with Cloud Code and I'm like, wow, like I can build again, digital products, build AI-powered. things. So again, that is becoming a democratized use case. I think it is something that companies are going to have to think about. Operational rigor, it has to be table stakes. Right? So we still want to look at things like percentage of defects post-production rollouts, even if those rollouts were 90 % AI generated. Right? There are still basic DevOps style checks and balances in place that we do want to make sure are put in place even with beta versions. ⁓ So yeah, so that is security, reliability, durability. I these are things that have to continue to be table stakes even with AI powered development. Well said. And one of the other point I also wanted to touch base on is that AI comes with this is the biggest hurdle with AI, which is hallucination. Yeah. How? What have you seen? How are you handling it? And what do you see in future needs to be fixed or will be fixed that you have a hope for? Yeah, hallucinations. And this is where it gets very much more technical that the engineers on my team within Mongo are super skilled at dealing with. I will say we saw, at least from an end user standpoint, more ⁓ hallucinations occurring with some of earlier models. That's where the right reasoning, the right embedding, the way it's configured really does matter. And that's where even with beta, you have to test that, right? You have to test these things even before you release something into beta. And even within beta, that's where it depends on what those workflows are. Are you building agentic capabilities in that solution that are really going to completely remove the human from the workflow. If so, those various scenarios of testing to really test, know, what are the hallucinations that we're getting and how are those going to impact, you know, the workflow and its journey, those have to be really tested. Now, if the agents are pulling information from various sources to give you an output that might not be actioned on, right, that's still the human that's actioning on that. that might be something you can test in a beta format, right? Like that you could go before you go GA, get a few users, maybe we did that with some of the internal ⁓ agent ⁓ solutions that we've created, ask it certain questions, I'll go to market one, like pull up information about a certain account, is that information the right information or is it hallucinating, right? So that's the kind of user feedback that we do wanna get because those users are the ones that are closest to the knowledge and the accuracy of that knowledge. But it is a very real thing. You see prompt libraries now all over LinkedIn. People are suggesting ways to ⁓ contain the amount of hallucinations that you can get. do think that a lot of the latest models, like with Opus, example, I haven't really had that problem. I've stored all my memory there. Of course, you have to review everything that an AI is generating for you, and it's been pretty spot on. So I've not gone what I've been. ⁓ haven't seen it as much there, but there are things that you can do in prompting that can contain the level of ⁓ drift, if you will, with the data output. Yeah. No, that's very true. And one of the things I remembered or what is happening, ⁓ I spend about 30 % of my time learning where AI is heading and where the world is heading in technology. ⁓ What you see a big shift today, so there are three things. One is prompt, prompt engineering. Second, which people are discounting a lot more is context engineering. And AI hallucinates a lot more because it's missing context, because it only has a million token, the best model. The other ones only add like about 300, 400 K in terms of token and token are loosely coupled like words, right? So only those many number of words it will remember. So then people started to evaluate and I built my own second brain using Obsidian. a story for another day, but it started to remember everything. But now that everything's still too much, so even that needs to be summarized. And when you summarize things, things gonna miss out, right? So context engineering is one thing I see as a big challenge and it is being solved. The third one I see, which is the biggest of all, and Elon Musk of the world and others are trying to solve it as well, infrastructure. I really see AI itself as an infrastructure. So recently I saw innovation called Tiny AI. Not a plug to them. I mean, I'm still curious to see what they're doing, but it's like a small hard drive and it has all the AI models of the world sitting in it and you don't have to pay for it. It's really powerful and you can plug into your Mac or any device. And so you are bringing your own AI. So that's an AI as an infrastructure. I see that as a big shift, right? That's what is coming in AI. The third that is coming, so along with context, I should have said that, is memory. So memory management is a big deal. These are the three things which is coming that I see. ⁓ you know, there are certain ⁓ evolution or evolution or innovation that is happening. So you see OpenClaw has been talked about. ⁓ NVIDIA jumped onto it, so they created NemoFlaw. Then Cloud Code has started producing a lot more features, such as remote dispatch and some of the other features. This is what I see coming. What do you see coming or what do you want to see it coming also is the question here. So from your own vantage point. Yeah, no, I think that's a great question. Just the memory aspect alone, I think is huge. Like I will be the first to say even in my current workflows, prior to ⁓ Claude being, ⁓ we do have Claude now rolled out ⁓ across the enterprise ⁓ for engineering at Mongo, but on a personal standpoint too, like I had a running GBT from when I was using chat GBT, I had a running chat because that's how it had quote unquote context or memory, right? But then I hit my chat limit. So then I would copy paste that chat and go to another chat, right? And then when I created my own personal Claude account, I copy pasted all of that and I stored it as memory. So I had to kind of do my own like stitching to create a starting point of my helping my brain, my brain partner, which is now what I do use is primarily Claude. But yeah, so I do feel like exactly what you said, context, context as infrastructure, that is your starting point. How do you create the right starting point? So you're hitting all the things that you said, you're minimizing hallucinations, you're starting from the right vantage point. It's indexing on the right, priorities too, because what I've also noticed again, just from a personal individual user, I'm giving it certain information. And then when I need help writing an output for something, it's not wrong, but it's prioritizing data points that I would have never, the indexing was, it was indexing on things that I would not have considered pulling into, whether it was a report, a memo, or an update, but it's because it didn't have the full context, right? So I 100 % agree, that is a place where I think there's huge opportunity. in all of the models, is A, we were starting with certain perplexity and things like that where you can, you know, has access to all the models, that's great, but using that to actually build a context layer infrastructure base that starts, that is your starting point for, you know, it has the memory, it has the full context, so you can truly use it as a comprehensive brain partner for your individual workflows, as opposed to trying to stitch together the way I did. Now this is a great solution and that's a good tip for the listeners as well that ⁓ you need to build your own context and I'm building in Obsidian, but you don't have to. You can simply put it in a Google Drive ⁓ Word document or something like that. But maintain it in AI. You can prompt it out so that whatever you talk to AI, it can start saving it as a transcript to it. then, or you can copy paste either way and then keep feeding back so that You know, there's a great example I use with my team also is that if you, what is context, right? So when you go to someone's house and you say, they'll say, what do want to eat? And you'll say anything. But then, then you have lactose intolerance, you're a vegetarian, you have certain other things, you never give any context there. And now you're expecting someone to guess your mind. That's what the prompt is. You're giving one line to the prompt and you expect that, that LLM model will guess it. It will guess it. but it has millions and billions and even trillions of data points for it. And it will pick up something randomly that might apply to you or maybe which is majorly being used on an average. And that's the hallucination in a nutshell. Exactly. And that's such a great example because you just talked about, the lactose intolerant on vegetarian. These are binary zeros and ones, but how about what have you eaten for the last 10 days? are so true. like these, that contextual historical trend analysis is also part of that. It should be part of the context in terms of what it's going to recommend to you to eat next. ⁓ And yeah, I would say on that, not using it as, at least I'm not using it as much anymore, but on the, were talking about projects and kind of the project tracking notebook LM for a long time was, ⁓ was a helpful. I love that. Yeah. I love that. Green G sheets and G drive all the way. So when you can, instead of. creating project folders with all the things, like just creating notebooks was super helpful because then I could ask it exactly. Like it had targeted context because we would use it for specific programs and specific strategic initiatives. So throw everything associated with that initiative into a notebook LM and make sure all of your artifacts are there so that it did have very targeted specific context. But yeah, I think, you know, it still required the steps of going and creating the notebook, ensuring that you're connecting all your data sources. I think the mass, like at least again on a personal level, like here's my Google Drive, here's my email, here's my calendar, here's all the chat history and all of the things that we've talked about. And that's really, like you said, the infrastructure layer of how to set context. If we can really optimize that, then it's just going to, it's going to bolster the effect, the productivity that, you know, gains that we are able to get from, from these tools. Great. No, I think you keep your eye on, on the things that how it is. saving the world or helping the world, whatever we want to call it. ⁓ What would you say to the CIOs to prepare for now? I'm not even talking about future, but now, because they might be lagging behind from where we are today. What should they be doing? And especially these are skeptics, right? So some people straight out reject AI and Rejection on anything is not good, right? It's like earthquake is coming and say nothing will happen in last 10 years or 20 years, nothing will happen, but it could destroy. So what would you do to prepare it? Similar thing for CIOs. It's a storm and earthquake coming to you already here. What should they be doing to get started? Definitely, if you have already have an AI strategy, ⁓ it's important. This is not something to sit back and see how it evolves. You need to have an AI strategy. You need to think about. agentic infrastructure. Is that something you want to build in-house? You want to go with one of the hyperscalers? Stitch together a number of point solutions. Like these architectural considerations need to be factored into your AI strategy because that is going to inform five versus build decisions. mean, everything.ai is out there. Every company is kind of trying to hit the gold rush in terms of the AI products available to enterprises to drive efficiencies in there. you know, in their ecosystem. Of course, the number one question that every CIO and every tech leader is asking is how are you using AI to, you know, help the enterprise? How are you using it to help the enterprise? Now having that strategy defined, having that, you know, those technical architecture decision points defined upfront is going, that can inform every buy, every build, every, you know, solution creation, every project or initiative, AI powered initiative that gets. prioritized. The fundamental of it is having that AI strategy aligned to both with, of course, at the CIO and technical leadership level, but even with the rest of the C-suite. And so that's what we did at Mongo the first couple of years, I would say, we're definitely innovation center. Let's get our hands wet, figure this thing out, build the Mongo GBT, build those other things without really a clear strategy. And this year, I feel, especially with our new CIO, we have a very clear strategy of how do we want to propel AI across the enterprise? And so I think that's really the most important thing. Think about AI governance. What does governance actually mean to your organization? Again, based on those things that we talked about, risk versus value, innovation and speed, how do we want to balance those two things? And having some semblance of, it doesn't have to be these bright, shiny, complex, robust frameworks. It's just a lightweight thing that you're going to use to assess how to admit AI initiatives and what your architectural approach is going to be to. solving those problems and building those AI solutions. Think about that now, define that now, because it's very quickly going to become too late. Yeah, it's on your head already and you just have to deal with it. Whether deal with adaption or deal with resistance, it's up to you. It's gonna hit you very hard. But AI is a friend, is what people should take out of this discussion right now. One of the things I also wanna pick... pick up on, you just mentioned that your organization is using cloud code. We are using it in a very siloed way because we are a small consulting shop. So I'm a little bit curious. Every strategy, you know, strategy has few components. You have strategy, you have strategic initiatives, then you have phases, projects, workflows, and then you have measurement, KPIs. How are you measuring or how the organization is measuring cloud code? ⁓ performing and comparing because that's what most CIOs and CTOs would do. They will compare with the old world. This is how our developers were performing. This is the KPI we had. What's the difference? Are you noticing anything? Is there anything in place that is giving you some differentiator? So we are in the maturity evolution. So I will say that we're building out what some of those KPIs should be at the minimum baseline, the percentage of AI assisted PRs. is something that we are going, we are starting to track now, right? Historically, we really didn't have that. How much of our code is generated purely from AI? And then what are those delivery like? What is the cycle time? How does the cycle time compare to what it was prior? So these are, we're building, in the process of especially now reestablishing our engineering productivity, developer productivity, metrics for success, our operational excellence metrics for success. And so these are some of the metrics that we're tracking. I think the story is still unfolding, right? Like time is now evolving to see how is this trend over time occurring? Are we seeing improvements in cycle time as a result of uptake in ⁓ code generated from AI? Are we seeing the complementary trend down in actual cycle time? Those insights are what we're working towards now. Okay. Now that's again a great insight and I love this conversation. know it will keep going ⁓ into a lot more details if we keep unfolding, but two questions I don't want to miss, which is coming to your own personal side. know, for me, three things are very important. ⁓ You know, leading with empathy, ⁓ motivations, you know, technology or solving business problems using technology. talked a lot about it. Leadership, we talked about it, but then there are some personal side that you mentioned. The children's book story, I have a 15 year old and he tried a lot of different things. He also uses AI. ⁓ I actually have a new company with him called AdmitSure. So while he's preparing for getting into different colleges, I said, why not you start building? So he, so we got funded by Google and he started seeing those powers. He got trained a little bit. And he saw goods and bads of AI and he can kind of coach to the people now. So I'm pretty proud of that. ⁓ You have a similar story there. Let's talk about it, the children's book story you had. I, so my husband and I both work in tech. So we are very tech forward, I think in this house. When we got our Google Gemini was first introduced, both I have a five year old and a seven year old, both girls. were having such a blast with asking Gemini, build me a picture of a princess riding a banana holding a hot dog. And it would, but it was a thing. So we would have some fun with that. ⁓ My seven-year-old is in second grade. They have Chromebooks this year. So inherently, the world is different. When we were growing up, my teachers wrote on chalkboards. We had notebooks and pencils and nothing. We didn't even have cell phones in the house at that point. It is inherently a vastly different world. We are very conscious about how we allow technology into their hands. mean, simply put, we don't even let them use their iPads or watch TV during the week. So there is that. We still are very, I would say, controlled and conservative when it comes to screen time and things like that. That being said, there is no reason why a seven-year-old can't enjoy the power of AI. And so the example that you were talking about the other day is my seven-year-old, she's a wonderful writer. She loves to write, she loves to read, she's super creative, she writes her own little books. ⁓ And so over Christmas break, she had written a book, a children's book, she even drew all the pictures for it. The story is about ⁓ little two moons, a mommy moon and a baby moon eating, they make a star cake. was very sweet story. She drew all the pictures for it. We sat together, we uploaded her pictures and her story into Gemini. asked it to basically create, her images as a starting point, but create them into children's book ready images, keep the same style of the pictures ⁓ and kind of round out, I would say, the language and that is from a grammatical standpoint. And if there's, you know, missing pieces from the sentencing structure, you know, fill those in. We did that together. I got her comfortable with giving it a prompt. She's also learning how to type in school. you know, prompting, ⁓ you know, Gemini specifically on, you know, I want to add this, I want to add that. And we built a children's book for her. And I'm in the process of getting it stitched together and printed and we'll read it every night before bed. And so yes, one could argue, well, is that hindering creativity? And I say, absolutely not. That is empowering her to think beyond the bounds of what she's able to put pen to paper on and giving her the tools to be able to prompt and to kind of find the way to take her own creation and create a product, an actual physical product out of it is the journey that I'm helping. to trying to instill in her at this age. But yeah, so I think it's here to stay. ⁓ think AI, using AI responsibly is super important. And again, even for the children, this is the, I think, the extent of what we want to be able to showcase to her at this time and have her develop her own, ⁓ hopefully, passion for technology the way her parents have. Yeah, no, that is great. And you're encouraging that. That's really good. A good example I also use about technology. So if you're a furniture maker, you use advanced tools that does not take away or scale your creativity. Same as when you use calculator when we're studying, you you could do calculation on paper, fine. But that's a repeatable task that you could let a calculator do it. Same thing happened with computers. So you have adopted technology all throughout. Why resistance on AI? It's the same thing. It'll bring a good and bad in everything, but if you adopt a good thing and you look at the good things, it's going to help you. And I definitely encourage kids. I in fact have a movement for ⁓ disabled kids where I'm providing free education on data and I have a school community on that too. And I really want to encourage everyone to learn AI. ⁓ Learn with the caution that it's going to bring a lot of risk to you. So like anything else, you use a digital card online, putting on Amazon. same risk you have, but in a different scale and proportion. So you need to be... Yeah, about those things and know how to respond if and when a risk gets triggered as opposed to ignoring it and acting like it's not gonna happen. Yeah, and everyone needs to learn and educate themselves and educate others. And that's the key for any technology, including AI. ⁓ One closing thought and question, and this I ask everyone. ⁓ tell your past self, know, you're 16 year or 18 year or 21, whenever you were there. And I have a lot of story about myself too. was, you know, my father used to pick me up, put me in school. So I got my wings when I started wearing braces. So I didn't want it to study so much. was good in studies, but I wanted to travel. went into music and I started working in different places. But one thing I missed at that time is I was not too reflective and I was ignoring what is happening around me and not enjoying. So I tell my past self that I should be enjoying that journey that has happening. A lot of great things has happened, but I kind of keep moving on. So what would be your thing? That's such a great question, Dave. Thanks for asking it. think. always trust your gut and trust, have trust and faith in your intuition is what I probably would have told myself because you know when I and I won't share my age I'm well-liked to be a Ferdberg. Well in any case I'm older so that we'll settle down to that. ⁓ I remember it like it's yesterday almost that's that's the irony of the whole situation it was not anytime recent but I do remember it as if it was yesterday and Just with this, like we're talking about AI and technology, you my husband and I, were recently saying too, we are the last generation that remembers a life before all of this. Like internet, we had a TV in our house, that was it. And it was this big boxy thing. And I had a big computer. When I grew up, when I was my daughter's age, no iPads, no cell phones, no any of that. So, and we are the last generation, I think, to like have that, preserve that memory. ⁓ By the time I was 16, of course, I think I had a very big looking cell phone. But yeah, think we're such sponges for knowledge. And that's a good thing. But I do feel like just reminding myself that your intuition even, especially with AI coming in and all of these data is an abundance today and is quite difficult. to navigate through all of it and know who you are and know what it is you want. We were talking about the other day, I'm getting influenced. You clicked on maybe one AI arbitrage ad on my Instagram and now I'm getting 15 of them. Are your AI consulting business, are you know, AI arbitrage business? So, know, yes, that is happening. We're all getting influenced by, you know, the data around us and the channels around us. And those are so much more relevant today than they were, you know, when we were, you know, 16 years old. ⁓ so I think, you know, just the, I would have told myself back then, what I will tell my children, you know, because they are kind of the mirror of what I wish I could have told my, my, you 16 year old self is just trust your intuition lead, you know, really truly believe in your, you know, it's, it's, it's almost like manifestation mindset. Like think about your gut is what's going to tell you what your true path is and always trust that intuition lead that path with intention. Don't just let the world influence you and bounce you around without really having asking yourself your question intentionally, what is the path that I want to go on in my life? And just trusting that. No, this is so beautiful. And I also relate to that. And I just thought these are forums that I look at. So my doom scrolling is about mindset, motivation, ⁓ music. And there was one more. And so there are forums that I constantly look at and see. So I'm welcoming all that. AI sending those stuff to music, right? So it's sending it to me and I love seeing those things and you know, I love connecting with people. So thank you so much for connecting with me. You have been incredible. And where can people find you? I know you're on LinkedIn, but there's there where people should look at and find you. And I'll provide a link to people on my blog if that's okay. That would be wonderful. I love, similar to you, I'm just excited to be in a space that has so much opportunity for learning. Thank you for reaching out and connecting. I'm so happy that we were able to meet and we have each other in our network now. And so yes, I'm available on LinkedIn. Feel free to share the profile. I started writing recently as well. So I have a sub stack and I've just been sharing insights that I'm passionate about. really, you know, I think entering a phase of my growth and my professional journey of, you know, continued learning. And yes, I want to continue learning from both experts, novices, people that are in the journey with me, whether it's AI, I have a very strong passion for music as well. So we share that in common. But yes, thank you for this, you know, wonderful dialogue and just looking forward to, you know, staying connected. Thank you, Priya. That's Priya Odeshi, Chief of Staff. to the CIO and head of IT PMO at MongoDB. If you're thinking about AI strategy in the enterprise, that is someone you want to follow. She just mentioned Substack. So do go ahead and follow her. Have a great day. Thank you. You have been listening to Think AI Podcast with Dave. Take one idea from this episode and turn it into action.