Why AI Fails: The Hidden Data Gap Killing ROI

Welcome to Applied AI Australia. I'm Ramon Rodriguez and here's the truth. Ignoring AI is probably the fastest way to fall behind. This show cuts through the noise and delivers only on what matters, growth, margins, and time. Each week, I translate AI into practical outcomes leaders can actually use. Today for our practice series, I'm with Samir Gajani, data scientist and AI subject matter expert. He's currently director of PWC Australia, though his views today are personal and not on behalf of PWC. Australian enterprises are spending an average of $28 million a year into AI. Yet 72% are claiming not to see measurable ROI. Today, we're going to unpack the missing layer that's costing executive boards and organizational leaders millions in wasted AI investment every year. Now, we're going to talk about what that means and how to apply it into your business in the next 48 hours. Let's jump into it. I'm Samir. I am Moroccan French. I came to Australia 7 years ago. Um, and you know, most of my experience and uh interest is about science. I love science and innovation. So, I naturally went to data science for most of my career. I did a lot of that. And then when AI came in, it was the first time you can have some models that are really really smart and operate on language and you can make really really interesting things out of it. So I naturally went into that. Um I've been at PWC. I'm a director at PWC at the moment but today uh I'm talking as myself in my opinion. Thank you very much for that. Love to get your take. There's obviously a lot of hype going on in the world of AI at the moment. What's real, what's not, but yeah. What are you seeing firsthand? It's very exciting first of all lots of things going on lots of new models coming and you look at the benchmarks and it's a bus in PhD level at this at that so it's really really exciting times in general and all of us can see it on the news now the thing is reality comes in and people want to do things people want to automate things they want to create new value they want to build an amazing application that does something extremely smart and what happens usually is that uh there's a lot of focus on the models, there's lots of focus on the technology and very often people underestimate the data component. So today talk about the data components. I think enterprises and organizations would have known that data is extremely important but in AI it's obviously to another level. Yeah, there is a paradox with with data. At the same time, there is a lot of it. A lot way too much more data than you can make sense of and more data that you can get wisdom and knowledge from. That that's the first side of the paradox. Open any data warehouse, any data lake, you will find lots of tables, lots of different logs, transactions, product, product data, lots of sometimes millions, billions of transactions and logs and events and touch points. If you have if you're managing customers and then you look at the traditional BI and data science, it takes all this data and it gives you a little slice of insight, something useful. So much effort to go from a lot of data to a bit of insight. And at the same time, there's not enough of it because lots of meaningful data points are missing. Ask any new joiner to a company. Ask them how many weeks or months it took them to get their head around the business. It will be quite a bit. And then ask them what helped you most. Was it having coffee with people talking shadowing or reading some really good documentation data that explains to every often it's actually the warmer it's things are not written at the same time lots of data but not meaningful that the key lots of data not immediately meaningful unless you process it hence we have all these jobs for the last 20 years of DI data science analytic etc and at the same time not enough because lots of missing meaningful at that point. So it creates this vacuum of some meaningful things and mentions and this one we paid usually people pay quite massively when they try to build an aentic solution and they say okay one of our inputs will be the documentation we have lots of trading guides and documentation we see them yeah but too bad you were not updated the network with the reality you know and that's across the board it's very general so with the AI implementations that you're doing what the data are showing at McKenzie in many data sources is so many projects and implementations are failing. Do you think a part of that could be due to poor data quality? If we take a step back, companies like an AI entropy are able to provide this very low cost intelligence as a service reasoning engine. It's a reasoning engine. It doesn't have much context because it doesn't render your context or mine. It's in general. So this reasoning engine and then the other piece of the puzzle is your context. If you give context to intelligence, you get magic. You get amazing things going on. And the bottleneck is the context because how do you pass context to this new intelligence, this new form of intelligence very different from everything we've ever seen. How do you pass that context? And we can call this job knowledge engineering. Not data engineering that you've been hearing about for years and years. It's knowledge engineer. context could be just as big if not bigger than actually the prompt that is going in. So listen, can you unpack that a little bit? Yeah, this is a lot. There's a lot here and that's a bottle when we said that is not meaningful. Most of the data is not meaningful. It's raw. In fact, we need to make it meaningful. Someone needs someone actually it's a team of people multi-disiplinary need to take this and bring it as a context to AI. So this is what we can call knowledge engineering. And let's unpack it. It has at least four different four different personas need to work together. You have the data engineer of course. Then we have the theme knowledge because we need to know what we're talking about. It's goal oriented. Knowledge is really goal oriented. Then you have the AI engineer technology and AI engineer because this knowledge will be fed to AI. So if you don't have AI in mind, how AI is working, it will not be a useful knowledge base because when you do this at scale, you need to manage relationships in data points. And the science of relationships is is is not an easy one. And the fifth element is they need to have the proper time and uh and sponsorship to actually take time to do knowledge engineering. You open a room, you find these four people working together. What are you doing? We're doing knowledge engineering. If you don't believe in it, you tell them go do some real work. But there is awareness. Awareness is very important about there is this missing link data AI something in between. We need to feed this context to AI. This is knowledge engineering and it's not easy. It's art and science. We just need to be aware about it and put the right work on it. So there's uh obviously data and you're saying knowledge engineering is the gap in between that and the LLM. Yeah. Exactly. Exactly. Because this knowledge this knowledge layer will be binding structured data unstructured data. There are many pieces of technology to put together but there is you cannot avoid the thinking and putting people around the table to as a task force to build this again it's a new it's a new skill set. So you're saying data and knowledge engineering are two separate things that work hand in hand. They work hand in hand along with knowledge along with technology of course and AI. If we had to put it down to the key things to a successful implementation of AI strategy, you've obviously got data. Now we've added uh knowledge engineering into the mix, what are some of the other things that you think are critical to strategy and implementation that you see in AI today? So this one is is the one that is a bit under underlook. How do you redefine it value? So this is a top down. Then there is enablement. Give your company the right technology, give your company the right skills and the right knowledge engineering, right? And then there is this bottom up. You need people to just go and do things. You give them the right knowledge. You know, let's let's talk about those four things, right? We said that theme knowledge etc. lots ofmemes if they're digital, they will use AI. So you have theme plus AI equal c-pilot magic equal GP enterprise or they will do some really cool stuff with these two skills. Then you have data and AI people they will let's build agentic solutions they will build stuff but they don't have the SM knowledge hence they need to combine those four same time if you have this enablement with knowledge B with knowledge engineering okay it will look differently from company to company then you enable power users and super users and ultimately every user they will plug the compiler to the knowledge base and things will work it's too wrong it was an integration challenge you can still plug your GB enterprise or copilot a bunch of data but it's too rule remember it's not meaningful I see it overlooked so so much but I think to the knowledge engineering and uh the data piece so many times you hear about AI not working and solutions not working but garbage in garbage out if it hasn't got the right context and data you know what really do we expect I think if you put it into basic terms it's like sometimes when you don't give the AI really any information to go off a very bad prompt and you get a horrible response in turn and people say chat GBT doesn't work right you know it's like asking a friend to give you advice on something but you don't tell them what the problem you're actually facing is it's sort of crazy yes I'm like this that's very interesting listen so prompting for me is management are you a good manager prompting means you lay the context not even talking about knowledge in the United States you lay the context you give what you want ask and you give guidance you think should I should they micromanage a little bit or not and knowledge engineering is not do I just give this file over there do I give them access to that database if you do all if you do everything it's not meaningful it would get lost remember how when we said that it doesn't fit in context the context is actually it's small that's a paradox it looks big but it's actually quite small compared to all the data out there and the solution is knowledge engineering so if we talk about knowledge engineering and good data how does an organization today go about doing that what's your advice yeah Saman so first of We need to be aware that there is a gap. But that's already a really good one being aware because otherwise we keep doing use cases and and saying okay the use case is in charge of solving the the knowledge problem. They will do it to some extent but ideally we have a strong knowledge engineering them in a transverse manner. It's being aware, putting the right skills around the table and having a bit of you know being goal oriented having some goals in mind like what are the big things that my company needs to for example we need to become much better with our customers at doing proactive customer experience. We need some big tickets like some big big objective because then we can walk backwards and say if that's our aim we want to become proactive with our customers then we need to look at all the touch points we need to anticipate they're unhappy happy but wait today we had two million transactions what did we learn what kind of wisdom we got today from from all our data points from from all the logs and the transactions that we got ah we didn't get any wisdom what's going on we have two dashboards over there okay our knowledge engineering is to be better I need to Well, when you know when a customer makes a transaction, you can write an entire article. It's a story about it. They made a transaction to buy this at that time. This is their persona. That was the context. It was a Friday morning. Uh it was summer and the day before they actually did something very different. And uh you know, you can write an entire context and story just around that transaction this time that you see over there in the transaction table. And this doesn't come by itself if you don't have a proactive knowledge engineering activity that's going on and looking at and connects in the dots. There's lots of connects in the dots as well. I see a lot of implementations and they actually don't even know what they're trying to solve and they're not even measuring it. But there's so many other things that to your point you could tell a story with and actually build out your knowledge engineering that I think so many of us would overlook. Yeah. What are the steps to fixing this? Yeah, I think now we need a bit of we need to avoid this tunel where we do some work and it's a bit buried. But we need some we need to walk backwards from some big tickets, some big objectives as I said the customer experience you know the proactive one or it can be for example um you know increasing safety I want increase safety we need some big tickets like this and then we need to walk backward and say if we were infinitely if we have infinite intelligence that's very important if we have infinite intelligence and heavy lift team to think and reason how can our data uh best serve our goal which is using AI to go from data to knowledge along with those four people that I mentioned before if we they they need to use AI. So step one for you was awareness obviously acknowledging that's a problem. So what's step two? So step two so that this knowledge engineering is a bit oriented with the northstar is to walk backwards. What are the big things that this company wants to solve or get better at? It can be some very specific safety um domain. It can be something about efficiency or it can be proactive customer experience. We want to become super proactive. We don't want people to complain. They want to go way before that and raise their experience. So what are those big tickets? That's step two. Step three is the task force. Yeah. The so that we walk backwards. Okay. So what I'm hearing is step two is sort of reverse engineering from their goal or nstar is Yeah. Exactly. What's the purpose? What are we looking for? And then there is um some sort of data discovery. We need to know which data we have. So companies have different levels of maturity discovery mapping before even starting this knowledge engineering. That's step three. Um then there is something around um data remediation and quality because this data will not always be clean. Yeah. And here we can use AI again. AI is an accelerator for everything. So you will find is my because I might have multiple systems. I might have low and poor data quality. But it was okay so far because it was we're just storeing it. But now we want to use AI to do some really good stuff and we need this knowledge and so now we need to clean our data. But good news they can use AI as a heavy lifting engine to do it. I think most people get lazy. Step four, definitely in terms of quality because I think a lot of organizations are going to be jumping from straight from I've got some data to I'm using AI to hopefully generate business outcomes out of it, but the in between the data and the engineering is absolutely off. Can you unpack step three and four a little bit? How do you uncover what data you've got and what defines good quality data? How do you acquire good data? Yeah. Yeah. Okay, that is a great question. So very often um people have data cataloges um and but it's a bit uh descriptive. So we have this data over there. We have actually years and years of data engineering and data cleaning. So lots of organizations already have make some effort to clean the data to some extent. Now the good news is not only it's rewarding to have better data right now with AI because then you can solve the bottleneck. Before it was okay to have an imperfect data because by the end of the day the output is you will have dashboards you will have a bit some ML algorithms but they will not make or break they don't multiply your efficiency by 10. Now with AI you leave in value in the table there is a massive opportunity to 10x everything you do in terms of quality. So now there is an incentive to go there and say I want to get the best of my data right now. It's not it's not acceptable anymore. So that's the urgency and then um it's really hard to to have high quality data because you have multiple systems you have you need to triangulate you need to you need to connect just imagine you have multiple products a very simple example and these products they have different names but sometimes different names mean the same before AI it was hard to solve this especially the syntax is quite different but the semantic is the same but now with AI it can tell you these are same trucks or these are the same products and they can group them together. This is data remediation AI powered. I recently had an experience where I took over a new brand within the organization and uh they for the the product they had six different names that were just legacy and no one could understand what it was and previously that would have taken months of trying to figure out but it's such a common paradox that businesses face. So before we get to AI, right? So I'm just looking at this framework. There is four to five steps that organizations need to be considering before they're even getting to the AI stage. Yeah. Okay. You know of using AI effectively. Is that right? So there is a paradox here because I keep saying we need to do these steps before aantic AI and then I say in the middle of these steps you can use AI. Okay. It's on purpose. It's it's true because you can use AI very from a technical point of view. You can say I want to use AI just for that equality so that I can build my knowledge engineering layer. And once you do this, you have this enablement that will enable maybe 20 different use cases that will happen. Not to think anymore about this, but they will just think about the goal orientation. They will think about what's the user experience, which process I'm solving. By the way, I have this amazing knowledge layer I can plug into. And then use cases start to become very successful. So when I say this is a prerec for AI, I mean AI use cases, verticals. But then those people who are going to do this heavy lifting, it's a lot of work. It's very very contextual work. They need to use very custom and specialized AI but from a technical point of view I'm using AI to do discovery of my data. I'm using AI to do data remediation. This is a oneot or I mean it's not vertically aligned to use case. It's really for your data. And then if we move on the steps there is the task force that comes in and build the knowledge layer. So here you need to build different types of knowledge bases. It's not anymore just SQL tables or data warehouses or even it's not anymore a data lake where you can just throw in data. Now you need some a layer of almost wisdom and knowledge and it can be a graph with connections. You know, instead of just saying I have a transaction over there, I say this type of customer bought this product and uh they're related to this other customer who like the same product and this product is actually very similar to this one. And you build a very big graph of connections and other custom ways. Some of them we cannot even talk about it here because they're way too custom. Hence, you need the right people around the table with these steps now. So for the CEO, they've clearly identified data engineering, knowledge engineering is a problem with their AI strategy. How does someone approach fixing this problem? So look, this is new. Everyone is learning on the way. So it's important to partner so that you have different perspectives and augment yourself as an organization would be my my short answer. Now over time, it's a very strategic skill and capability. So you need to own start owning it. So I think it's a balance between the two. But it depends as well which organization you are, what kind of skills you have. It's strategic. So it's important to somehow keep the the capability and the secret source. But the end of the day, you know, you have all these models coming and the data is your only real asset. It's your data. And to your point, that is something that no one can take from you and it is really important, but we often don't treat it like that. Yeah, completely agree. But I think we failed to remember that that is wisdom and uh and you know so far it was okay not not to get the best out of it just because it's too much effort why I'm going to recruit how many data scientists and insights people how many dashboards I'm going to do and even when I have these insights how do I action them I'm very smart in my boardroom I have all I know what to do for the next quarter but then how does it impact the dayto-day transaction this was there was a big gap but Now with AI you can actually scale all these smart actions. All these smart actions you can go scale them at transaction level. This is what the Jansi is doing. It's not anymore I'm smart every quarter but I'm smart every minute whenever there is a transaction because the cost of intelligence goes down and you can bring you can bridge the gap between data and software engineering and you can bring the this intelligence to the transaction level. So the incentive to crack data, to crack insights is much higher than before because the actionability now is much much much higher. Cuz if we think about so many decisions businesses make sometimes they're new ideas and uh new executions that have never been done before. But if we're honest, there's so much that organizations do in terms of transformation decisions that is essentially doing the same thing over again, maybe slightly different. And we tried to put a whole new strategic approach on it. And often to forget that sometimes the best learning you can take is what's gone wrong in the past and how things have actually worked out. But often we fail to a remember to do that and b don't have the accurate data to actually lean on because it turns into one of your biggest competitive advantages. I think of recently I had an employee leave. They've been with the organization for quite a long time, 20 something years. And when I was hiring for this role, it's an extremely technical role. So it was hard to backfill it. And something that I did was all of this employees emails, files, spreadsheets, work she' done over the last 20 years. I remember asking her, "Can you please get all of that and upload it into Google Drive?" She was horrified why I was doing that, formatting it, formulating it, and something as simple as uploading it into Notebook LM. when I was able to onboard the newer person and you know upskilling them for that role would normally take six odd months I got them to full capability and their team within six to seven weeks because whenever they had a question they can refer to what did someone do on the 7th of January 2001 there you go bang right so that's a simple term a practical term that it works but where it goes wrong is if you don't clean the data you're going to spend more time trying to figure right what's what and cleaning up the AI slop then actually getting value out of it. So I great example that's really good. You did actually your knowledge engineering because you started from that data that that was the brilliance. Now imagine this at scale. How do you need at scale? You're exactly right and uh it can be applied to anyone because I said it was a highly technical role. It's a bit of a lie to be honest but the role it was for was an executive assistant and I can argue with anyone that is a highly technical role because it depends who you're working for 100%. But a scale at an organization level what does that look like? Yeah, look look it's uh as we said right there are those those steps to happen and there is this bottom up approach to mix with top down as well because you know when you have the right uh you when you're walking backwards you at least you're sure that you're going to the right direction. So this top down is important but then by the end of the day in your story it's your curiosity your excitement that that led to doing something interesting. You need to, you know, organizations need to mind the those positive energies and to get people who are curious to share with others because we want to uplift everyone. By the end of the day, this is all going the sky is the limit. There's no everything will lead to better quality, better experience, new innovations. There's no limit. So, it's it's a positive it's a positive energy for everyone. But we need to mine those you know those initiatives and and and make sure that we share with others and everyone is uplifted and then scale them. How do we scale them? The steps that we mentioned which are more systematic. It's not anymore one person, one individual being smart and doing things. It's how do we do it at at system level to have a big system. And the problem with systems is that it's not linear complexity. When you have 10 items, it's hundred combinations. When you have a thousand, it's a million. So, it needs a proper it needs a proper structure and a proper task force and new skills. You have a new job. Now, a knowledge engineer is a new job. Um, and it's very rewarding because the bottleneck will keep increasing. So, GPZ 5.1 just got released. Yeah. And you know uh while we speak and um it keeps happening three months you will have better there is an exponential curve of intelligence that that goes what's the exponential curve of your wisdom in your company and your knowledge the these two need to you need to run after after intelligence otherwise the gap will keep increasing and then it will make it much easier for an AI native company to come in and say I have no data it's okay because intelligence is way higher than contexting anyway and people with text they're not using it. So to avoid this it's important to to force this knowledge layer to to run as fast as possible at least not to diverge too much from the AI. Yeah. Yeah. No, I completely agree. And if we think about how that applies to a business, when a business is facing a problem, whether it's a merge, investment, uh, acquisition, whatever the problem is, right? A leader, a business leader should be able to theoretically go to AI, put their problem in, get a solution, and happy days. But the reality is that isn't the case. A large part of that is going to be due to the data and the context that we're giving in because a lot of the business challenges that organizations are facing someone has faced before somewhere in the world and I'm sure there is some of that that's going to be captured in a large language model. So I think there's so many problems and solutions that can be solved not just at a scientific level where we're curing cancer and challenging physics but when we think about business problems I think whenever I had a really big problem for my team is I built a thinking tank so all the hard knock problems that you can't solve take it to the thinking tank and let's see what AI can do to come up and solve it and we didn't have the right context engineering or knowledge engineering it was horrible it was sort of 20% But when we put the time up front to give it a brief and actually give it the right level of detail and input, it was chalk and cheese because it's a reasoning engine. We can use LLM as a as a knowledge base, but that's very risky because they've seen lots of things. They have some knowledge, but it's very risky. It's better to use them as a reasoning engine in which you plug your context, which is a very different thing to do. If you think about it, these models, they were trained on trillions of tokens. They have seen so many websites. It's almost when you ask them a question, you're pretty much telling them top of your mind with everything you've been dreaming and seeing. Can you tell me this? And then they go very very deep and they give you some information. It's it's a very it's a very strange way to to get knowledge out of the memory of that LLM. Whereas if you use it as a reasoning engine because it also learns to connect the dots between many things. It learns to to think to reason like humans do from language. It's not really thinking but it's language. So if you use it as a reasoning engine then you're you're taking much less risk. You're not anymore invoking some memories here and there. You're actually using the power of thinking but then you give it the context and you say don't even make things up from your memory what you learned. Just I'm going to stop you there. Hold on. That was absolute gold what you said. So reasoning just explain that part once more for the listeners. Yeah. Yeah of course. So if you look at those LLMs they don't just remember fact they actually looked at connections connecting the dots. They looked at all those within websites stack overflow. They looked at recipes they looked at news. They looked at conversations and the way they were trained it's it's a really good way of training where data was cleaned before then there was reinforcement learning. So they became a bit of reasoning. They can connect the dots and reason. That's why they beat lots of benchmarks. So by using them as reasoning engine and brain power, not as memory, you know, not a databases. No, don't use their memory, use their reasoning power and reasoning engine and then you plug them on your data. It's just a much much safer way to get value out of them because they don't make data out of anything. You just look at your context, your data, and they use the brain, you know, the reasoning engine to make sense of it, to reason, to connect the dots on your data. I think so many organizations are doing that backwards. Imagine you open your l your GPT or whatever and you say, I have this problem, how do I go about it? or I have a white page I want to write an email or so it will make it will give you some it will invoke uh some memories best practice something it's so it will give you something so that's one way to use it generates things then you have um other ways to use it where people try to get information they say I have this kind of car um and they try to get a number like what's the what's the value of this what kind of oil I should put on they try to invoke the memory of the LLM. This is a bit risky because it might it will get it right for things that are very common and very popular but sometimes you might hallucinate. This is where having incination happen when you try to get to invoke the memory of that LLM and then the safer way to use it is just to say don't try to give me any you know [clears throat] knowledge just look at my data that's your knowledge and your only knowledge and use your brain power not the brain but use your reasoning power to to make sense and to solve my problems or to clean my data for example and and this is a really good way to use it but all all of them are actually complimentary but some are more risky than others. Don't give it access to some data that's that it shouldn't see. It's just the old good habit of minimal access that we apply to people. We should apply to AI. Now, we've spoken about your five-step framework that they need to be aware of. But if they were going to take away two or three things from everything we've spoken about today, what would it be? Look, it's a bit hard because there are many things to consider at the same time. So probably the first one is uh awareness right being aware what's as an organization am I aware what they can do actually first of all what's the range of the possibilities that's one then where is my bottleneck so we say that very often the bottleneck is data knowledge so I'm passionate about this one it might be something else maybe imagine a company where the technology is not even enabled they're not on cloud or you might have all kind of bottlenecks or you don't have enough you cannot just rely on tech but very often that happens to actually a bottleneck uh and the ability to turn it into knowledge so first of all awareness about what AI can do then awareness about where is my b why don't I have AI doing amazing things for me right now why and you can find your bottlenecks and then once you have your technology you have your people and you have your data you need to connect the dots then I want people to become upskilled and empowered to use AI. I want data to be uplifted to become knowledge layer, wisdom layer, and then I can plug my AI and start to make things happen. And of course, you need your governance, you need your your expertise, and you need collaboration around the table. Um, but it's it's a pretty safe bet to map your data and to uplift it to a knowledge layer. I think it's a no regrets in any case and it will enable so it's a horizontal thing to do. It will enable so many things across the board. from everything you've spoken about we all hear about AI sometimes working sometimes not working but the key thing I every listener needs to be doing if you're going to take away anything is if you are implementing AI and you are using it currently and scaling it and it's not giving you the amazing things that Samir is describing you need to be asking yourself is why why isn't it doing those amazing things so often where we don't think about the basic questions that you've just asked Why? We're often too quick to blame the model, blame the technology, blame the people. But some retrospective thinking is what I think a lot of us need to be actually implementing and using. I mean pretty much always the bottleneck is not AI. The AI is leading the entire curve. Not only the bottleneck, not it's the complexity, right? It's not the bottleneck. Within three months, it will be less of a bottleneck and the gap between bottlenecks and this AI is bigger. So this is why there is a bit of sense of urgency to close those gaps. And as technology evolves and it gets more and more complicated and the bigger your investments get, the harder it is to untangle. If you're investing in AI and you've got implementation, why is it working? Why is it not working? That is the exact way you solve any business problem. If our capex is invested and we're down, we don't just blame the people and organization. Why isn't it working? And if you apply that to AI, I think you're going to come to your answer pretty quickly cuz it's not the technology. It's make making it work for you. It has been built for everyone. Everyone means no one in particular. So how do you make it work for you specifically and your context? It's your responsibility as an organization. Thanks for coming on Samir. And if anyone wants to get in contact with you, what's the best way? Is is that LinkedIn send you a connection request? Yeah. How do people find out about you? Yeah. LinkedIn. I love interacting, making new connections, always um up for a conversation for about AI. Look, now it's Friday evening having great pleasure talking to you, Ramon. Now remember, you can't control the speed of change, but you can control how quick you would gap. Thank you. My name's Ramon Rodriguez. This is Applied AI. Thanks for listening and I'll catch you soon.