In this episode of “Data Chronicles,” host Amy Raygada interviews data expert Yiuliia Tkachova on data platform reliability. They cover best practices, common challenges, and emerging trends in maintaining robust and trustworthy data platforms. Whether you’re a seasoned professional or a curious enthusiast, this episode offers valuable insights into building reliable data infrastructures in our ever-changing tech world.
Speakers:
- Amy Raygada – CEO & Founder, Cosmodata Strategy & Management
- Yuliia Tkachova – Co-founder & CEO, Masthead Data
Watch the Replay
Read the Transcript
Speaker 1: 00:00 I’m going to go over some community announcements real quick. So I’m announce this last month, but I’m going to announce it again. So we have two new data, mesh Learning MVPs, Tom De Wolf and Kenda Ari, I’m sure that you all should know them. You see them on LinkedIn. They’re very prominent voices in the community and data mesh space. So welcome them as data mesh MVPs and our upcoming meetups. So on August 6th we will have a data mesh learning panel, data governance and data mesh, and that’s going to feature Kara and Hawkinson and then Kenda Ari and then also Andrew Sharp will be the moderator for that. So that’s on August 6th at 8:00 AM San Francisco time or 1700 central European time in Paris. And then of course every second Tuesday of the month we have the Data Missionary Roundtable and Amy will be there. So you can scan that QR code and that’ll just take you to our meetup page.
01:02 If you’re not already registered with our meetup group, you should, that way we’ll get notified of all these meetups, but then you can also register for that specific event or for those two events. Great. And then just letting you know, we’re going to post all of our talks to YouTube and actually we will be posting this conversation to Spotify as well. So I should actually post the link to Spotify, so I’ll do that at our next meetup, that way you all can get subscribed to that. And then of course, if you aren’t already, you should join our Slack group. This is a great way to communicate with people in the community. You can interact with your peers, ask questions, you can help answer questions that people have. So yeah, it’s just a good way to interact with the community and also you can learn a lot. And then of course go to the data mesh learning website@datameshlearning.com. You have hundreds of resources. We list all the events there, lots of use cases. So it’s another great resource in your data mesh journey if you’re trying to learn about data mesh. And that is all I have. So yeah, I’ll hand things over to Amy who will get us going.
Speaker 2: 02:19 Hi everyone. Thank you so much for joining today. As already Paul introduced me. My name is Amy Regata. I have working in software engineering in data for the last 16 years of my life. From software engineering to data management. I have seen everything from many angles and I have with me today on the first episode of Data Chronicles, the amazing Julia. She is the CEO and co-founder of Mass Hat Data basically is a data observability tool for GCP users, but Julia also has seven years of experience in data product management. She’s super passionate about building teams and she also has a cool podcast that you should check as well with Scott Herman Straight Data talks. So if you want something also like no BS and just honest conversation, you should go through it as well. It’s pretty cool. And you’re laughing because it’s true, right? And this is what it’s all about.
Speaker 3: 03:17 Some there.
Speaker 2: 03:18 Yeah, I was checking. So that was like, okay, I love it, I love it. And this one, these Data Chronicles is about the same. It’s just like a super honest chat about experiences, the pain, the gains, and how you can go through the different layers and different topics. So today we are focusing on data reliability on the data platform, which is super important nowadays and a lot of times it gets neglected. So let me shoot you with the first question Juliet. So how do you define data platform reliability and why do you think it’s crucial in our data driven world? And sorry, before you answer for the audience, if you have any questions just post it on the chat and we will ask it to Julia here during our conversation.
Speaker 3: 04:10 Yeah, thank you so much Henry. Just an amazing introduction and question. I’m just going to add 2 cents before answering your question because this is all based on my experience and from what we see when we work with our customers. So mass hat, we’re building a dig observability solution which includes understanding what is happening with clients, data tables, data pipelines, and the cost of their platform. So if you think about the analytics, we provide the analytics of the entire data platform by just using logs and metadata that also guarantees the full coverage of what is happening inside the data platform. And I have a privilege, let’s put it this way, to see what is happening in client’s data platform. The beauty of our solution is that we don’t need to access the data to understand what’s happening there and this create this comprehensive coverage where we see which tables are failing pipelines, how often it’s happening, who owns those pipelines.
05:22 And when I think about data platform reliability, I see that as descendant in every processes is going smoothly in a way that when something is failing, because in data it’s happening all the time, there is it’s topia to think that everything’s going to work in data not this is also comes from my experience. You need to make sure that everything is connected, that the access, the people have enough access to perform task or you can also see which people have access where they not supposed to have access. Basically data platform reliability comes with connectivity, smooth of processes. The platform delivers data from point A to point B without any anomalies or errors. I’m not sure if it’s, I didn’t came up with a straight definition about that, but suddenly when everything is working as expected, you can say that the platform is reliable. But the point is how do you know that everything is working as expected? And this is what I see from a lot of teams once we came in during discovery phase, when we talk to these teams, oh we have everything nailed down. I’m like, okay, and this is the conversation about the arrogance. I guess
Speaker 2: 07:06 People
Speaker 3: 07:06 Think
Speaker 2: 07:09 That’s true. I have data and it’s perfect and I can do everything I want, but they don’t take care of all the meaning in general of a data platform and how it’s to mix together. As we were speaking the other day, it’s not only the architecture part, it’s not only the data quality part, it’s not only the governance part, but it’s everything together. What it makes a data platform reliable, right?
Speaker 3: 07:38 Yeah, exactly. You cannot just check one table and say the data looks okay here. That means that the data platform is reliable. No, this is not how it works. The data platform is reliable if all the pipelines are executed within the expected timeframe. If none of the data pipelines are failing, if the all tables receive expected volume of data, there is no freshness issues. I mean the update of the tables and also the data being delivered to the dashboard or end user you envision without any delays in other problems down the stream when data modeling happening over wherever. And unfortunately when we talk about data platforms from what we see from all clients, there are 10,000 pipelines and there are 20,000 pipelines and more, honestly 20,000 pipelines is just a medium sized data platform for us. I mean from what we see from all clients just and how would you have mental capacity to know that everything is executed as expected and this is where the conversation we need to have with data engineers folks, are you sure that everything is happening as Univision Because in the end of the day there are so many unknowns that they didn’t even expect.
09:10 One of the use cases we had was a client, again, I might be saying I don’t want to come across as I’m breaking, I’m just saying what I saw from the clients, I’m not pitching the solution here. Yes, just to be no bs. So one of the use cases we had is the client wanted to, it was a lead data engineer who just was on board in the organization and it was sort of selling tables, which is a lot and he needed to make sense what is happening in there. So he needed the lineage and because we had the lineage here installed mass had started to look but also because we provided out of the box detection of errors that are happening with pipelines the next day he understood that the pipelines crucial ones that were covered with their internal monitorings were not performing as expected because their monitorings we not working. And that was a hilarious story because they were like, but we have the monitorings business crucial pipelines and they were not executed as we expect.
Speaker 2: 10:25 At least we had something because I have seen teams in bigger companies that they don’t even have monitoring. And one of my first questions when I arrive is like, okay, how do you know this is really working or not? And then you understand that also there is a lack of knowledge in the people who works in the data teams as well. And that’s one of the biggest problems as well because you don’t educate the people to grow with your platform and with your business goals and your data goals.
Speaker 3: 10:56 But is it possible, is it possible to educate those data people? What I see from today that data engineers and shortage and there is lots of root causes of that from not being able, I mean in general data teams are struggling to communicate the well they provide and I feel like when there is a layoffs, data teams are being on the top choice to cut. I mean what I’m trying to say there is so much data, so much sources data, people are a shortage of data people and organizations and it’s hard to keep up with the velocity. And I mean when I encounter encounter a data person who says I know that is happening in data platform we have, I’m like, okay, are you planning to go to vacation someday? Not even challenging him if he knows everything inside out because I don’t want to challenge somebody’s feelings. But this is a reason it’s impossible to keep up with the growth with changes and you need to have these monitoring to understand if the data platform is reliable or no, otherwise if you cannot measure it, you cannot know it, not even improve it, you cannot know it for sure.
Speaker 2: 12:24 I believe that you can still scale up these people, but when you have some automated tools on top of that who will give you out of the box this kind of solution to already check the monitoring, to check the quality and to check these kind of things, of course it’s a big help. If you are like a startup for example, and maybe you have just one or two data engineers that usually how it goes, they are not only data engineers, they are machine learning engineers, engineers, they’re everything in between and they don’t have a time and they’re severely born out. So this is a big help. But how would you think the change management should be in the minds of data management in general for the heads, for the directors of data, for them to understand the importance of this data platform reliability before it’s too late?
Speaker 3: 13:17 This is such a great question and this is where in converse data leaders, first of all, it’s good that you highlight that we need to talk about appropriate size of organization and its maturity on their data journey rate. First of all, if we’re talking about the startup that has 100 tables, certainly they don’t need any observability. It’s mentally and if they have a data person, they can mentally understand what is happening, what is data models, whatever, even up to 1000 tables depending on the skillset of that data person. So there is no need I I’m sure that that could be comprehended if there is a data person again. And also we need to understand what they’re doing with data. If they’re acting on data, if they’re putting it into production, they have, how do you say, if they have operations, daily operations, building data, they need to make sure that the data platform is reliable.
14:28 Otherwise they’re just collecting data to store it in data warehouse. I mean ideally they would need to check it maybe in a few years from now they will start to act on it. But literally folks you need to make a decision. What are you doing with data first? Okay, ideally yes, but nobody has the capacity even to act on it now to do simple models. So just let it go. The time will come to you and you will kind of, I just want to be realistic. Okay? There is no need to oversell it. Yeah, no need. But if we are talking about the organization, where is it Chief data officer, it means there is some level of maturity using data and there is data direct or whatever, like data engineering team, BI team there is kind of erroring everything. So we assume they’re already using and put data in action. So what we found out through my product discovery, and again with my conversation with different clients and prospect business users care about data quality, literally know what they need to know is those horses and these kind of things. If this figure in a dashboard is accurate, okay,
Speaker 2: 15:53 We agree on that
Speaker 3: 15:54 And this is what business, yeah, yeah, yeah, yeah. Business users but data engineering director, depending on level of their, how do you say, inner maturity, they start to care about the observability. This comes to connectivity of their data platform because you cannot achieve data quality in that part if your pipelines are not working correctly and business users don’t care if your pipelines are correctly connected and performing well. They literally have no idea how that influence them in the end of the day. And it’s hard to sell data observability solution in a way where signals about any anos or pipeline issues in your data warehouse. So data platform via, okay, we’ll ping you if your daily users are, it’s a different conversation and a different value proposition. And this is what I found out that it comes back to data directors or to data officers, what they want to achieve in their job.
Speaker 2: 17:20 I agree to disagree with you in a few things. I truly believe that data quality and data reliability is quite important. And of course business users, they don’t link the stuff because I have seen that in my current company. I see for example that the business users, they will like to have data quality, but they’re also a huge project going on. And basically from this project they think that we just plug and play is a hosa and they are just like boom. And then you get data flowing and it’s magic, right? Because that’s what business thinks. They don’t know the nitty gritty, the hard work, everything that needs to be put on to be able to provide this, these kind of correct information. But I’m a truly believer that you also need to have these difficult conversations and some way can get quite nasty I will say.
18:14 Because you need to really come and speak with the business and tell them, okay, you want also data quality, but can you give me business test cases? I would like to know what’s the deviation for these numbers so I can put a test out of it as well so you can get the right information. I need you to cooperate with me with more information so we can give you the best SLAs or SLOs in this case. And it’s something that also needs to come from the change of mentality, from the data directors, from the head of analytics, from the other management people that will push. I have a good example in one of the companies that I work in the past there was this team from monetization and there was thousand of versions of leads and revenue all over the company. Oh yeah, my favorite one.
19:04 Okay. Yeah. And there was some people doing it in Excel with different stuff, some people doing it, it’s from some other system in finance, you name it, right? It was all over the place, like four or five versions of it and no one really knew which one was the correct one. And one day we were supposed to, I mean we had to really push to the business users, to the monetization team to give us some information about how to calculate these metrics correctly first of all. And they were like, well actually we don’t even know. And I’m like, how come you didn’t even know? Okay, let us think about it. So we have to involve product there, right? Okay, product, how would they calculate leads in the right way or what kind of revenue shall we calculate for you? That makes sense. And then we had already other teams there in the mix.
19:56 It took some time, but I have some templates so I went through it, we speak, there was escalations because then the monetization team, why do we need to provide, you should know how to do it. And I’m like, okay, how am I supposed to know how you want to calculate a metric? And then you come to the data team say data is not good. So look at the right because we can try to have a lot of good observability, good metrics, good everything, but if you don’t have this cooperation with business, we are still getting not maybe the right numbers. So at the end for example, we finally got the nice metric, harmonize it, be able to show it, but we had to escalate even this theme so that their director had to push with them as well to be able to get the input from them to generate a valuable output out of data so we can put all the observability in place and make sure that we have the right stuff going on.
20:55 And the funniest stuff here is at some point because okay, we spread the word people were happy some other teams, because we were different business units want to participate on this kind of workshops, there was a presentation that was gave to the shareholders. I’m laughing already because I remember and we see the number of leads and revenue on this presentation, not our numbers. I don’t even know where they got it from. We don’t even know where it was coming. So imagine that we are still thinking, okay, there are more data sources that we don’t even know at this point. So I think also the discovery event, storming sessions with the teams to understand all these data sources little by little until you understand what exactly is out there, it’s crazy. And then I laugh because I didn’t want to cry at that moment. I was like not possible. And the numbers are so off, it makes no sense at all.
Speaker 3: 21:57 It’s definitely something painful when just after all the work was done to discover that the numbers are different. But you know what I just wrote down where you were talking, I think one of the problems when we’re dealing with business users is that as a data people when we’re dealing with business users, because I was in that role, that data quality, data reliability, whatever you call it, just the data is correct. It’s just implied by default. And business user have when pitch to them, okay, we need to do A, B, C, D, this is exactly what you’re told. Why do you ask me? You’re supposed to know it. You’re supposed to calculate it correctly.
22:49 And this is I think a big shift that have to happen. And they had or internal world whatever of business users that we don’t know your data, you work with it daily because of your domain expertise. You have some kind of expectations and knowledges while when you come to data people, you expect them to know what the numbers in every dashboard and have a genuine feeling like if it’s correct or no, no it’s not happening. And this is the point where we need to have this conversation to align our expectations. And what you’re saying to, it’s quite different, which you’re talking to me. I mean the point that you’re providing about aligning on metrics and how we calculate there is no other way we need to align the expectations. And sometimes this is a hard talk, I assume you had that in data contracts at your organization where you precisely wrote down what is the metric and how it’s calculated.
24:04 It’s interesting because when I was talking to Andrew Jones and so he shared with me, he said, okay, building a data platform, pulling data from different sources, pulling it in, data warehouse, architecting, all of that took us two years and that was started to add data contracts and that was started to align everything and just after that, so it’s mixed and it’s kind of mixed to me what you’re saying about we didn’t know that there are different sources coming in. Yeah, exactly. Maybe the project need to be done step by step. We need to pull we to discover all the sources and then align on the metrics. But yeah, I figure out that could be rid of difficult and big organizations. What I’m talking about when I, first of all I distinguish really the data observability and data quality space and the reason for that because I think it was mixed up by some vendors on the market because when we talk about data quality, it’s basically the recipes like a SQL against some column in data warehouse to understand if this figure is calculated correctly, what is the range of this metric?
25:36 What is the now distribution? If the idea unique in this particular column, it’s basically a very targeted assessment. Data quality could be performed through various ways. There are open source solutions like great expectations. You can use DBT to implement test data form, propose assertions, like there is various ways to do that. Basically a majority of the times a 99% going to be a SQL SQL solution. It’s a sql. When I’m talking about observability of data platform, it doesn’t imply any quality checks. It implies the house of data platform. And because quality checks going to be always targeted, you cannot cover your entire data warehouse with quality checks. Otherwise you go mental. And we’re not even going to mention how many alerts you’re going to have. You don’t want to know about it.
26:42 How I envision it, it’s fully automated coverage in a way. Analytics of your data warehouse where you see that pipelines are running as expected, all the pipelines are executed where you want them to be executed. All the tables receive expected volume of data and if not only after that you receive the anomaly and it’s not a scheduled query in case of the quality checks because you always need to schedule it. The queries, okay, it’s something automated that sends you an alert when something goes off. And plus to that one of the pillar of reliability of data platform, we envision the cost of it.
Speaker 2: 27:32 And thank you so much because I was about to ask you that question because there is a really thin line and a lot of confusion in the data engineering world about data platform reliability and data reliability which kind of equal data quality is implying there some way somehow. So making this differentiation I think is important for the people listening today so they can really focus on whatever they want. And I think that right now for data platform reliability is of course all this monitoring everything you mentioned, but what do you think also data platform reliability from a security perspective, how do you see that part there? Like automating a scripts for example to check how your data is, I don’t know, showing PI data or sensitive data or how the projects are built up or these kind of things. What’s your view on that?
Speaker 3: 28:29 To be honest, I don’t have an extension. We don’t do security and it’s not my core, how do you say core competency. I cannot say that I’m security expert, whatever, but I just love seeing how some jobs in clients data platform are own it in a sense of service account or principal account. Like it’s fired from personal Gmail. It’s just my favorite thing to see some kind of ETL is just fired from its personal Gmail instead of Yeah, I always see that on and off.
Speaker 2: 29:19 Well I have seen in, I don’t know, I was a data engineer at that moment and we were two data engineers and one guy was kind of sick and tired and this guy had access to the, we only had one admin account so we have to share in between and he was in a sick leave and then one day he got upset because he got fired, he knew the credentials and he dropped the entire database. He delete everything from a WSI was panicking, all the services were down, all the dashboards were down for the entire company and we were only two and I was pretty new in there, so I to fixed it after three days trying to recover something, the backups were quite old. It was a mess. But this is things that you can check also if you have good Im roll set up even some automation in the data platform that we’ll check that everything is in place. But it’s crazy how many companies, as you said, coming from a Gmail account or personnel personnel or these kind of chair admin accounts that the password never changed and they know just the server name and can’t connect anytime and screw you over so
Speaker 3: 30:33 Well we’re not judging here. Okay,
Speaker 2: 30:36 Not at all. Not at all. I mean we are all
Speaker 3: 30:39 In different journeys, but listen, there is a big company in Europe, it’s like southern of people working there. And I know for a fact four years ago they have situation where they have dropped all the data in their S3 and IWS and worse to that, they discovered that about 40 days after they dropped it all the historical data that couldn’t be recovered anymore. But they literally had no clue that that happened. Somebody did it and that’s it. Yeah. Okay.
Speaker 2: 31:27 So I hear from the public is saying security is only as strong as the weakest link quite through
Speaker 3: 31:39 Folks.
Speaker 2: 31:41 It’s crazy. Yeah, it’s crazy. And I see it also, I give coaching and mentoring to some companies and one day these people from a startup in Berlin just came to my session and we want to do AI in our data platform. And I’m like, okay, what’s your data platform? Well, we have MongoDB, but now we’re going to get GCP and we’re going to start doing some data there. And I’m like, okay, so what kind of AI you want to do with that? I don’t know. But what happen if I hire one data engineer, will he be able to do AI for me and also keep the S
Speaker 3: 32:22 To listen, I’m sorry,
Speaker 2: 32:25 It’s
Speaker 3: 32:25 To listen
Speaker 2: 32:27 Exactly. And I was like, okay, this is a 45 minutes free mentorship that I was given. And I was like, this cannot be happening. I didn’t know how to answer. I keep answering questions. And I told him immediately, this is not how it works. You need to do this and this and this. First with a platform, you need to do this and this and that. The data engineer is basically a data engineer. He might know a little bit of modeling but maybe not, doesn’t know at all. So you need to maybe hire someone else to help you at least a working student, but it is not going to be perfect. But first focus on the data platform and what you want to do. It doesn’t have to be big, but you need to start with someone. No, but we want AI because everybody’s using AI and we want to offer ai. And I’m like, so my next question is about that topic. What do you think or what do you see coming as a trend using AI for this topic of data reliability for the data platform?
Speaker 3: 33:23 What I see is that more of a data, not data people, more organization gets sober from ai. They come, they sort of getting more realistic. They started to do it more cautiously as it’s also very expensive and it’s not just capabilities CPU servers and everything. Let’s put it this way, you already have your data platform. You already know everything. What is happening in there and you ready to move on to the next exciting step? And there are some organizations like that, but they are not R into it despite one of the things is they started to look very cautiously what the use case and if that use case really deserve AI or it could be managed with machine learning which is much, much cheaper or it’s more efficient, sometimes it’s just statistics and that’s it. You don’t necessarily need AI for everything. Of course you may need AI to impress someone and it feels like Google is doing that Google Cloud, they want to impress they’re building up their marketing capabilities. And I think I still trying to figure out why I’m not impressed with it. Maybe I don’t understand it fully, which also could be a case. But this is a wedge where they closing module deals with enterprises and I’m still trying to figure out who they impressed with it with all AI things like executives or whoever. So
Speaker 2: 35:05 Sorry to interrupt you, but the fact that you are not excited, I think it’s because you understand that AI is not everything. So I think it’s not the other way around. It’s because we understand that we cannot give this big jump into the AI topic without having a solid foundation, right?
Speaker 3: 35:24 Oh listen, I want to add AI to Mass Hadley dashboard, but every time I’m thinking about the case, there is no case for ai. This could be done with easier techniques. You don’t need to spend fortune on AI to nail something down because it’s still, the other thing about AI is that it just doesn’t, it’s not only servers, CPUs and everything, but also it’s people who would need to scale up. So either we would need to hire someone new who is immensely expensive today on the market or scale up our current workers employees and which is also sounds good to me, but it’ll take them twice longer to do the same task as of today. And this is a question to me as a founder, what I want to achieve with it.
36:24 But that’s a trend that I see from big organizations that have all the money in the world, not all the money in the world, but some money more than obviously. So this is a nice trend. We’re not talking about Netflix, meta, whatever, Google, but we’re talking about organizations that maybe IPO companies. They have budgets, they have data teams, data engineering teams, but they cannot hire the best data engineers or AI engineers in the world. They don’t have budget for that. So what they’re trying to do right now, they’re trying to scale up their current data engineers to give them interesting and to give them sexy tasks around AI to find out the use cases, to workshop that, to do internal hackathons where they can actually nail down the business cases while they outsource. Outsource in a way where they add vendors, make their cost more efficient regarding their data platform to optimize their data platforms.
37:32 They’re seeking for vendors like we to add monitorings so their team wouldn’t have to build this monitorings because what data engineer want to build monitorings, everyone wants to scale up and do the AI things. And this is what more mature data managers are doing. They’re trying to focus their data teams to provide value. They channel the focus of their teams to actually figure out what they can do with AI because they have these capabilities but they don’t want to waste their data team time on routine work. Let’s put it this way, not sexy work. This is the trends that I’m observing now because they fully understand if the data is not reliable in their data platform, they cannot move on to ai. But there is no such thing like 100% reliable data, 100% trustworthy data. They also understand it, let’s put it this way, you need to set the, how do you say, honest expectations with yourself.
38:48 What percentage of the data, what is the expected percentage, what is the expected percentage of reliable data? You can cover 90% of your data warehouse with checks and everything, but the rest of 10% will cost you from the effort standpoint, from the people resources, the same amount as a 90% and it not necessarily worth it. You need to expect to accept the expectations on that and be honest with yourself that this 90% is good enough because it’s different organizations if you work in some organizations that can afford it. Go ahead I organization. Yeah, yeah, yeah. It depends. It depends on your business. That’s it. Some organization may need data contracts for every pipeline which gets data in and out. So it’s two data contracts per one pipeline. And let’s imagine the medium sized organization has 10 to the pipelines and they can afford doing it go hat it’s possible, but is it worth it how much time the data team will spend on it? Maybe they can create more meaningful output where they can investigate AI and do something. So it’s kind of depends on the organization, their goal, their focus and budget as well.
Speaker 2: 40:29 I dunno answers budget
Speaker 3: 40:30 Is very important. Answer the questions.
Speaker 2: 40:31 Yeah. Yeah. And budget is something that I think every organization now out of the blue they got from the pocket, oh, AI budget, let’s make it happen. But what happened with everything else in between that, right? Again, the foundation before you really start
Speaker 3: 40:48 Not really, I’m likely to be talking to this organization to kind of getting suffer from this being more realistic. I mean they’ve been realistic like okay, if we need ai, we need to grow use case. And that use case really tailored to ai, not to mal because
Speaker 2: 41:13 A lot of the organizations that I speak with because, well, I also provide in my company these kind of workshops that I have different ones for data products and everything, but I have one for a I strategy and use case discovery and I did one of those workshops last year and then it was for directors as well, but not only for data but in general because all of them wanted ai. They don’t understand anything because some of them were from marketing from hr, but they wanted to jump in the AI train and then there was a talk keynote about it. We had the workshop done and all that. Then I get some feedback. They were really keen, we did some discovery, we chose some use cases that really will bring value, but at the end some of the feedback was we already know that. So this was really repetitive because we know how it needs to be done.
42:06 And I’m like then if you know how it needs to be done, then YU haven’t done it and you want to spend money. Like look in this company for example, they wanted to hire external people to do AI when they had actually five data teams in the entire company with 17 people each data team because the people from marketing were keen to do ai. So they had some budget to hire someone external. And I found out that when I was recovering use cases from the entire company, a 1000 people company, and I understood that marketing was doing quite a lot, but they won’t feel the trust to use the other five data teams that were around there from different business units to help them or because maybe they didn’t even understood what these data teams were doing or how the data platform was working. So when you try to get to management to make them understand why they need to invest in their own data platform but not on external people.
43:07 To do that, you have to be really clear what are the capabilities you have in this part of the reliability is quite important. So you can really offer them, okay, we’re going to get good data to be able to create a model or whatever. But you see a lot of companies that do this and then it’s trash in trash out. I had worked in this company that I’m working now that they have some external consultants from a quite expensive consultancy company and basically they do like a shell of a project, but you still see in trash, in trash out. And this data platform that is there is a Ferrari in a cell that really doesn’t have any foundations. There is no data observability, there’s no data security. There’s a lot of things that where is the monitoring, where is the stuff that we need here? Where is the I am roles? There’s not even that in place. So they spend two millions per year on this expensive data platform that doesn’t have not even the basic requirements to be reliable for the customer. They will like to have, they don’t have of course a lot of adoption, but then we need to clean up the house before you can actually advertise yourself as a reliable platform to do things with data. And it’s crazy how you see this in different big organizations with money, but they just don’t know how to invest it correctly.
Speaker 3: 44:29 Maybe we’re talking about this disconnect between business strategy and data strategy in this case because this is a foundation, you cannot have AI strategy if you don’t have data strategy, but also your data strategy should be connected to business strategy. And then we go, okay, what is it data strategy means at all? It’s just some people think that the data strategy is tooling that they’re going to be using the ai, well the AI strategy even, I have no idea what that actually means, but for me AI is just a part how it’s a flavor to data products in a way where you add AI to simplify the things so it worked them faster or enhance what you have today. One of the cases that ING bank did, they shared with me and for a second it’s ING, European Union Bank, one of the biggest banks in the world, they use AI today and you can imagine how their security standards and everything, so what they do, they need to understand how their B2B largest customers are doing around the world.
45:54 So they need to collect signals from different medias to understand if somebody infringed human rights or if somebody went bankrupt somewhere or some scandal happened to them. And so it’s like resources to parse a day for them to understand if there is any mention of their client they have how many, I don’t remember the exact number, but you can mention how many btb customers have. They have it’s thousand, it’s a dozen of thousands. So they used to use machine learning for that and that was really hard because now they use an L lm and that is so much easier for them. It’s designed to process the language and it simplifies so many things for them because first of all, the translation before they needed to translate first then feed it to machine learning to parse for the keywords. And right now L LAMB handles that everything from the out of the box.
47:13 And one of the use cases that they highlighted how L lambs are better than machine learning because there was a article about jaguar, Jaguar animal that attacked a boy in a zoo. So basically what have happened, the machine learning model and service that was powered by machine learning, it captured that article and sent it over to customer success of Jaguar. And he was perplexed like why I’m having this at all, why did I receive it? And this is because the technology wasn’t good enough to do this task. And then they added the AI in a LM and it works just perfect. It increased the results. It is more efficient, less false positives and it just works perfectly. And this is a use cases where you’re looking for to add ai, you may have already something working and you just need to improve it with AI for your current task. And this is where you start. It’s just makes so much sense. You don’t use AI for the sake of AI and I’m so against it. And then this is unfortunate as the case because this guy took your consultation saying we need AI 45 minutes of pain, we need the eye just to add the slides and all of that, whatever. It just doesn’t sound house to me unfortunately.
Speaker 2: 48:54 I have another question for you that data governance finally is becoming a trend and people is more prone to hire a data governance manager or go via data mesh, go to the DMA book, whatever they would like to, but at least I’m happy that they are kind of taking care of this important topic. How do you link data governance with data platform reliability?
Speaker 3: 49:22 Oh wow. Well data governance is such a big topic and it depends what you include in there. Like the management, the access management, the data discovery, and we are not in that space at all. I’m trying to keep away from that space. And despite of that we do have data catalog and data lineage. But what I understood first of all, if you have monitorings but you don’t have this business’ data lineage or data discovery, it was nothing. And the reason for that, if you cannot understand what is the implication of your errors in the data warehouse, if the pipeline fails, you need to understand what the implication down the road, which data users affect. And this is how we deliver that through the lineage. We actually show that visually how things are affected down to the user and looker dashboard. I believe this is part of the governance, but it’s not the governance in a sense for business users where we have, oh, it’s cloud governance.
50:42 Actually in our case it’s more a tool to help data engineers and this is how we see the usage. I’m coming from the perspective of basically we have a UI and I know how the usage is happening. So in our case, users every morning open the Slack channel, see visually what is the anomaly, what are the errors or what is the failures with the cost, whatever. And then they go to see that in UI and then they check lineage if there is something severe. But if you’ll ask me how often they go to Lineage to check something from the data engineering perspective, it’s not happening often. It’s more in the case when they be doing something or be writing some model or it’s very ad hoc. It’s not like every day. So I think it very much depends on the size of organization when they need to have the governance team or responsibility.
52:01 But it also ties back to culture in the organization for me where people need to think what kind of data they stored and why they need it. And if they stop using some dashboards, they just need to have guts to delete it. Please folks, do it. Take amen to that. And then if you don’t need something, it’s easier to breathe. It’s easier to live. If you stop wearing something, you either give it to your mom or friend from your wardrobe thing or you just, I dunno, it’s the same with data. You need to, it’s also the data governance. Why some such a mess? Because I think they last decades there was a notion in data communities that let’s store everything, we might need it, but things got outdated as well. Even your user behavior that was happening in five years has nothing to do with the user behavior you have today. You just have to have the guts to delete data or at least to archive it. And I guess it has more to do with culture, let’s put it data governance culture.
53:30 And unfortunately people don’t want to take the responsibility to do those things. This is also what I see because at Mass Head we have this functionality where we show the dead end tables, which have upstream compute consumption. Basically they’ve been updated on a daily basis. There is a data model upstream, but no consumption downstream. There is no dashboards, there is no usage from the console. It is dead end tables and they cost like $500 a month, one single table in which we can only about the compute, not even storage. And we have this report for our organizations and they didn’t act on it. And some of the reasons that we might need it, and this is just painful to hear. Okay, so if you might need it just do you want to archive it or nobody been using it for a while? This is the, yeah, so it’s more about culture thing for me unfortunately.
Speaker 2: 54:37 Yeah, I think data mesh helped quite a lot with this topic because data mesh is whole sociotechnical stuff. So data platform reliability is a big part actually of the whole transformation with data mesh. But I see more and more doing some hybrid kind of data mesh with reliability, sorry, with data platform and kind of teaming up because people is understanding more, right? The need to have a really stable and well functioning data platform to be able to offer the people. But this day at the same time, they marry the best practices from governance with ownership, with data quality, with observability, with security, with everything to be able to provide a good service, but also to have a data. I mean not everybody’s technical, but having a good data governance managers who is able to advocate and also push about these topics from the data platform to happen. It helps tons.
Speaker 3: 55:42 Well, this is one thing you see, we have such a different experiences, so we are talking from the perspective working in a huge organization with so many silos. And I’m talking from the point of my personal experience, which is obviously smaller startups, also large different organization, but also it’s important to understand I’m not exposed to their inner world of those. Of course in everything I’m exposed to data teams and very interesting thing that I personally admire and the cultures that I’m fostering here internally is accountability and leadership of the task that you are doing. And this is something that I value in people I guess, and this is a culture that I’m fostering in a way. If somebody is doing task, I want them, I’m not coming to them despite my product manager background with recorded description, how that should be done. I’m not doing it.
56:52 I have no mental capacity to work with engineer hand in hand and provide all the requirements. I want that person to take responsibility for the task they are doing and the same. And I think this is where I’m coming from, this approach when I’m working with the customers, I expect them to want to do things and I think it’s also a reasonable expectation. Maybe not, maybe my bar is too high, but this is if you’re doing, sometimes you need to put an extra certain it and if we’re going to need it like challenge, maybe it’s an extra mile. I don’t know.
Speaker 2: 57:39 I think it’s basics as well. I agree with you, but yeah, Julia, we’re running out of time in three words. Oh really? Yeah, just two minutes. We talk so much and I think we can keep going here, but just in three words, if you can, for our listeners here in the audience, if they would like to start improving their data platform reliability right now, what is the first steps they should take? Really quickly?
Speaker 3: 58:06 Folks, identify your critical assets. Identify what is the most important things, understand what is happening, do the benchmark, assess what is happening with them today and trace down how you can monitor it. Like implement simple data contracts. There is be O from GGP, you can have different monitorings that are provided within cloud. If you, you’re on Google Cloud, then message me. I can provide you message to test it. Just understand what is happening with those critical things. Then if you test it, iterate on it, show the results to your leadership and you can iterate and make it at scale. Just do the benchmark from the critical things and then scale it to understand the house. I guess.
Speaker 2: 58:59 Thank you so much, Julia, for your time today. It was amazing to share with you and always having a conversation with you. It’s fun. I appreciate your time here today.
Speaker 3: 59:09 Yeah, thank you so much. It was pleasure talking to you. Yeah,
Speaker 2: 59:12 Thank you. To our listeners, thank you. Thank you for our listeners for tuning in and joining us next time for more insights on the world of data. We will have Andrew Jones about data contracts next month and data quality, because I want to keep on the same track. So we’re going to have a fun one in August. Yeah, me too. So until then, stay data driven, stay with data quality as well. Thank you so much guys. Have a nice day.
Data Mesh Learning Community Resources
- Engage with us on Slack
- Organize a local meetup
- Attend an upcoming event
- Join an end-user roundtable
- Help us showcase data mesh end-user journeys
- Sign up for our newsletter
- Become a community sponsor