A summary of the discussion during our Data Mesh Learning community panel held in May
In May, the Data Mesh Learning community hosted a panel discussion on how to become a data mesh engineer, highlighting the technical and soft skills needed to succeed.
To start the conversation, participants shared their backgrounds and experiences working in data science, data engineering, and software engineering. They then explored the skills and roles involved in data mesh teams, including data engineers, data platform engineers, solution architects, data architects, and data product owners.
The speakers explained that a data mesh engineer is an extension of a data engineer and performs similar tasks, but with a focus on microservices and software development. They discussed the creation of data contracts and the skills and techniques used to build microservices architecture and pipelines.
The participants agreed that overall, organizations need a well-rounded data mesh team with a balance of skills and expertise.
Becoming a Data Mesh Engineer
Panelists:
- Laveena Kewlani, System Architect, PayPal
- Kruthika Potlapally, Lead Software Engineer, Clairvoyant AI/PayPal
- Jean-Georges Perrin, Intelligence Platform Lead, PayPal
Moderator:
- Eric Broda, President, Broda Group Software Inc.
Watch the Replay
Read the Transcript
Download the PDF or scroll to the bottom of this post
Ways to Participate
To catch an upcoming event, check out our Meetup page.
Let us know if you’re interested in sharing a case study or use case with the community.
Data Mesh Learning community resources
- Engage with us on Slack
- Organize a local meetup
- Attend an upcoming event
- Join an end-user roundtable
- Help us showcase data mesh end-user journeys
- Sign up for our newsletter
- Become a community sponsor
You can follow the conversation on LinkedIn and Twitter.
Transcript
Eric Broda (00:00):
Welcome to the Data Mesh Learning Communities meetup. Today’s topic is becoming a data mesh engineer. Now, I think we all realize it’s a pretty broad topic, so let me kind of set the stage for today’s discussion. First off, as as we know data is the fuel of the modern enterprise. I don’t need to say that too much to this crowd. We all know it’s the foundation that enables better decision making, improves operations creates innovations and delivers these outstanding customer experiences and outcomes that we’re all looking for. Now, as a data mesh community, we probably understand core principles of technology quite well but data mesh is implemented by real people, in fact, real data mesh engineers. And these require, these people have some exceptional skills, but really, what are these skills? And obviously there’s some technical skills, but what are the soft skills that are also needed to be successful?
(00:55):
And, and along that lines, how are these skills built? Well, these are all questions that we’re gonna be answering today during today’s session, and we’ll probably be touching on a variety of different topics related to that. Now, a along the way, our hope is that you as a data mesh practitioner gain a few insights to help you better build and deploy your skills to your organization as they implement data, products and data mesh. So that’s our hope. Now today we have several members of the PayPal data mesh team that are gonna share their insights based on some real world experiences. So I’m gonna ask each of them to introduce themselves. So Levina why don’t we start with you? Tell us a little bit about yourself and what you do for a living.
Laveena Kewlani (01:38):
Okay. I am actually a solution architect in PayPal, and I build data and mesh for my living. So, just kidding. I have other things as well. So, yeah I work as a solution architect in PayPal and in my day-to-day life, I design components of different products, data-oriented products. One of the major products I’ve project I’ve been working on is data mesh. And it was really interesting to work on. So, yeah.
Eric Broda (02:11):
Fantastic. Ika, tell us a little bit about yourself.
Kruthika Potlapally (02:15):
Hi everyone. This is Ika. I have been working as a lead software engineer big data and data platform since about one and a half years. And I have been involved in building data manage for PayPal in again, since the past one and a half years. And some of my skill sets range from working on big data technologies, both batch and streaming, like Kafka Hive, so on, and also on the cloud services and the language that, languages that we have pre I have predominantly used our Java, Python, seql very similar to what we have been using for big data and software engineering. And in addition to the technical skill sets I have been involved in working and collaborating with multiple teams across the organization. Some of them being business and some of them being enterprise teams. So we are all here today, and we would love to discuss our experiences with you. Thank
Eric Broda (03:24):
You. Fantastic Ika. Thank you Jean George, tell us what you do for a living, my friend.
Jean-Georges Perrin (03:32):
Yeah, what am I doing for a living? Okay, so my, my title is intelligence platform Lead at PayPal, but I, I, I kind of self-identify as a software person, engineering architect, or whatever. I had the pleasure of leading a great team, and you’ve got two members here to build and to design, build and deploy in production, a data mesh at PayPal. More recently I also published this little little book data mesh for all ages. And here you can see the Chinese and English version all available on Amazon. That’s not for a living, but you can contribute to my living with by buying it <laugh>. And I love sharing knowledge and around, around data mesh and technology and data in general. So I’m more than happy to be here and pretty excited to be once more with this fantastic group.
Eric Broda (04:28):
Fantastic. Thank you Jean George. Thank you, Levina. Andika. Now real quickly here on an administrative note, each of these panel members are talking on their own behalf and not for their employer or clients. So now, as for me, my name’s Eric Broda. I’m the president of Broda Group Software. It’s a boutique consulting company focused on accelerating financial firms enterprise data mes journey. My team is currently working on two pretty big data mesh engagements. We’re building a global payments mesh and we’re also leading a team that’s establishing a data mesh to support global management climate data. But more importantly it is my honor to be your panel host for today’s discussion. So, without any further ado, why don’t we get started? So again, this topic becoming a data mesh engineer. So I’m gonna say, why don’t we start with levina? So, so I think Ika, you gave us a pretty good background, but Levina, tell us about your background and experience. When did you start working on data mesh and, and kind of what are some of the things that have been going on relative to data mesh in your so far in your career?
Laveena Kewlani (05:36):
Okay. So give you a little background, like, how I eventually landed up to this position is I started as a backend developer. Then I went and did work on data science and data engineering products together. That involved building models and pipelines for the product, like end to end. And then I worked as a data scientist for some time, like pro product projects that needed data scientist skills. Then some of the pro projects, which I also needed ci, cd pipelines, software engineering. And because it involved computer vision and everything. So it was more of a combination of a data sign, like data scientists, software engineering, data engineering, everything together, which eventually, like when I landed up this to, into the, the, into the project of data mesh, the entire work history that I have from 2013 onwards actually helped me a lot in in like, that helped me a lot in order to get the vision of what actually is happening. So if I would have started as a data engineer, like from the scratch without much of an without much of an you know, background of backend development for software engineering, or if I would have been just involved in Python, not wouldn’t have touched Java, c plus plus C, my sql, et cetera, et cetera, it would have been very difficult for me to be able to envision this pro project and to design the components that is, that are helping different parties that are using data mesh. Okay. So,
Eric Broda (07:25):
Okay. Perfect. I
Laveena Kewlani (07:26):
Think it’s a combination.
Eric Broda (07:27):
Okay, great. So, so Kika before you did this data mesh stuff, it looks like Levina had some data scientist experience some data, some software engineering is, did you have a similar background before you started this data mesh journey?
Kruthika Potlapally (07:42):
I had a background of software engineering and data engineering. So I was previously involved in building, streaming, a platform for streaming data to process streaming and batch, as well as building microservices. And so that experience of combination of software and data engineering helped me understand and build a data platform and work with data engineering teams for data mesh.
Eric Broda (08:13):
Okay. So it’s safe to say you both had some solid experience in software engineering, some, some data engineering, even data science experience, all of which I think is, is great. A great background for, for applying your skills and capability on data mesh. Now let me, let me just talk a little bit I wanna take a step back. You’re, you’re obviously data mesh engineers but what is your broader data mesh team actually look like? What are the roles in your data product team you know, is there a platform team where’s the data engineer, data mesh engineer fit, kind of paint the broader picture of what and then ika, we’ll start with you. What, what does the broader team data mesh team look like?
Kruthika Potlapally (08:56):
To me, data me engineer is just an extension to a data engineer. Someone that works with data, someone that use, utilizes a data platform to be able to provide data to the business u users or ML engineers or data scientists. And so our team comprised of data platform engineers as well as data engineers. The one addition to the team that we had was that was important to us was data product owner. So data product owner is the one that helps in coming up with the data contract, and he he or he, she owns the domain and everything relating to that domain. So the roles that we have had are very similar to traditional software engineering or a data engineering team. In addition to this one extra role of having a data product owner. So we had software engineers to build the platform, and we had data engineers to come up with the transformations and provide make the data available to our end users.
Eric Broda (10:10):
Okay. Cri, perfect. Now, Lavina, I’m not sure if you, are you on the same team as Tika or are you on a different part of the, the data mesh team? So
Laveena Kewlani (10:19):
In addition to what Tika said we also, like, we also had engineer we also had architects like solution architects and data architects. The solution, the person who was a solution architect like me, I used to interact with the product owner from different area, be it the consumer or the person who is going to construct the data contract and then design the product and the entire pipeline. So as, as we already mentioned, that data mesh at PayPal is more of a microservice process. Like it’s a microservice based, you know, stateless architecture. So this was the design that we came up with, the help with, with in collaboration with the tech leads architects, both solution and data and the product owners.
Eric Broda (11:14):
Ok. Now I’m gonna come to you Levina. So, so if you were to kind of summarize in, you know, in 30 seconds or less what does a data mesh engineer actually do? What does the day in the life of a data mesh engineer, how would you do, how would you answer that question?
Laveena Kewlani (11:34):
So they do exactly the same things, which were, which is a actually been done by a data engineer who you know, transfers the data. He does ETL jobs, has jobs and everything, spark everything with additional you know, twist to the, to the extent where a little bit of software development is also envisioned, like usually when you create an E T L pipeline, you don’t think of microservices, right? But when you think of the process of filtering the data and, you know, creating a product which is needed by and creating a pipeline, which can be used again and again by the different, you know different domains basically or, or the data products, then you have to like, use the, use that work in redundancy. So there, the data engineering work needs to be replicated. So there a little bit of software engineering comes to a play. So there we are doing exactly a data machine engineer does the same things, but a little bit of, you know, spices of, there
Kruthika Potlapally (12:47):
You
Laveena Kewlani (12:47):
Go. Software engineer.
Eric Broda (12:49):
I like it. Now I, I wanna kind critique, I want to drill down a little bit into that. One of the things I thought that I, that PayPal actually recently released was a open source of data contracts, data product contracts, which I think was a fantastic a fantastic event. And I, if I’m not mistaken, it’s, it’s got over 200, maybe 300 stars already on GitHub. So this is something the community is really taking up. So, so first off, I wanna give, give the team here kudos for that, because I think it was this team that actually put a lot of that together. So, so ika with that in mind, like, tell me about how you as a data engineer and your team created these data contracts. What was the, the, the impetus for it, and how did the data engineer provide and craft that core artifact?
Kruthika Potlapally (13:41):
So when we started off, what you’re seeing is many revised versions today, but when we started off we started a data contract with the idea of having a contract between a producer and the consumer, and everything relating to what the input data sources are, what the output data sources are, and how we are going to monitor or monitor this data or meet the essays of this data that we are receiving. So initially, our data contract consisted of schema of input data sources are output data sources, and a few checks that we were performing and the statistical scores that we were having as part as part of our observ observability process. Eventually as we as we have evolved as a platform evolved, we added more details with respect to ownership stakeholders SREs data policies that are part of the organization and obviously having information with respect to the schema and what the description, the data dictionary all of this were that, that were part of the version one were included as well today, the data contract that you see encompasses everything with respect to ownership stakeholders schema data dictionary SLA information organizational policies.
(15:23):
So so the way the data contract is created is we work with the data product owner in coming up with what, what are the data sources that are required and how and how we are going to shape our resulting data sets. So here the idea behind data mesh is very similar to how data engineering or big data projects have been working. It’s just the shape of the data that changes based on how a data product owner curates the data contract and the final resulting tables.
Eric Broda (16:01):
Okay, perfect. So, so obviously now Levina, we’ve heard about these data contracts, but you mentioned also a few other things that, that a data engineer do, a data mesh engineer does, that’s you mentioned obviously you guys have a role in building the microservices architecture, and you, you obviously have a role in the establishment of the pipelines that allow the, the data products to talk to each other. Tell me a little bit about how you know, tell me a little bit about what the skills and the techniques you used to, to, you know, in the architecture that you used to build those microservices in the pipeline. What are some of the, what, what are some of the, the ENG data mesh engineering skills that you needed to put that together? Oh, yeah. I think you’re on mute levina.
Laveena Kewlani (16:49):
Yeah. So basically I would like to reframe your question a little, if, if you don’t mind. Sure.
Eric Broda (16:55):
No problem.
Laveena Kewlani (16:56):
Said how to become a data mesh engineer. I would rather put it out as how to build a data mesh team, because you need a lot of people in there. You need a person who can envision data as a data who’s basically a data engineer. You need a software developer who is able to see the data as a product and a service. You need a product owner who has seen the software development, who has worked like, or has an experience of software development products, as well as a little bit of data product. You need a state architect, and you need, you need a solution architect. You need all of these people. So because, and then there’s a balance of the skills and the, and the work that is being done. Like, okay, a data engineer, he will be more comfortable to develop e ETL pipelines, do batch processes on spark or scale whatever language he or she’s comfortable to.
(18:01):
But then the software developer mindset will come into the role and they’ll say, okay, why don’t you just make it a trigger or something and reduce the redundancy, create a microservice out of it, stuff like that. So the skills on like, in, in, in a very broad way, if I could say they’re not anything additional that you need, is just the visualization that you need at, like, you know, that some of the stuff that is mentioned in the ax book, you, you should be able to visualize that. And thanks to a leader like JayZ who was able to let us visualize something like that so complicated, in much simpler words, so that team like Ika and me and the entire team who were working together to build that, were able to come together and bring forth the a data mesh that is actually created out of no additional skills, but the same data engineering skills and a little bit of software development, which very, which is very normal to understand, like, oops, oops, concepts.
Eric Broda (19:09):
Ok, now, so, so ika, let me build on that question. So, so, so Levina said, there’s, there’s, I’ll paraphrase software engineers, architects, a whole raft, a whole group of people. But let me ask specifically what, what’s the difference between a software engineer and a data mesh engineer?
Kruthika Potlapally (19:29):
So, I would say data mesh engineer, again, is an extension to data engineering. One additional feature that we provide is how having the QA or the observability in built in the platform software engineer, again, comes with the same skillset as a data I would say software engineer, data platform. I think it’s building up, building that microservices for the data platform. The skillset between a software engineer and a data mesh engineer is, I would say data mesh engineer encompasses a little bit of software engineering, wherein the user, wherein the engineer needs to understand the microservices and to be, and also probably build in case or support the data platform team. But data engineer and the data mesh engineer are very tightly in line. They’re the ones that make the data available to the end users based on the transformations that they, the, the business requires.
Eric Broda (20:34):
Okay, perfect. Levina, let me go to you now. So, so let, let’s, let’s say I am a person maybe two years out of university College. I’ve, I’ve done some software development. What are the skills that I should be looking to build if I wanna be a data mesh engineer in the next year or two?
Laveena Kewlani (20:57):
Okay, so given that you already have software development skills, so you must be aware of the languages like Java or Scholar Python, et cetera, and then you must be aware of like the stuff that is generally used like abstraction, encapsulation, that helps in defining, you know, an API based or microservice, like API based microservices and stuff like that. But then if you want, if you wanna become a data mesh engineer, you need to be able to know how the data pipelines work, right? So this data pipeline work strategy needs, like, it’s, it’s mandatory for a software developer to understand that and see data not just a you know, raw file that is sitting somewhere in the application db, but to know how to make that raw file into a filtered, usable data for a data scientist and data engineers. So this entire process of transferring the data from a raw unfiltered data, which is lying somewhere in the application, DB two, some, you know cluster, say Redshift or any, any cluster G case, anything. So this entire, this entire, this entire steps that are needed to transform the data into filtered and the usable form, this is something which is crucial for a software developer who is two years or three years in the industry to know how, like this is basic. So if he knows that, I think he’s good to go.
Eric Broda (22:31):
Okay, perfect. Kristi could,
Kruthika Potlapally (22:32):
Just in addition,
Eric Broda (22:33):
What, what are your thoughts?
Kruthika Potlapally (22:35):
Could I add one other skillset? The other is collaboration and communica good communication skills. You would be working with a lot of business teams, infrastructure teams, enterprise teams, and it’s important to be able to communicate and collaborate with.
Laveena Kewlani (22:53):
Perfect. We’re, that was like, without even saying Yeah. That, like, this thing goes without even saying. So for example, once we build, like, I’m just giving you an example how important the collaboration is, that once we build the data mesh it was very, it was a very you know, new and the, and, and for some it was very complicated, complicated product or the design. But just because we were collaborating with people in order to get the stuff done, we collaborated to an extent that they started liking it. So within the collaboration itself, like your work starts getting sold out.
Eric Broda (23:33):
Okay. Perfect. Now we’re gonna, we’re gonna touch on that in a little bit more detail. Cause I think you’ve touched, you’ve highlighted a key part where it’s, it’s not just technical skills, but there’s other soft skills. But I’m gonna, I’m just gonna ask the next question to Jean George, who is kind of the, the data mesh manager or executive, and he has a team of these data mesh engineers. So, so Jean George, what is, what is a data mesh engineer career path look like? You know, where should they start? And, you know, once they’ve been a successful data mesh engineer, where do they go?
Jean-Georges Perrin (24:07):
That’s a, that’s a, that’s a, that’s a great question. Thanks. Thanks, Eric. I think it starts at, at the data engineering. Okay. And, and if you think about what Jamag is saying as well, she says that, oh, data engineers are kind of dis disappearing or something that should not, that should not be a d a difference with between data engineers and, and software engineers anymore. But I, I think it still starts at the, at the data engineering level. Okay? So you, you, you need to understand what the data transformation is, what e ETL is, what what a, what modeling is, et cetera. So when you add the, the skills, okay? So to get into becoming and building basically the out, when you’re thinking about the output of or the outcome of a data engineer is being the kind of a data pipeline, the output of a data mesh engineer for me is a data product.
(24:58):
Okay? and of course, the, the, the data engineer or, or the data mesh engineer is not in charge of saying, Hey, I’m going to have my own SLAs. Okay? but you work in conjunction with the data product owner. So that’s one pass revolve. As a, as a data database engineer, you can become a data product owner, okay? You say, if you want to go towards a product side of the world and you can then start building and having this very strong link with the business, okay? Which is really a required, and that’s, that’s an, that’s an evolution. And also evolution is data architect. Okay? the, when you’re thinking about the, the data quantum model and the DA and the, and the, the, the, the data product, one of the key element inside is the interoperable model. You’re exposing, okay? So if your model is not interoperable interal, you lose a lot of the benefits of the, of the, of the data product itself. So, and I think this is a skill that comes with data architecture, okay? A lot of data engineers naturally start doing it, but I think this is also something as you mature that you will be ma more mastering. So that would be, I think, two outcome of, or two, two possible evolutions from for data machine engineer, data product owner, and and the data data architect.
Eric Broda (26:29):
Okay. So I wanna, I wanna come back now to, well, thank you Jean George Levine, I wanna come back to you on the, the non-technical skills now in Jam’s book calls it the data mission is a sociotechnical endeavor. Now I found that the socio part or more specifically the organizational part is actually first off, it, it is as important, if not more important than the technical capabilities, but it’s actually a lot harder. So, Levina, what are your thoughts on that?
Laveena Kewlani (27:06):
See, I feel the technology behind the data mesh to come up with a data mesh is not, it’s not it’s not very hard. It’s just the perception, how you see the entire star. Like you are, for example, it’s very easy for you to visualize stuff in 2D on your screen, but when it comes to 3d, you need to, you need to be a little creative, right? So this is, this is what for, like, when I was trying to understand date data mesh from Zags book and the conversations that we had within our team, this is what was my thinking, that you have to think in 3D or maybe 40, right? And for a technical person, in order to make them understand this vi vision, it was easier. But for the end users, it was, it was difficult because sometimes they’re just comfortable in their, you know, day-to-day, day-to-day monotonous process, no matter how many times, how many glits comes in.
(28:12):
But, you know, change, we say it’s very easy to say that change is the most, is the most permanent thing, but we know how resistant human beings are to changes. The same concept applies everywhere. And it was difficult not at, not just at technical level to implement stuff and get approvals and all that in order to make them understand what actually we are doing. It was when it was eventually d done, it was really appreciated. But it was like, it was, a lot of, it was not a lot of work, but it was a, it was it was a repeated effort that we had to put to make them understand that how useful it is, see how, how, how it is reducing your workload, see how easy it is. So I think it, it’s, it’s very true what’s written there.
Eric Broda (29:02):
Okay. Ika, what are your thoughts?
Kruthika Potlapally (29:06):
Very similar to what Lana said. Any change is tough to bring in, but so what helped us is collaboration and working together. So a lot of times when we were building the platform a lot of time we, we have to go through the architecture approvals at the enterprise level. So initially when we began not everybody understood why we were doing this or why what purpose it would solve, but repeated collaboration, working with individual enterprise teams helped. And as I said it, it wasn’t just that, it was also working with the business teams. So it is important to bring ha to bring in this mindset of not just the technical skills that are needed for engineering, but it but the product and good social skills are equally required to be able to pitch in the product and to bring in the users.
Eric Broda (30:16):
Okay. No, I like that. Levina, tell me a little bit about you. I think you mentioned the, the word collaboration, or, or I can’t remember if it was ika collaboration was a key success factor. G gimme, tell me how that worked and gimme a, maybe gimme an example kind of brings it to life.
Laveena Kewlani (30:33):
Okay. So I cannot tell you, but I, I did give, I’ll give you an in general example. Sure. But there were some processes in the data mesh that needed triggers. Okay. A triggers sort of you know, set up. So the team who was you know, there, there are platforms, like they’re relevant platforms to do relevant stuff in PayPal. So we had to go through that enterprise team for that particular trigger. Now, we wanted to automate that trigger. We didn’t want it to, you know, do that trigger thing manually. We wanted to automate that trigger. And that, and that came with additional requests, <laugh>, of basically one or two tickets that need to be raised from that ticket, and a almost a sprint of work from that team, that enterprise team. So it was very difficult initially to, you know, it initially convinced them that, okay, what we are doing here is, is I know it’s a little extension of what you already do, but it’s not that tough. You know, we are just asking, like, you do a for Apple, we are just asking you for a, for maybe, you know, an Antarctica. It’s just a only <laugh>. So stuff like that. So eventually after repeated collaborations and making them try to understand and not giving up <laugh>, so we eventually had that automation.
Eric Broda (32:04):
Okay.
Laveena Kewlani (32:05):
So, so, and that speeded up our process, like data mesh that is speeded, like, that changed the entire, you know that made our product like maybe five times more famous.
Eric Broda (32:19):
Oh, fantastic. I like that. Jean George, I’m gonna ask you a question here. So, so I’ll broadly say it, say it this way, or ask the question this way. Clearly there’s some organizational skills perhaps the ability to understand the business, the ability to collaborate, the ability to sell. I suppose if you were to provide some advice to again, somebody two years outta college or university looking to become a data engineer, we know the, so the technical skills are, are important. What advice would you give to that person and say, if you wanna be a data engineer here, that the organizational skills, you, you need to think about?
Jean-Georges Perrin (33:01):
I think you need to, you need to get, to get into business. Okay. So it, it kind of relates very close to my my older, our oldest son just graduated from college. And it is going to, to join the, a, a major American bank. And the thing is, you don’t learn in college the business. You, you don’t, you learn you’re very good at art scale. Okay? I think in the US compared to France, at least when I was in France, you you learn also some self skills. So, but I think what a big miss is still the business. So when you join PayPal, understand our products, okay? Understand our, our portfolio of companies and, and that’s going to help you understand the outcome of your work. Okay? It’s not just the output, but the outcome of your work as well.
(34:00):
So don’t, don’t, yeah. You need to keep learning. Okay? That’s some, that’s some advice I give to, to youngster. I would say keep learning. It’s a, we, you, unfortunately, you choose a profession where you are going to have to learn all the time, okay? And if you stop learning, it’s so, you’re so, so, so continue working, continue learning, but make it sure that what you learn is going to be also business oriented. Okay? So if you go in healthcare, understand healthcare, understand and healthcare in your country, okay? As, as we can, we, we can joke together, Eric, I mean, healthcare in the US versus France versus Canada is really different. Okay? So, so you really have to understand to get this context, so to make, to make yourself more relevant in the context of your of your company. Because the goal is not to build data products for dev for the sake of building data product, okay? It’s the sake of building and bringing data to get better insight about the business, get, get a lower risk get increase profitability, increase, market share, whatever. Okay? You’ve got to understand what the outcome of your work is, not just the output.
Eric Broda (35:28):
Okay. Perfect. Now Levina, let’s, let’s go. Next question for you. Loosely speaking, what, what John George is, is saying, which I strongly agree with, is you need to have some business grounding. You have to understand the context and what kinda outcomes you’re trying to get to. But that requires a skill that I think not all engineers, software engineers, data engineers, data mesh engineers have, and that’s, I’ll call it in quote, sales skills. So when, when you think about your journey at PayPal or anywhere else, how did you sell as a data eng, data mesh engineer, how did you sell the concept? How did you get people energized? How did you, what skills did you need to change the behavior of the individuals that you’re talking with?
Laveena Kewlani (36:22):
Okay, so when we build data mesh, we were targeting to help a certain category of people, not certain, like everyone associated to data, but then we categorize those people into different categories, like into different sets, okay? There are one who are going to create the data products, and then there are one who are going to use the data products that are getting created. So the users have different sets of skills, and the requirement and the creators have different sets of skills. And the requirement, each sprint, we dedicated to each group. And when we dedicated to that group, when we were interacting, when we dedicated sprints or a, or a quarter to that particular group, we made sure that we are, we are empathetic towards their problems, and we are involving them in, you know, backward feedback. Like when we build it, we go to them, okay, are you okay with it?
(37:26):
So when you have this kind of setup of, you know, back propagation feedback, et cetera, et cetera, and you are empathetic towards their need, and eventually you give them, you know, the interactions that you have with them, they see by the end of, okay, three or four sprints or something like that, they see, okay, <laugh>, all the conversations were not waste. They were, you know, leading to somewhere, which is helpful for me. So this path is going to be this path, this development of the relationship development of the path of eventually delivering the pro product is something which was helpful. So this is, according to me, this is sales, the involvement from the scratch till the end. And same goes with the creators. So when it comes to sales, I think it’s not a one day, like you just carry a bag and you start selling after you build it. It’s like you have to, as a salesperson, you have to involve the, involve in, at least in tech, you have to involve the person, the user, from the very beginning to the very end as a salesman, so that your end product has usability.
Eric Broda (38:34):
Okay. No, I agree strongly. The, the, the concept that I’m familiar with is in order to drive change in behavior in any organization you need to, you obviously you need to do the work, but you need to socialize it and communicate it. And, and that takes a lot of work. And sometimes the data, you know, engineers view themselves as the doers as opposed to the communicators or socializers. If you were to offer Ika some advice to somebody wanting to become a data mesh engineer, how do you balance that technical with the, the, like, what’s the balance for you and how do you balance that technical versus social versus, you know, being able to sell, if you will?
Kruthika Potlapally (39:23):
So, before I started this project, I was mostly a technical person. After working with JG and Kim they gotten, they brought about a very different perspective to how we worked technical skills, a part Gigi was he, he helped us come up with good documentation. We all, it all started there. And JJs a pretty creative person. So, and a great storyteller. If you actually look at his videos the way he introduces a concept, it’s so easy just going by data mesh for all ages. You know, if you just take a look at that book, you know that you are reaching out to kids <laugh>, and, and it’s so easy to learn something in about five, 10 minutes. So I think that skillset set apart, I think we have had pretty good leaders guiding us and help us gain some of the art some of the skills when it comes to the art of storytelling and being able to pitch a product to different stakeholders.
(40:44):
Then comes when it comes to, if you’re a part of a software engineer building the data platform, I would say learn the product sense and the impact that you’re making to your users that comes with you. Build something get the feedback from your users, and then work and evolve. So knowing what, understanding the kind of sense or product sense and the kind of imp impact you’re going to create on your users, that’s very important. That, again, comes with how you pitch your product and how you work with your users. And if you’re working as part of the data transformations team, know the business as JJ and Lavena rightly mentioned, it’s very important to know what your data product is going to do, or how it is going to help before you even start developing or working on the product. So all of this requires, again, collaboration and being able to come up with your pitch about who you are catering to and what is the use of whatever you’re doing. So, and then I think it all comes down to art of storytelling as well. You need to have some kind of creative skills to be able to grab the attention of your audience. So I think presentation, skills, documentation, and continuous collaboration.
Eric Broda (42:13):
Fantastic. Now, Levina, just real quickly then, I’m gonna take a question from the audience right after this. The, the word storytelling resonates very strongly. It’s, I, I, I say slightly different words, but they mean the same thing, which is what is the narrative? What are you, you know, what is the story you’re trying to tell? And we know, what we’ve heard is that the data engineer, data mesh engineer in order to drive that change in behavior needs to be a good storyteller. Okay. So, so, so levina, how do you become a good data mesh engineer storyteller?
Laveena Kewlani (42:53):
Practice and practice and practice
Eric Broda (42:56):
And listen to jg, apparently <laugh>?
Laveena Kewlani (42:58):
Yeah. I am actually a person. I mean, I’m, I’m a camera shy person. I have stage, I, I literally had stage here and I thought, okay, it’s fine. I’ll, I’m going to work as a, you know, in the background, it’s okay, but it’s actually, sometimes it’s the push from your leaders, and it’s a push for you yourself if you’re working on something and you’re, if you, if you think you’re good at it, you should be able to, you know, sell it. So if you wanna sell it, and eventually in the first attempt, you’re not good in rephrasing the words and, you know, making this very long sentences to a short one, you need practice. So, good storytelling needs imagination, good imagination, really you know, good creativity and, and a little, a little practice for a person like me, <laugh>.
Eric Broda (43:50):
Okay. Excellent. Now that’s a great answer. I’m gonna ask a question from Rui, hopefully, I, I pronounced the name correctly. How is the person, I’ll, I’ll ask this to Tika to start, how is the person that the data mesh engineer can reach out to, to get the business domain knowledge for building a semantic layer? Should the data mesh engineer be the owner of that knowledge? Is that, is that part of the data mesh engineer toolkit?
Kruthika Potlapally (44:22):
The ownership I think is not just with respect for a data, a data master engineers building the data transformation. It also, the main ownership lies with the data product owner that works with the business and brings the data in. So I think it’s a combination, but predominantly lies with the ownership predominantly lies with the data product owner.
Eric Broda (44:48):
Okay. Levina, what do you think?
Laveena Kewlani (44:51):
I kind of think what Ika says is hundred percent, right? So the, the entire proc, the entire concept of data domain is to share the information. And when it comes to, okay, semantic versioning and all that, I think it majorly is the product owners who decide whether this versioning is semantics is correct, not et cetera, et cetera.
Jean-Georges Perrin (45:17):
Okay. I would, I would add Eric there that you, you’re not building this thing in a vacuum, okay? You, you are, you’re building it for someone. So, so your interface between the, your customer and the engineer is a, is a data product owner, but the, the cus the customer, the consumer of the da of the data you’re building on the producer of the data, you’re building as a responsibility as well. So part of the same team. So the engineer should be able to identify two the stakeholders and be able to reach out to ask for questions like when you’re de, when you’re defining the semantic layer or a meaning of some stuff. Well, data governance comes into place where you need to be able tore to reach data governance.
Eric Broda (46:16):
Okay. Feel good. I’m Ru Rui has a second question, but I’m gonna extend it a little bit. It, it, it has to do with some new technologies that may or may not have to be in the data, in data mesh engineer’s toolkit. So, so let me start with you Ika specifically Rui is asking about graph databases. What role do they play, if any, in your mind with the, the data engineer the data mesh engineer role? Is that something they should be picking up on?
Kruthika Potlapally (46:50):
If you’re working with microservices, probably, but I wouldn’t say that you can start simple if you’re building something, if you want to build a data mesh and if you’re starting out to build, you can use the same same some R D B M S databases for your micro to save your mic data from your microservices or your APIs. And yeah, you could use grabb at all, you find the need for it. But you can, for the matter, mesh can be simple post or MySQL instances. And with respect to your data transformations, if, if it depends on what your data transformation layer is going to be. It could be cloud services, it could be on-prem services. It all depends on the use case and what your storage layer is going to be.
Eric Broda (47:45):
Okay, perfect. So, so if it makes sense, then, then use it If it doesn’t, there, there may be alternate solutions. Mm-Hmm. levina, I’m gonna ask another question kind of related to that and, and, and it’s really about what, what is in the data mesh engineers toolkit? So, so we’ve heard a lot about open AI chat, you know, chat, G P T, the gp, the, the large language models. So levina, if you, if you’re, if you’re thinking about data mesh becoming a data mesh engineer, does that play is that, is that something that you ought to be thinking about? Does it fit into a data mesh engineer’s tool belt? Is it where, where, where do you see that?
Laveena Kewlani (48:24):
Okay, so if you talk about extending, so data mesh is basically a platform that provides you data products. Now utilizing that data product or creating that data product can be automatized to any level, be it using a charge G p t also. So it depends that how much capacity you have to automate stuff, or the end product use. For example, the search engine that you have at the end product, for example, if you do have it, then instead, then you can add a charge g pt, a bot, sort of something who gives you, or that, that is capable to give you the exact requirement that you know, exact product that you’re looking for, given that if there are like thousands and thousands of products on the ui, right? So that, yes, they do have a role to play in this, but again, it depends like how much capacity, how much in depth you have in your product in there. But you can start as, as Ika said, you can start simple. Like you can really start simple and then you can cascade it. That’s what we did. Like we started simple and we built it for like next six months, the entire simple concept around it. And then we added additional products. Now also, we participated in a, in a competition in PayPal, where we actually introduced mesh G P T.
Eric Broda (49:52):
Yeah, I love it. And, and, and, and I won’t ask you anything proprietary or anything, but how did it turn out?
Laveena Kewlani (50:01):
Yeah, so the decision is still pending in there, documents are in, but it was taken with a lot of enthusiasm.
Eric Broda (50:10):
Excellent. Excellent. Well, I’ll tell you, we, we are at time. So, so I, I’m just gonna close with a few question. Well, one last question I suppose for each of you. And we’ll start ika with you you know, throughout your career and now as a, you know, having arrived as a data mesh engineer any lessons learned advice for the audience perhaps that we did not cover that you’d like to offer? I think you’re on mute, by the way.
Kruthika Potlapally (50:44):
Sorry. Yeah, I think we covered a lot in the previous discussion and, and this one as well, but less lessons learned is get your, have a product sense. It’s equally important to have soft skills and work on working on communication as much as your technical skills and understand the business. Ultimately you are building a product for your business users. You can just go about building whatever you want to with the massive amount of data that we have, like in PayPal, but know what your business needs. So I think, yeah, these are our three takeaways. Excellent. My three takeaways.
Eric Broda (51:30):
Perfect. Levina, what are your thoughts? Any advice that we might have missed? So
Laveena Kewlani (51:37):
It’s more or less, same as what Ika said, it’s know your users, know the business, consider data as a product, not just, you know, data points lying somewhere in the data and in the application db they <laugh>. Like you can really do wonders with that application, DB data. The other thing is sales skills, soft skills, and a lot of practice. A lot of practice. And of course, a very important skill that Pritika mentioned previously is storytelling. So maybe like, that’s something I learned from JG is that you can really tell really complicated stuff in a very simple animation.
Eric Broda (52:19):
Very good. So, so JG as the eminent storyteller in this crew here, I’ll give you the last word. Anything we might have missed? What, what, what is your advice to that college student? Let’s say it was your son perhaps who just is entering the workforce. Let’s say that person’s been in there for two years and they wanna become a data mesh engineer. What advice would you give as a closing remark here? Oh,
Jean-Georges Perrin (52:49):
Learn Java.
Eric Broda (52:51):
There we go. I can hear you now. Learn Java, <laugh>, <laugh>. I know there’s more, there’s, there’s more. You wanna say <laugh>?
Jean-Georges Perrin (52:59):
I think, I think, I think, I think I think it’s a very interesting world. I think I’m really convinced that the future of data engineering is going through some kind of data mesh. Okay, maybe not this ation, but the next, or the evolution or something. We need to change the way we’ve been doing with data. So as you are young and not completely your, your brain is still alive and not, you know, completely atr amplified by by dozens of years or this, the tens of years of, of, of experience embrace, embrace data mesh l look at it. And this advice is, is also for more senior people. Okay? It doesn’t have to, it’s, it’s a revol. You know, the one, one thing, when I, when I read the first, when I discovered data mesh and read the papers from Jamag, she was saying, it’s a revolution.
(54:00):
I said, come on. I’m like, this is not a revolution, it’s just an evolution of everything. But I realize that over time, that peoples that have been in data eng purely in data engineering for 20, 25, 30 years, it is actually a revolution. Okay? So be be ready to embrace it because it’s, it’s really there. Okay? Whether look at, look at the data mesh learning community, okay? When I joined, we were like under 5,000 people and I think, like I was still, oh, okay. And now we are over, we are close. We, we are close to eight, 8,000 people. Okay? The, the number of, since I’ve posted on my blog at jgp.ai, some articles on, on data mesh, my traffic has almost gone tenfold. So there’s a really big interest in data mesh. The industry is going there, so embrace it, okay? Goes there. And the, the key I think is really data contract is one key data product is one key. But to make a data product, you need to understand the product you’re building. So you are a product owner, you are, you’re owning something, you are part of a business. Okay? So, so that’s, that’s my advice too to young and old people, <laugh>.
Eric Broda (55:16):
I love it. So let, let me get the, let me, let me first off say thank you to the panel participants, Levina, Ika and Jean George here. Here’s a, just a real quick summary. Here’s, here’s what I heard. So, so here’s my advice as I parse through the, the stuff that my eminent colleagues here have mentioned is, is first off, data mesh may not be revolutionary, evolutionary, it’s somewhere kind of in the middle of those. But here, here’s the thing is it’s a change in behavior, okay? And anytime you wanna change a person’s behavior, let alone an organization’s behavior it is something that is gonna take time and patience and, you know, it takes the technical skills. Absolutely. We’ve talked a little bit about those. You’ve seen, we’ve, we’ve, we haven’t necessarily emphasized the technical skills, cuz in many respects they’re table stakes. But it’s those softer skills around understanding the business, understanding how to communicate.
(56:11):
And as we’ve heard loud and clear, being able to tell a story those are the skills and capability that will help you start your data mesh. But it’ll also lets you socialize, sell it, communicate it so you can continue to its logical outcome, which would be data, cr data products across the entire organization. So, so once again, to my panel members Levina, k Ika, and Jean George, thank you very much for your wonderful insight and I look forward to having further panels with you folks in the future. So with that in mind, I will say goodbye to the audience. Thank you very much for taking the time to listen to us and hope you come back for future sessions. Bye now. Thank you. Thanks, Eric. Bye-Bye. Thank you. Bye now.