🚀Entering Tech #65: How to become a data engineer

Businesses need large amounts of data to make informed decisions. This data is often available only in raw formats and is hard to make sense of. 

Data engineers collect this unprocessed data and turn it into actionable and usable information for data scientists, data analysts, and business analysts, who use it to make business decisions. Data engineers are the unsung heroes, as they make it easier for data scientists and analysts to earn a living. 

Data engineers wear many hats! They source datasets relevant to business goals, develop algorithms to transform raw data into insights, design and maintain data pipelines, collaborate with management to understand needs, create new data validation methods and analysis tools, and ensure everything complies with data governance and security policies.

Too many tasks, you might say; remember, they are heroes, right? 

Before we go further, let’s tell the story of Adeolu Adegboye, who leveraged his data science background to transition into data engineering. Adeolu started learning data science when he thought the world was ending during the COVID-19 lockdown. In 2021, he got an internship as a data scientist and then transitioned to data engineering in 2022 because engineering had always winked at him.

*Newsletter continues after this ad.

Get more from your salary with Eazipay

‘

Join 150,000+ SMEs and employees to get up to 100% extra salaries and more this Workers’ Day! Visit www.myeazipay.com or download the Eazipay Business app to sign up to begin. Contact cx@myeazipay.com or 07000332947 for more info.

Data engineering is a purely technical field that requires programming knowledge of Python and SQL, along with skills in data modelling, ETL, data management, and data architecture. Adeolu strongly recommends that prospective engineers be versed in the different cloud computing platforms—Amazon Web Services (AWS), Microsoft Azure, and the Google Cloud Platform (GCP).

According to him, data engineering is a holistic skill that requires learning a bit about everything, from database handling to analytical reasoning, machine learning, data security, data storage, and so on.

On the soft skill rung of the ladder, problem-solving, communications and stakeholder management are the most important skills. Data engineers are required to ensure the end user/client understands the full context of what the data provided can do for them. 

If you’re wondering where you ca learn data engineering, Adeolu recommends that you follow the 100-day plan for newbies in data engineering by The Seattle Data Guy. He teaches a range of topics and provides a spreadsheet that houses over 50 courses, challenges, and materials. Adeolu claims that if followed religiously, the 100-day plan could potentially land you an internship or even a full time role if you’re a badass. 

Data professionals I have spoken with for this series say Data Camp is a safe haven, if not heaven, for data professionals. Courses on the platform are made by data professionals and cater to different phases of your learning journey. While courses on DataCamp are priced, Adeolu recommends free alternatives on Codecademy and Coursera. 

Below are some of his recommendations. 

*Newsletter continues after this ad.

The 3i Africa Summit!!!

🚀 Are you ready to be part of the FinTech revolution in Africa and across the globe? Join us, as we prepare for the unforgettable 3i Africa Summit that unites industry leaders, businesses, investors and innovators.

Venue: Accra International Conference Center, Accra, Ghana.

Date: 13th – 15th of May, 2024.

💥Register here to save your spot!! You do not want to miss out on this!!

However, if you are looking to take a stab at it right away, Adeolu claims that following the 100-day plan mentioned earlier by The Seattle Data Guy can land you an internship or an entry-level job. If you’re not planning on going through that route, we have some other tips for you. 

As we have recommended in other episodes, getting your first role as a data professional is not rocket science. The short answer is: put yourself out there, send out cold emails or messages, have a tailor-made CV, join great communities, and attend job fairs and industry events.

Adeolu also recommends documenting your learning journey—on Substack or Medium—and sharing it on social media as a surefire tip for getting noticed in the data engineering field. Applying for freelance and volunteer projects on sites like Omdena is a great way to build your portfolio. 

They say you are the average of the five people you spend the most time with. But I’d say finding the right community is like finding the perfect pair of jeans. Joseph Ojo, a data engineer at Nomba, a Nigerian fintech, says joining a community is the best way to upskill and discover new trends—and even job opportunities as a newbie. He recommends joining communities like Young Data Professionals (an invite-only community) and Data Engineering community.

To stay updated on the latest industry trends, Joseph recommends living partially on Medium and subscribing to relevant newsletters like

• Start Data Engineering
• Data Engineering Center
• ByteByteGo Newsletter
• SeattleDataGuy’s Newsletter

P.S. You can also share feedback—or a shoutout—with us about this edition by sending an email to newsletter@techcabal.com. Did we leave something out? Is there something else you’d like to read? Let us know! It helps us write even better editions.

Sign up for The Big Daily

Ready to get smarter or just hoping to sound like the smartest person in the room at your next networking event? Subscribe to The Big Daily newsletter to get filled in on the day’s most important happenings. Subscribe now → thebigdaily.substack.com.