<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Data Engineering with Satyam]]></title><description><![CDATA[Documenting my journey from learning Data Engineering to building reliable, real-world data pipelines.]]></description><link>https://datawithsatyam.hashnode.dev</link><image><url>https://cdn.hashnode.com/uploads/logos/6aaa28afe3ef9403987e61a5/7c147bf3-d342-4c55-b0af-2a1145d8d9a3.jpg</url><title>Data Engineering with Satyam</title><link>https://datawithsatyam.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Fri, 25 Sep 2026 01:45:09 GMT</lastBuildDate><atom:link href="https://datawithsatyam.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Why I Chose Data Engineering]]></title><description><![CDATA[My Past
I still remember the second week of May 2017.
Our Class 12 results were everywhere, and I was nervous. My Mathematics exam had gone badly because of some family issues, so when the result fina]]></description><link>https://datawithsatyam.hashnode.dev/why-i-chose-data-engineering</link><guid isPermaLink="true">https://datawithsatyam.hashnode.dev/why-i-chose-data-engineering</guid><category><![CDATA[data-engineering]]></category><category><![CDATA[Python]]></category><category><![CDATA[SQL]]></category><category><![CDATA[#apache-spark]]></category><category><![CDATA[Career]]></category><dc:creator><![CDATA[Satyam Prajapati]]></dc:creator><pubDate>Mon, 21 Sep 2026 06:59:01 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6aaa28afe3ef9403987e61a5/04b14c1b-ac30-4510-b772-f481713f54d0.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2>My Past</h2>
<p>I still remember the second week of May 2017.</p>
<p>Our Class 12 results were everywhere, and I was nervous. My Mathematics exam had gone badly because of some family issues, so when the result finally went live, I didn't even check my marks first. I went straight to the bottom of the page.</p>
<p><strong>PASS.</strong></p>
<p>The website had taken several attempts to load because of the traffic, and I had been hitting the reload button continuously—almost like pressing the brake repeatedly in a traffic jam.</p>
<p>Once I knew I had passed, I finally checked my marks. They were pretty much what I expected. Then came the question every student eventually hears:</p>
<p>“What next?”</p>
<p>My original plan was to pursue B.Com and prepare for banking exams alongside it. But just a day before admission, someone suggested to my father that B.Tech would offer better career opportunities and placements.</p>
<p>I trusted my father's decision and enrolled in B.Tech CSE. At that time, I thought the path was simple: B.Tech → work hard → get placed → start a career.</p>
<p>But college turned out to be very different from what I had imagined. The infrastructure, location, student support and placement ecosystem were not what I expected.</p>
<h3>Discovered Data Science</h3>
<p>By the beginning of my second year, I had started becoming interested in Data Science. I wanted to understand how I could enter the field, so I looked for guidance from my college.</p>
<p>Eventually, I went to my CSE HOD. I expected some direction. Instead, he honestly told me: <strong>“Beta, yeh field toh nayi hai. Mujhe bhi zyada idea nahi hai iske baare mein.”</strong></p>
<p>That conversation was disappointing, but it also taught me something important: If I wanted to build a career in a new field, I couldn't always wait for someone else to show me the path. So I started learning on my own.</p>
<h3><strong>2021 — Starting Over</strong></h3>
<p>I graduated in 2021, right in the middle of the COVID era.</p>
<p>I started applying for entry-level IT and Data Analyst roles while continuing to learn Data Science. But most opportunities either required experience or came with another paid course promising placement.</p>
<p>Eventually, I joined one.</p>
<p>After completing the course, I realized that much of what was taught was already familiar to me. When I asked about actual interview opportunities, the "placement assistance" mostly turned into job links from LinkedIn where hundreds of people had already applied.</p>
<p>That experience taught me to be more careful about confusing learning opportunities with actual career opportunities.</p>
<p>By 2022, after almost a year of trying, I realized I needed to start working—even if the first job wasn't exactly what I wanted. I joined a technical recruitment role in Noida. It wasn't where I wanted to end up, but I kept looking for a way into technology.</p>
<h3><strong>The Startup Chapter</strong></h3>
<p>Soon after, an opportunity came through my roommate at a startup. We met the founder, heard his vision for the product, and decided to work on it. For the next six months, we worked intensely on the technical side of the product. There were long hours, sleepless nights and plenty of things I had never experienced in college.</p>
<p>Eventually, we built the product. But building a product and building a sustainable business are two very different things. The startup had limited funding, and eventually I realized that motivation alone couldn't replace a sustainable business model. Motivation can keep you working for a while. It can't pay your bills. I later joined another startup, but that experience lasted only a couple of months.</p>
<p>Eventually, I joined a startup incubator—ironically moving from being inside startups to working with them from the other side of the table.</p>
<p>I spent around 1.5 years there, working with startups, founders and technical consulting activities. I also briefly explored another direction during this period: CAT. I managed to reach the upper Tier-2 range, but my target was specifically IIMs, so I decided not to pursue that path.</p>
<h2><strong>Why I Wanted Change</strong></h2>
<p>After almost three years of working in different roles, I realized that I wasn't feeling truly satisfied or connected with the kind of work I was doing. I knew I wanted a change, but I wasn't exactly sure what that change should look like.</p>
<p>Around that time, my cousin told me about an opportunity in the government sector. Since I had already prepared for some of the subjects that overlap with CAT and government exams, I thought, <em>why not give it a shot?</em></p>
<p>So I started preparing.</p>
<p>But then something unexpected happened.</p>
<p>During a meeting with a CEO friend, I was introduced to one of his friends who was already working in the IT industry. Like I normally would, I introduced myself and talked about my education and previous experience.</p>
<p>During that conversation, he suggested something I hadn't seriously considered at that point: “Why don't you prepare for a Data Engineering role?”</p>
<p>He even told me that once I was well prepared, he would recommend my profile within his organization. I didn't know where that opportunity would lead, but that one conversation did something I wasn't expecting.</p>
<p>It reignited the interest in technology and Data Science that I had almost left behind. I started comparing my options—government exams on one side and getting back into technology on the other. And the more I researched, the more I felt that technology was the direction I actually wanted to pursue.</p>
<p>So I decided to restart.</p>
<p>I went back to the internet and started researching what skills and technologies were actually required for Data Engineering. And that's when I got another surprise. I had already studied many of them.</p>
<p>“Python. SQL. Data handling. Programming fundamentals.”</p>
<p>I wasn't starting completely from zero. I just needed to revise, practice, and get myself back into “tech mode.”</p>
<h2><strong>Discovered Data Engineering</strong></h2>
<p>After finally making up my mind to give 100% to Data Engineering, I started researching what skills and technologies were actually required in the industry.</p>
<p>The list was much bigger than I expected.</p>
<p><strong>“Python, SQL, Cloud platforms like AWS, Azure and Google Cloud, Git &amp; GitHub, Databricks, Apache Spark, PySpark, Apache Airflow, Snowflake, and many more.”</strong></p>
<p>At first, it felt like there was a lot to learn.</p>
<p>Instead of trying to learn everything at once, I decided to first identify what I had already studied, revise those fundamentals, and then gradually move towards new technologies.</p>
<h3><strong>Why It Clicked</strong></h3>
<p>I also started looking at Data Engineering not just as a collection of tools, but as a journey that data goes through:</p>
<p><strong>“Data Ingestion → Profiling → Discovery → Cleaning → Validation → Transformation → Storage”</strong></p>
<p>This simple way of looking at the field helped me organize my preparation.</p>
<p>Instead of constantly asking myself “Which tool should I learn next?”, I started asking:</p>
<p><strong>“What happens to the data, and what problem am I trying to solve at each stage?”</strong></p>
<p>And that changed the way I approached my Data Engineering preparation.</p>
<hr />
<p><strong>© 2026 Satyam Prajapati. All rights reserved.</strong></p>
<p><em>This article is based on my personal experiences and learning journey. You may share or link to this article with proper attribution, but please do not reproduce or republish it in full without permission.</em></p>
<p><strong>Published: September 2026</strong></p>
]]></content:encoded></item></channel></rss>