There was a point earlier in my journey where something clicked. A shift in how I think about building software. I don't know exactly how it happened but it opened me up to limitless possibilities. It's a mental model of seeing any software as a data pipeline.
On one hand, we take data from the users and save it in a database, mostly after some transformation. On the other hand, we take data from the database and present it to the user in one way or the other. In this way, you can think of every feature as taking data through certain steps in either direction. To the user, or from the user.
To make this concrete. You get an array of objects from an API. Each object has a category field. But the UI needs to display them grouped by category. That's a transformation. The data is the same, you're just reshaping it to match what the next step in the pipeline needs.
Once you start seeing features this way, you realise the real skill isn't knowing a framework or memorising syntax. It's knowing how to reshape data between steps.
Why data transformation matters
In software development, some data structures are better suited for certain operations than others. You hear of loops and you think of arrays. Lookups and you think of maps (or objects in JavaScript). So the ability to convert data from one structure to the one best suited for the task at hand is GOLD. Without this skill, you are limited in what you can do with data.
How to build this skill
The honest answer is data structures and algorithms. It's not an easy thing to learn. But just like learning anything else in this field, you have to get your hands dirty.
Sites like HackerRank and LeetCode give you problems to solve. If you fail to solve one, look at other people's solutions. Copy the question to any AI chat and have it walk you through how to solve such problems. And when to identify problems that require similar thinking.
If you learn best by watching and listening, there's no shortage of tutorials on YouTube teaching data structures and algorithms in your preferred language. If you learn best by reading, you can find series of articles or even have AI generate a document to teach you.
Whichever route you take, practice to confirm understanding. Without practicing there's no way to confirm you actually understand what you learnt. Any gaps may go unnoticed.
Repetition
Understanding data structures isn't so straightforward. It takes solving the same class of problems a number of times for that light bulb moment. So if you take nothing away from the resources you use, just try to remember how it was solved. You can try and solve a problem the same way as it was solved in the tutorial. But from memory. In the absence of understanding, fall back on memory.
With time you'll start spotting where one data structure may be better for your task at hand. And from understanding or memory, you will know how to get it in the format you want.