Career Development

There is a common stereotype that software testing is just a dull exercise in checking what should already work. In reality, the cost of a missed bug in a serious product is far higher than a minor visual glitch or a button shifting out of place. It can lead to failures in critical workflows, data loss, service outages, and major financial damage for the business. People often say that QA is just there to check developers’ work. That is a superficial view. The role of QA is not simply to confirm that the code works, but to try to uncover every scenario in which it can fail. QA engineers have to think differently from the people who built the system. They need to think like real users, including those who will inevitably follow unexpected paths or use the product in ways no one originally planned. The myth that QA is routine work has lasted for years. Yes, regression testing is part of the job. But if you force an engineer into rigid checklist-driven testing, you will miss bugs. Strong QA does not follow a script mechanically. It explores the product, looks for edge cases, and finds new ways it can fail that no one considered during planning. QA Tasks and Approaches In most organizations, there is a distinction between QA (Quality Assurance) and QC (Quality Control), and both need to be considered together. It is not enough to verify the quality of the product being shipped. Teams also need to analyze the development and testing process itself, identify weaknesses, and look for ways to improve the product over time. Tasks break down into two major streams: Bugfix and regression. It is not enough to confirm in a ticketing system that a developer has fixed a bug. The goal is to make sure it does not come back a month later. That is why any critical fix should be covered by a regression test. This becomes a safeguard against product degradation.Feature testing. The work starts long before the build. Documentation and requirements are studied not just to understand how the feature is supposed to work, but to anticipate how it might break neighboring components. Depending on complexity, the right testing approach is chosen: from simple checklists to full test cases and automation where it makes sense. Good teams aim for high automation coverage, but they also stay realistic. Test coverage varies across modules: legacy code and experimental features require different approaches. In some areas, developer-owned unit tests may be enough. In others, QA needs to validate functionality through more complex integration scenarios. What Good QA Looks Like and How to Build It Good QA means stable tests that provide sufficiently complete coverage of a feature or product. It is also critical that QA and development teams actively collaborate. For example, when handing a product off to development, QA provides reports on what was tested and how, what risks remain, and what can be improved in coverage and overall product quality. It is also good practice to write tests alongside the code. For example, developers may write TAP-based functional tests for PostgreSQL or unit tests for Go code. At the same time, QA engineers prepare to test the finished product: they analyze documentation, talk to business analysts and technical product managers, and clarify expected behavior. Based on that information, QA specialists create test cases and checklists, automate them when appropriate, and add them to CI. It also helps to have a DevOps engineer who can take infrastructure work off QA’s plate. If there is no DevOps engineer, that responsibility often falls to the QA engineer as well. Manual vs. Automated Testing Automation is invaluable for repetitive tasks where manual re-checking leads to attention fatigue. In these cases, automated tests improve test quality and reduce errors caused by human fatigue or inattention. Exploratory manual testing complements automation. It helps uncover product behavior, edge cases, and unusual scenarios that automated tests may not cover. When building automated tests, test design techniques can be applied deliberately. For manual testing, the engineer's own experience matters most: you need to look at the product broadly and systematically, analyze what affects its behavior, and try to break it from angles nobody thought to consider. For example, one of my colleagues once tried to bring up a cluster in a nonstandard way by skipping one step, and the system failed. A user may follow their own path, and the developer simply may not have accounted for it. From an exploratory testing perspective, it is valuable to have custom tooling for fuzzing and generating random input combinations. For example, an in-house combinator can shuffle files with SQL queries to produce unexpected execution paths and input combinations. ASan is commonly used for detecting memory errors, while Valgrind is widely used for memory debugging, leak detection, and profiling. For web application testing, Selenium Grid supports cross-browser execution at scale. For automated tests, Pytest can serve as the test execution framework, while Allure Report generates CI-integrated reports from the test results. TestRail handles test case management and coverage tracking at the team level. Testing Against Open Source and Third-Party Components QA becomes more complex when a product is built on top of an open-source core or depends heavily on external components. In that case, the team is not testing only its own code. It is also testing how upstream changes, extensions, integrations, and internal modifications behave together. For example, a product based on PostgreSQL may include its own commercial logic while still pulling changes from the upstream project. That changes the QA process. If a bug is found in upstream code, quietly patching it only inside the commercial product is not always the right approach. In many cases, the issue should be reported back to the community, especially when the defect affects the original project and not only the commercial layer. The same logic applies to extensions and external components. Before adding a third-party component, QA should treat it as untrusted until it has proven stable in the target environment. That means exploratory testing, compatibility checks, regression coverage, review with SCA tools, and verification under realistic workloads. If bugs are found, they should be reported to the maintainers, and the component should be added only after the fix is available and the behavior is stable enough for production use. Upstream bugs are often harder to investigate than internal code defects. With internal development, the team usually has full visibility into every commit. If something breaks, tools such as git bisect can quickly point to the change that introduced the regression. With upstream code, changes often arrive in larger batches. When something fails after a merge, QA and developers may need to analyze a much larger set of external changes before they can isolate the root cause. Communication works differently, too. With internal code, the developer who owns the change may be one message away. In an open-source project or a third-party component, the feedback loop is longer. Reports need to be reproducible, technically precise, and useful to maintainers. This is where strong QA becomes more than bug reporting. It becomes engineering communication. AI and Vibe Coding: Hype or Genuine Value? Vibe coding is having a moment: you describe a task to an AI, and it writes the code for you. In QA, this can work, but with some important caveats. The most practical current use cases are generating sets of basic test cases (for example, pairwise combinations) and writing small helper scripts. This helps remove the “blank page” problem and saves engineers from some routine work. But it is important to understand the boundary. AI is like a junior engineer who never gets tired but often hallucinates. It cannot reliably anticipate deep architectural specifics of a product or subtle edge cases. At some point, a human must step in. Otherwise, the vibe can end with a critical bug in production. How to Become an Effective QA Engineer and Grow in the Profession At university, testing is typically covered as part of a programming curriculum. That is enough to learn the basics and understand whether the field interests you. After that, you can continue learning independently or take dedicated QA courses. For many serious QA roles, strong Python skills are one of the most useful foundations for automation, along with a solid understanding of Linux. This is especially true in systems development: you simply cannot test a complex product effectively if the terminal intimidates you or you do not understand how the environment works. But hard skills alone are only tools. The key soft skills for QA are systems thinking and the ability to make a clear technical argument. Finding a bug is not enough. You also need to explain why it should be fixed and make the case to the developer. This is not about avoiding conflict. It is about clear, constructive communication in a shared technical language. Where Can QA Engineers Grow? Technical track: QA Architect or Lead SDET. This path means continuous growth in coding, test architecture, and CI/CD. You become the expert who can build a quality engineering infrastructure from the ground up. Management track: Team Lead or Head of QA. Here, the focus shifts to processes, hiring, and mentoring. The idea that managers do not need deep technical knowledge is dangerous. To lead a team of engineers, you need to understand their pain points and the complexity of their work. And yes, some QA engineers eventually move into development or analysis. Testing experience becomes a real advantage there: engineers with a testing background tend to write cleaner code because they can see in advance where it is likely to break. Useful Resources A short list of resources to help you get started in QA and stay current with what matters: How Google Tests Software by James Whittaker, Jason Arbon, and Jeff Carollo: A useful book on how testing can be organized at scale, with strong ideas around risk, ownership, automation, and engineering culture. Lessons Learned in Software Testing by Cem Kaner, James Bach, and Bret Pettichord: A foundational text covering practical testing wisdom that holds up regardless of stack or methodology.ISTQB: Not every strong tester needs certification, but ISTQB is very useful for learning shared testing terminology, test design basics, and the difference between QA, QC, verification, validation, test levels, and test types. ISTQB materials are especially helpful for beginners who need structure. Ministry of Testing: One of the best-known global testing communities, with articles, discussions, courses, events, and practical material on quality engineering, exploratory testing, automation, and modern QA careers. Test Automation University: A practical resource for learning automation, API testing, UI testing, CI integration, and related engineering skills. Final Thoughts QA requires a lot of patience and focus, but knowing when to switch context matters just as much. Sometimes that is exactly what helps you solve a problem after spending a week chasing an intermittent failure with no clear result. A solution can almost always be found. Do not forget physical activity or anything else that lets your brain step away from the task for a while. It gives you the energy to come back and look at the problem from a different angle.

By Alex Vakulov CORE

You learned prompt engineering. You built a chatbot. You finished a course. You added “GenAI” to your LinkedIn headline. And still, the interviews go nowhere. That does not always mean the AI job market is fake. More often, it means your signal is weak. Most candidates are showing that they can use AI tools. Employers are trying to hire people who can make AI useful inside a messy business, with real customers, bad data, edge cases, budgets, and risk. That is a very different standard. I have spent more than two decades working on systems where loose thinking becomes expensive very quickly. In enterprise platforms, weak architecture creates operational pain. In AI systems, weak judgment creates confident mistakes. That is why companies are not just hiring “people who know AI.” They are hiring people who can make AI dependable, measurable, and worth the cost. AI and big data top the list of fastest-growing skills. Source: World Economic Forum, “The Future of Jobs Report 2025.” The opportunity is real. But the winning profile is narrower than most people think. The Real Gap Is Not Learning. It Is Proof. A lot of job seekers think the market wants more certifications. It usually does not. What employers actually want is proof that you can take a vague business problem and turn it into a reliable AI workflow. That might mean improving support response quality. It might mean extracting fields from invoices. It might mean enriching product data. It might mean helping internal teams search better across thousands of documents. In every case, the question is the same: Can you turn AI from a cool demo into useful work? That is the hiring filter. Prompting Is Not the Skill. Precision Is. People still talk about prompt engineering as if it is a magic trick. It is not. The real skill is writing clear instructions for messy real-world work. Prompt engineering is the process of writing effective instructions for a model. Source: OpenAI, “Prompt engineering.” That sounds basic, but most candidates still operate at the level of vague intent. For example, “build a support bot” is not a serious instruction. A stronger version sounds like this: Handle password resets, order status checks, and return requests. Escalate angry customers. Do not invent policy. Log the reason for every escalation. Use only approved support content. That is not fancy. It is clear. And clear is valuable. This is one reason many smart people struggle to land AI jobs. They have learned how to ask AI interesting questions. They have not yet learned how to define work so precisely that a machine can do it safely and repeatably. That skill matters in AI engineering, AI product management, AI operations, AI consulting, and AI strategy. If you can define success clearly, you immediately become more hireable. Pretty Output Is Not the Same as Correct Work AI has a dangerous habit. It often sounds right before it is right. That is why evaluation matters so much. Evaluations (evals) are a way to test your AI system despite this variability. Source: OpenAI, “Evaluation best practices.” In plain English, this means one polished answer is not proof of quality. A summary can read well and still miss the legal risk. An invoice extractor can look accurate and still miss tax values. A product recommender can sound helpful and still suggest the wrong item. This is where many candidates lose credibility. They show outputs. They do not show checks. A stronger portfolio piece does not stop at “here is my AI app.” It says: Here is the task. Here is how I measured success. Here is where the system failed. Here is what I changed. Here is what still needs human review. That instantly feels more senior. If you want to stand out in AI hiring, start reviewing AI output as if your name is on it. Because in production, it will be. A Good AI Builder Can Break Work into Steps Another missing skill is decomposition. Can you take a big, fuzzy workflow and split it into smaller steps that AI can handle well? That is what real projects need. Take product catalog enrichment. A weak candidate says, “I built a product content generator.” A strong candidate says: First, classify the product. Then pull trusted attributes. Then draft the copy. Then check factual consistency. Then route uncertain cases to human review. That is a very different level of thinking. The same is true in support, search, compliance, reporting, and internal tooling. Employers are not paying for random prompt collections. They are paying for people who can structure work. And this is good news for job seekers. Because decomposition is not reserved for machine learning researchers. It is also a skill used by architects, product managers, QA leads, technical writers, analysts, and operations people. Many people are closer to AI work than they think. Trust Is Part of the Job Now There is another reason companies hesitate. Some AI tasks are easy to reverse. Others are not. A bad draft can be edited. A bad wire transfer cannot. A weak product description may annoy a customer. A wrong financial or medical recommendation can do real damage. That is why responsible deployment is now part of the skill set. Help manage the many risks of AI and promote trustworthy and responsible development and use of AI systems Source: NIST, “Artificial Intelligence Risk Management Framework (AI RMF 1.0).” The candidates who look strong in this market are the ones who naturally ask questions like: What is the cost of error? How often can this fail? Can we verify the result? Where should a human approve the outcome? What should the model never be allowed to do alone? This is not just a compliance issue. It is a product skill. It is an engineering skill. It is a leadership skill. And it is one of the clearest signals that someone understands production AI rather than just experimental AI. If the Economics Fail, the Idea Fails One more thing separates a hireable candidate from an enthusiastic learner. Business math. Not every workflow deserves the biggest model. Not every AI feature deserves to exist. Model choice and usage patterns directly affect cost, and official API pricing pages make that tradeoff visible across tiers and token categories. That means a serious AI professional should be able to say: This task is simple, so use a cheaper model. This task is high-stakes, so pay for a stronger one. This workflow runs at scale, so measure the cost before rollout. This use case saves time, but not enough money to justify production. That kind of thinking changes how hiring managers see you. Now you are not just someone who can build with AI. You are someone who can make a sound decision with AI. What Hiring Managers Actually Want to See If you have learned AI but have not landed a job, stop asking, “What course should I take next?” Start asking, “What evidence would make an employer trust me?” A strong answer usually includes one real workflow and five kinds of proof: A clear business problem. A precise task definition. A simple evaluation method. A sensible review process for risky cases. A rough explanation of cost and value. That is enough to make a portfolio piece feel real. Not flashy. Real. And real wins. The AI job market is not only looking for people who can talk to models. It is looking for people who can think clearly, reduce ambiguity, catch mistakes early, and connect technical work to business outcomes. That is why many candidates feel stuck. They are training for “using AI.” Employers are hiring for judgment. Once you understand that, the path changes. Build fewer demos. Show more decision-making. That is how you stop looking like someone chasing AI jobs, and start looking like someone ready to do one. For more practical insights on AI careers, software architecture, and building production-ready systems, connect with Faisal Feroz on LinkedIn and read more on his blog.

The ATM Didn't Kill Bank Tellers' Jobs There's a story economists love to tell about ATMs and bank tellers. You've probably heard it. When ATMs were introduced in the 1970s, everyone predicted they would eliminate teller jobs. They didn't. By the 2000s, there were actually more tellers than before the ATM existed. The story became a load-bearing parable for anyone who wanted to argue that technology doesn't kill jobs, cited by economists like Daron Acemoglu and David Autor, by tech executives like Eric Schmidt, and more recently by politicians reaching for reassuring historical analogies when asked about AI. David Oks recently published a sharp piece that complicates this parable. His conclusion: ATMs didn't kill bank tellers, but the iPhone did. Teller employment entered prolonged decline in the 2010s because mobile banking made the branch itself irrelevant. Once customers stopped coming in, the institutional context that gave the teller role its value simply ceased to exist. Oks draws a clean distinction from this: it is paradigm replacement, not task automation, that actually displaces workers. The ATM tried to fit capital into a labor-shaped hole. The iPhone changed the shape of the hole entirely. It's a compelling argument. And for software developers watching coding agents write functions, generate tests, and draft pull requests, it offers a certain comfort: you're not being replaced, just assisted. The real disruption, if it comes, will look completely different. I think that comfort is premature. Oks argument in itself is not wrong, but it is incomplete and the part it leave out matters a lot. The iPhone Argument Has a Gap Oks is right that the iPhone, not the ATM, killed the bank teller job. But his broader thesis that paradigm replacement is the mechanism of displacement, not task automation, isn’t historically accurate. Manual weavers weren't displaced by a paradigm shift. The power loom did the same thing they did, faster and cheaper, and they became redundant. The technology simply fit itself into a labor-shaped hole and the labor disappeared. Scribes, typing pools, telephone switchboard operators, elevator operators, etc. none of these occupations required a paradigm shift to be eliminated. In each case, task automation was sufficient. The job existed because a specific set of tasks had economic value. When those tasks could be performed more cheaply by a machine, the role contracted and eventually disappeared. What Oks has actually identified is a specific condition under which task automation fails to eliminate a role. That condition has two components: the automated task must be embedded in a broader human interaction that has independent value, and the cost reduction from automation must enable expansion of the overall activity. Both conditions held for bank tellers. The teller's residual tasks such as relationship management, complex service interactions and cross-selling financial products were genuinely valuable and hard to automate. And cheaper branches meant more branches, which meant more tellers. Demand expanded to absorb the efficiency gains. When those conditions don't hold, task automation alone is sufficient to eliminate a profession. The question for software development is which set of conditions applies. Where Software Development Sits Right Now At this moment, the conditions for software developers look more like bank tellers than manual weavers. But only just, and the gap is closing. Current coding agents are genuinely capable at specific set of tasks: writing boilerplate, generating tests, documenting code, drafting pull requests, handling well-specified implementation problems. These are real and substantial parts of software work. But they remain components of something larger. The tasks that define senior software development include understanding what problem is actually worth solving, navigating the organizational and technical tradeoffs in an architectural decision, holding the accumulated context of a system's history and making decisions about about second-order consequences. Current AI tools handle poorly these activities. This is why the most experienced engineers are the heaviest users of these tools. Staff+ engineers, according to recent survey data from The Pragmatic Engineer, use AI agents at higher rates than any other level. They're not threatened by the tools; they're amplified by them. Their judgment is the thing the tool can't replace, and the tool makes their judgment more productive. In Oks' terms: the automated task is still a component of a richer service relationship, and the remaining human tasks still have genuine value that can’t be easily automated. There's a second reason the current moment feels stable, and it's more structural. Almost everything about how software development is organized today was designed for humans working at human speed. The way we gather and surface production issues. The way we structure pull request reviews. The way we run sprint planning, write tickets, conduct incident postmortems. These workflows were built around human cognitive limitations and human communication patterns. Dropping a coding agent into them is like replacing a factory's steam engine with a single large electric motor and keeping the drive shaft: you get some efficiency gains, but you're not anywhere near the real potential of the technology. This means we're still in early transition. The friction isn't just technical capability; it's organizational inertia. And as long as workflows are designed for humans, the human remains load-bearing in ways that aren't purely about cognitive ability. Why the Trajectory Matters More Than the Current State Here's where the bank teller analogy breaks down in a way that should concern developers. ATMs had a hard ceiling. They could handle cash withdrawals, deposits, and balance checks. They couldn't have a conversation, make a judgment call, or handle anything outside their programmed transaction set. The teller's residual role was structurally protected by what the machine couldn't do. AI coding tools don't have an obvious ceiling. The tasks they handled poorly six months ago they handle better today. The tasks that seem safely human (system design, architectural judgment, understanding business context) are precisely what the next generation of tools is being explicitly built to address. The boundary between "what AI can do" and "what requires human judgment" is moving. The debugging agent is a useful case study here, because it illustrates something more significant than incremental improvement. Traditional observability and debugging workflows were designed around humans: a log file, a Slack thread, an on-call engineer piecing together a causal chain under pressure at 2am. The first generation of AI debugging tools tried to fit into that workflow: read the logs, suggest a fix, slot into the existing process. What's emerging now is different. Debugging agents designed from the ground up to collect data in machine-readable formats, pre-correlate signals before a human ever sees them, deduplicate noise, and surface a ranked hypothesis rather than a raw stream of events. This is not task automation slotted into an existing workflow. This is the workflow being redesigned around what machines need rather than what humans can do. It's the first concrete step toward something that would have sounded like science fiction three years ago: systems that close the loop on their own failures, with humans in an oversight role rather than a diagnostic one. The human's job shifts from "figure out what broke and why" to "decide which failure modes matter and whether the agent's response is appropriate." That's a real and valuable role. It's also a much smaller one. This pattern will repeat. PR reviews are being rethought not as human checkpoints with AI assistance, but as automated verification layers with human escalation paths. Architectural decision-making is starting to be approached with tools that can compare the tradeoffs made across a codebase. The demand elasticity argument (there's always more software to build, so developers will always be needed) deserves a direct response, because it's the most common counterargument and the most misleading. It's true that the demand for software is enormous and largely unsatisfied. New markets are opening: small businesses that couldn't afford custom software at previous price points now can. There is genuinely more to build than we're currently building. But new demand mostly benefits new categories of developers serving new markets: the solo developer building custom tools for local businesses, the non-specialist who can now direct agents without deep technical knowledge. It doesn't necessarily sustain employment at current levels in the enterprise and product development tiers where most professional developers currently work. When a team of five engineers with AI agents can do the work that previously required fifty, the question is whether organizations respond by building ten times as much software or by not rehiring the other forty-five. The historical pattern in most industries, including banking after the iPhone, is the latter, at least in the short to medium term. What This Means The structural conditions for developer employment look reasonably healthy for the next several years. The remaining human tasks are real and valuable. The workflows haven't been redesigned yet. The organizational inertia is substantial. But the trajectory is not ambiguous. The tasks that AI handles poorly today are the explicit targets of the tools being built right now. The workflows are starting to be redesigned from the ground up rather than retrofitted. The "richer service relationship" that protects the developer's role is narrowing as the agent takes on more of what previously required human judgment. The compression threat is more immediate and more certain than the paradigm-replacement scenario, but both are real. In the near term, the result will be the same output from smaller teams. In the longer term, "software development" as a distinct full-time profession organized around the ability to write and make decisions about code may dissolve into something else: a capability that technical knowledge workers exercise incidentally, the way managers today use Excel without being spreadsheet professionals. Oks ends his piece with a distinction that's worth noting: the ATM substituted tasks, the iPhone made them irrelevant. The people reassuring developers by citing the ATM story may not have noticed that we are already in the iPhone moment. Change in technology-driven labor markets follows a familiar pattern. It happens gradually, incremental improvements, partial automation, roles that adapt and absorb, and then all at once, when the paradigm finally shifts and the institutional context that made a role economically legible simply ceases to exist. We're in the gradual part. The "all at once" is not scheduled, but it's on the roadmap.

By Thomas Johnson CORE

Two to three years ago, the narrative surrounding AI in software engineering was quite simple: Inevitably, LLMs and AI tools would improve so much that they would replace most of the engineering workforce, including senior engineers. LLMs would write production-grade code, and organizations would need little more than a couple of product owners, a carefully crafted prompt, and a deploy button. Entire conferences were held around this concept, LinkedIn and Tech YouTube were drowning in "doomsday" posts. This prediction aged quite interestingly. What actually happened was both surprising and uncomfortable. Yes, AI tools can generate fairly good code at an extraordinarily fast pace, but the bottleneck in software delivery did not disappear. It shifted onto the shoulders of senior engineers, and the compounding effects of that shift are only now becoming visible. The Bottleneck Moved Here's the uncomfortable arithmetic of AI-assisted development in 2026. AI coding tools now write roughly 41% of all new commercial code, according to recent industry analyses. Feature velocity is at an all-time high, and yet experienced developers report spending more time debugging, more time in code review, and more time untangling architectural decisions that no human actually made. A 2025 study by METR found a striking disconnect: Developers using AI tools felt approximately 20% faster, but their measured task completion time was actually 19% slower on real-world codebases. The gap between perception and reality is nearly 40 percentage points. Two cognitive biases explain this: automation bias (where we overtrust automated output) and the effort heuristic (where less typing feels like less work). The code generation part was never really the hard part. Understanding what you have built, why it behaves the way it does, and what will break when you change it — that was always the bottleneck. AI made the fast part faster, but it also made the slow parts dramatically slower. Cognitive Debt: The Invoice Tech Companies Are Ramping Up There is a term gaining traction in engineering circles that captures this phenomenon precisely: cognitive debt. Unlike technical debt, which lives in the codebase and can be measured with linters and static analysis, cognitive debt lives in the minds of the developers working on the system. It is the growing gap between the amount of code that exists and the amount any human genuinely understands. Addy Osmani described this as comprehension debt: the hidden cost that does not show up in velocity metrics. The codebase looks clean, tests are green, and PR counts are up, but underneath those reassuring dashboards, parts of your system are running in production that no one on the team can explain. Either because that code was written by a non-human, or because the human that written it got laid off and replaced by an LLM. To be totally fair, this phenomenon existed even in the "pre-AI era." I have seen this firsthand in large enterprise environments — some microservices deployed to production where, due to team restructuring and layoffs, the people who developed the code are gone, and the people left maintaining it never built a mental model of how it works. The current tools are making this far worse. The organizational assumption that reviewed code is understood code no longer holds- engineers are approving code they did not fully understand, and that approval now carries implicit endorsement, thus quietly distributing liability. Margaret-Anne Storey, whose February 2026 research helped popularize the term, put it plainly: A program is not its source code. A program is a theory that lives in the minds of the developers. When AI generates the implementation and humans merely review it, that theory can fragment or disappear entirely. And when it does, even simple changes become dangerous. The Review Bottleneck Is a Senior Engineer Bottleneck So, who is left holding the system’s mental model together? The senior engineers. The ones who remember why that architectural decision was made under pressure eight months ago, who can look at a diff and immediately know which behaviors are load-bearing and which are cosmetic, who can tell the difference between a safe refactor and one that will quietly shift something users depend on. These engineers are now the scarce resource the entire organization depends on, and here is the irony: the same wave of AI adoption that increased the volume of code requiring review also triggered the layoffs that thinned the ranks of the people capable of reviewing it. Between 2023 and early 2026, the tech industry shed hundreds of thousands of jobs. Over 245,000 globally in 2025 alone, with 2026 on pace to exceed that figure. But the layoffs were not evenly distributed across seniority levels. Companies cut junior and mid-level roles aggressively, believing AI tools could absorb the work. The remaining senior engineers did not get a lighter workload. They got a heavier one, with fewer people to delegate to. This is what burnout looks like in 2026 — a slow erosion. Engineers who stop pushing back in design reviews because they do not have the energy, code reviews that become rubber stamps, and architectural choices made by default rather than deliberation. The people most likely to burn out are the people hardest to replace. And it gets worse.... The Broken Pipeline: Where Are Tomorrow’s Seniors Coming From? This brings us to the part of the story that keeps me up at night. If the current senior engineers are overwhelmed and burning out, or worse, laid off, the natural question is: who replaces them? The answer, increasingly, is nobody. Entry-level developer hiring has collapsed. Ravio’s 2025 Tech Job Market Report found that entry-level hiring dropped 73% year over year, while overall hiring dipped only 7%. This is a deliberate strategic decision playing out across the industry. In 2019, new graduates represented 32% of Big Tech hires. By 2026, that number has cratered to roughly 7%. The pipeline narrows invisibly. The reasoning from a CFO’s perspective is straightforward: why pay a junior developer $80–100K plus six months of ramp-up when a senior engineer with AI tools can cover triple the output? The math makes sense on a quarterly earnings call; it is catastrophic on a five-year horizon. Schools do not (yet) produce senior engineers - experience does. Debugging someone else’s code teaches you how systems fail, while writing boilerplate teaches you how systems are structured. Reviewing pull requests, even if stressful at first, teaches you how other people think about problems. Every one of those learning opportunities is a task that AI now handles, or that simply does not happen because there is no junior on the team to do it. A significant reduction in junior hiring between 2024 and 2026 means a proportional reduction in candidates for senior roles between 2031 and 2036. The industry is eating its seed corn. The Easy Button and the Erosion of Sharpness There is another dimension to this that extends beyond organizational hiring strategy and into individual skill development. When a tool exists that can do your job (at least the visible, measurable parts of it) for you, people will use it. It is human nature to be lazy. But consider what happens at the individual level. A mid-level engineer who has leaned heavily on AI code generation for two years stops building the neural pathways that come from working through problems manually. They lose the 30 seconds of working memory where they would have wired together the algorithm, considered edge cases, and built a mental anchor for that pattern. Multiply that across hundreds of completions per week, and the atrophy becomes significant. Luca Rossi describes two cognitive modes that matter here: create mode, where you actively build mental connections between ideas, and review mode, where you assess existing work with lower cognitive engagement. AI tools push developers from create mode into review mode by default. You stop solving problems and start evaluating solutions someone else produced. Review mode feels productive - you are reading code, spotting issues, making edits, but you are not building the mental model that lets you reason about the system independently. All of this is happening while the bar for what it means to be a qualified engineer is rising. FAANG interviews in 2026 have shifted from pure algorithmic puzzles toward scenario analysis, debugging exercises, and system reasoning under realistic constraints. Companies are looking for signals that cannot be autocompleted: the ability to read logs, investigate a performance regression, and explain why a request path suddenly slows under load. The interview process is selecting for exactly the kind of deep understanding that routine AI-assisted work erodes. The paradox is stark. The tool that was supposed to democratize software engineering is simultaneously making it easier to produce code and harder to develop the judgment needed to produce it well. A Humble Prediction: The Market Will Bifurcate If current trends continue (and every indicator suggests they will accelerate), the software engineering talent market is heading toward a painful bifurcation. On one side, a shrinking pool of senior engineers with genuine system understanding, architectural judgment, and the ability to reason about code they did not write. These people will command premium compensation and face relentless demand. They will also face relentless cognitive load because the organizational layers that used to absorb complexity beneath them are gone. On the other side, a growing population of developers who entered the profession during the AI era, who are proficient at prompting and reviewing but who never built the foundational mental models that come from years of hands-on struggle with real systems. They will be productive in narrow contexts and fragile in novel ones. They will pass AI-assisted coding assessments and struggle in incident rooms. The gap between these two groups will widen because nothing in the current incentive structure encourages closing it. What Can Be Done I do not pretend to have a complete playbook for this. But I believe the conversation needs to start in a few specific places. First, engineering organizations need to stop measuring AI adoption purely through velocity metrics. If your team is shipping 40% more code but your senior engineers are rubber-stamping reviews because they are overwhelmed, you have not improved; you have accumulated invisible debt that will come due at the worst possible moment.Second, the industry needs to redefine what a junior developer role looks like in 2026. The entry-level work is no longer writing boilerplate, but reviewing AI output, testing edge cases, writing better prompts, and building the judgment that AI cannot provide. The junior developer of 2026 looks different from the one we hired in 2018, and our job descriptions, onboarding, and expectations need to reflect that. But the role MUST exist. Eliminating it is organizational and market-wide amnesia in the long run.Third, individual engineers (at all levels) need to be honest with themselves about whether their daily workflow is building understanding or just building output. If you cannot explain a function to a colleague without referencing the prompt that generated it, you do not understand it well enough to own it, and that gap will inevitably catch up to you. AI tools are revolutionizing the software development space, but contrary to popular belief, AI did not replace senior engineers; it made them irreplaceable, overloaded, and increasingly alone. Organizations and engineers need to be aware of the hidden effects AI-assisted coding has and adapt accordingly.

There's a class of failure that doesn't page anyone. No SLO breaches, no latency spikes, no 3 AM Slack messages from an on-call engineer clutching cold coffee. The system is working — by every conventional measure it's healthy — and yet something is deeply wrong. Money is hemorrhaging out of the infrastructure at a rate that won't become visible until the CFO opens a billing dashboard, squints at a number that seems obviously misformatted, and then realizes with a specific, cold dread that it isn't. This is what runaway cloud spend actually feels like from the inside. Not an explosion. A slow bleed mistaken for normal circulation. I've watched this happen to teams that were, by all accounts, technically sophisticated. Engineers who could discourse fluently on consensus algorithms and distributed tracing, who had meticulous runbooks and well-tended Grafana boards — and who had absolutely no instrumentation on what their systems cost per request. The money question was someone else's problem. Finance's problem. The CFO's problem. Right up until it became everyone's problem simultaneously, in a conference room, with a spreadsheet nobody had any good answers for. The SaaS startup whose AWS bill doubled to $500,000 in a single month didn't have a cloud problem. They had an instrumentation problem wearing a cloud problem's clothing. Orphaned virtual machines — instances spun up for a load test, or a one-off migration, or some experiment that concluded months ago — sitting there, billing hourly, invisible because nobody had tagged them to a team or a project or a cost center. Reserved-instance coverage that looked adequate in aggregate but had grown misaligned with the actual workload topology. The machines doing real work were on-demand; the reservations were funding a ghost fleet. This isn't exotic negligence. It's the default state of systems that grow faster than their accounting practices. The $2.4 million cloud bill where 80% of charges were data egress — that one is almost elegant in how completely it exposes a conceptual failure. Egress fees are the tollbooth you forget exists until you've already driven through it ten thousand times. Architects design for compute. They think in CPUs and memory and IOPS. Network transfer is ambient, assumed-cheap, treated as infrastructure rather than metered consumption. But cloud providers have always made money on the exits. Data flowing inward is free or nearly so; data flowing outward is where the revenue hides. A system that fetches large payloads, transforms them, and then ships the results to another region or a third-party analytics endpoint can accumulate egress charges that dwarf its compute costs — and nothing in the default monitoring stack will tell you this is happening until the invoice arrives. The deeper pathology here is architectural, and it predates cloud computing entirely. Distributed systems were designed by people who had to fight for every byte of memory and every millisecond of CPU time. Scarcity was the operating assumption. The engineering culture that emerged from that constraint treated resource efficiency as a first-order concern — you measured it, you optimized it, you were embarrassed when your code was wasteful. Then the cloud arrived with its promise of elasticity, its pay-as-you-go rhetoric, its infinite-seeming provisioning capacity, and something in the collective engineering psyche decided that scarcity was solved. Spin up what you need. Scale to meet demand. The infrastructure will handle it. This was always a category error. Elasticity is not abundance. It's the ability to acquire resources quickly, which is genuinely useful — but those resources still cost money, real money, money with line items and quarterly reviews attached to it. The "elastic" metaphor implies that the system returns to its original state, like a rubber band. Most autoscaling configurations do the opposite: they scale out aggressively and scale in lazily, because the engineers who configured them were optimizing for availability, not for cost. Of course they were. Availability failures page you. Cost failures invoice you three weeks later. This asymmetry in feedback latency is, I'd argue, the root cause of most cloud waste. You feel a reliability failure immediately, in your monitoring, in your error rates, in the angry emails from customers. You feel a cost failure at month-end, abstracted behind aggregates and allocation reports, at a distance from the specific code that caused it. The causal chain is so long and so obscured that attribution becomes genuinely difficult. Which service? Which deployment? Which query that suddenly started doing full table scans because someone dropped an index? You're doing forensic accounting on systems that didn't bother to leave evidence. Consider what actually happens inside a Lambda-based microservice when the retry logic goes wrong. A downstream dependency starts returning 429s — rate limiting, legitimate, expected under load. The Lambda function catches the error, implements exponential backoff, retries. Fine. Normal. But the backoff parameters were configured for a dependency that's usually briefly unavailable, not one that's rate-limiting at scale, and the jitter is insufficient, so you get retry storms: dozens of function instances all backing off to similar intervals, all hammering the dependency in synchronized bursts, all being rejected, all retrying again. Each invocation is cheap individually — fractions of a cent, execution measured in milliseconds. But you're running thousands of them simultaneously, each one burning GB-seconds of memory while waiting on a backoff interval, and the function is stateless so there's no circuit breaker state shared between invocations, and AWS will happily keep invoking your function at full concurrency because from its perspective, demand is high and capacity is available. Nobody gets paged. The error rate might actually look acceptable — most requests eventually succeed. Latency is elevated but within the p99 SLO. Meanwhile the bill for this three-hour incident is climbing toward what would normally be a week's worth of Lambda spend. This is the failure mode that the "cost as SLI" framing is trying to address, and it's worth being precise about what that means mechanically. A Service Level Indicator is a measurement of some property of the service's behavior. Latency, error rate, throughput — these are the canonical SLIs because they directly reflect the user experience. Cost doesn't appear in that list because it doesn't affect the user, not directly. But cost does reflect system behavior in ways that the other SLIs might not. A function that's executing correctly but expensively is exhibiting a real defect. The defect is just measured in dollars instead of milliseconds. Define it concretely: cost-per-request, tracked as a rolling average with a time window short enough to catch anomalies before they compound. For a Lambda function handling API traffic, this is derivable — you know the invocation count, you know the GB-seconds consumed, you know the memory configuration, you know the egress bytes. The math isn't complicated. What's missing in most stacks is the pipeline to compute it continuously and route it somewhere actionable. YAML - alert: CostPerRequestAnomaly expr: | ( increase(cloud_spend_dollars_total{service="payment-processor"}[30m]) / increase(http_requests_total{service="payment-processor"}[30m]) ) > 0.02 for: 15m labels: severity: warning annotations: summary: "Payment processor cost/request exceeding $0.02 threshold" runbook: "https://wiki.internal/runbooks/cost-anomaly" The alert above is simple to the point of being naive — a real implementation needs to handle the edge cases around division-by-zero when request volume drops, needs to account for fixed-cost components that don't scale with traffic, needs to be tuned per-service rather than applying a uniform threshold. But the principle is sound: instrument cost the way you instrument latency. Put it in the same pipeline. Give it the same alerting treatment. Let it page someone. Zombie resources deserve particular attention because they're so easy to dismiss as solved problems that keep not being solved. The inventory of forgotten things in a mature cloud environment is, in my experience, always larger than anyone expects. Unattached EBS volumes, left behind when instances were terminated but the delete-on-termination flag wasn't set. Elastic IPs not associated with any running instance, costing $0.005/hour each — individually trivial, collectively real money at scale. NAT Gateways in regions where you decommissioned the VPC workloads but left the gateway standing because the Terraform state wasn't cleaned up and touching Terraform state makes everyone nervous. RDS snapshots accumulating indefinitely because the backup retention policy was set aggressively and nobody wrote the cleanup job. Elastic Load Balancers with no healthy targets, passing health checks against nothing, billing for capacity they're delivering to nobody. The cumulative drag of this kind of waste is hard to calculate but easy to feel when you run the audit. It's almost never catastrophic individually. It's ambient cost noise that compounds month over month, gradually shifting the baseline upward so that each budget cycle starts from a floor that's slightly higher than the last one, and nobody can quite pinpoint why the efficiency curve keeps drifting in the wrong direction. Tooling exists for this — AWS Trusted Advisor, Compute Optimizer, the idle resource detection in Cost Explorer — but tooling that generates recommendations is only useful if someone's job is to act on them. That organizational detail is where most cost hygiene programs quietly fail. The recommendations accumulate in a dashboard somewhere. Engineers see them, acknowledge them, add them to a backlog, and then prioritize the feature work that someone with authority is actually asking for. The idle resources survive because their survival costs nobody anything immediately measurable. The fix isn't more tooling. It's accountability, specifically the kind that creates immediate feedback. Tag enforcement at resource creation — if you can't create a resource without a team tag and an environment tag and a project tag, the tagging happens. Automated cleanup of untagged resources after a grace period — not a suggestion, an actual termination, which focuses attention remarkably. Chargeback rather than showback: show teams what they're actually being charged for their cloud consumption, real money against real budgets, not just informational usage reports that feel abstract because no actual transfer occurs. Showback is useful. Chargeback is clarifying. Autoscaling deserves its own reckoning, because the failure mode isn't as simple as "it scales too much." Horizontal Pod Autoscaler configurations are typically written by engineers whose primary experience with the service was getting it to scale up fast enough during an incident. The scaling-out parameters get tuned aggressively; the scaling-in parameters stay at default or get made more conservative, because prematurely scaling in caused latency problems once and nobody wants that phone call again. The result is a ratchet: the cluster grows to accommodate load peaks and then stays grown, because the hysteresis is asymmetric. At a per-node cost of, say, $0.20/hour for a reasonable compute instance, running thirty nodes when twelve would suffice represents nearly $25,000 in annual waste — for a single workload. Multiply across services in a mid-sized platform organization and you're looking at numbers that fund engineering headcount. The reactive versus predictive scaling trade-off in the original article's table is real, but it understates the implementation cost of predictive scaling. Getting good predictions requires either historical data with stable periodicity (traffic patterns that repeat weekly, that kind of thing) or ML-based forecasting infrastructure with its own operational overhead. Most teams don't have clean enough signals to train reliable forecasting models, especially if their traffic has high variance or is driven by irregular external events — marketing campaigns, news cycles, competitor outages generating unexpected traffic. Scheduled scaling is more tractable: if you know from two years of logs that traffic increases 40% on weekday mornings at 9 AM in your primary timezone, you can pre-scale before that ramp rather than chasing it reactively. This doesn't require ML. It requires looking at your traffic patterns, which is a thing engineers often don't do because it doesn't feel like engineering. The honest trade-off isn't between reactive and predictive scaling as equivalent strategies with symmetric costs and benefits. It's between the certainty of reactive (it's always correct, just sometimes late) and the efficiency of predictive (it's cheaper when right, occasionally wrong, and wrong in ways that are visible and embarrassing). Most platform teams are better served by reactive scaling with more aggressive scale-in parameters and shorter cooldown windows than they currently run, plus scheduled pre-scaling for known patterns, than by investing in a forecasting infrastructure they won't maintain properly. What does a careful builder actually do on Monday morning? Probably not a complete FinOps transformation. Those take quarters, involve organizational dynamics that engineering can't resolve unilaterally, and have a tendency to generate dashboards that everyone nods at in the monthly review and nobody uses between reviews. Start with the instrumentation gap. Identify the three services that collectively drive the most spend — AWS Cost Explorer will tell you this, broken down by service type, in about five minutes. For each of those three services, answer the question: do you know what a normal cost-per-request looks like? If the answer is no, you don't have a cost monitoring problem, you have a cost observability problem, and that's the thing to fix first. Add the metrics, derive the baseline, set an alert threshold at 2x baseline as a starting point. You'll tune it. But you won't tune something you haven't measured. Then: run the idle resource report. Not for the purposes of immediately cleaning things up — though do that too — but to understand what the organizational failure mode is that produced those resources. Someone created them. Someone forgot them. Was there no offboarding process for decommissioned projects? Was there no budget owner for that cost center? Was the tagging policy unenforced? The idle resources are symptoms. The absence of process is the condition. And then — this is the uncomfortable one — have the conversation with whoever owns budget decisions about treating a cost anomaly the same way you'd treat a reliability incident: with a postmortem, with a timeline, with root cause analysis, with action items. Not blame. Not forensic punishment. The same blameless retrospective process you'd apply to a production outage, applied to a billing spike. Because a billing spike is a production incident. It just bills you for it differently. The systems are already distributed. The costs are already real. The instrumentation is the part you chose not to build yet. Build it.

In the cutthroat world of technology, ageism has long cast a shadow over seasoned professionals. Layoffs targeting workers over 50 — epitomized by recent waves at Meta, Google, and Amazon — reveal a bias favoring youthful energy over accumulated wisdom. Yet, as AI tools explode in capability, a paradigm shift emerges: artificial intelligence isn't just automating jobs; it's supercharging the efficiency of older workers, blending their decades of insight with machine precision. This fusion could herald the death of ageism, positioning "long-living" professionals as indispensable assets for innovative companies. The Ageism Crisis in Tech: A Stark Reality Tech's youth obsession is no secret. A 2023 AARP report found that 1 in 5 workers over 50 face age discrimination, with tech hit hardest — median employee age at major firms hovers around 30-32, per Levels.fyi data. High-profile cases abound: Intel's 2024 layoffs disproportionately axed veterans, while startups shun "overqualified" applicants fearing cultural misfits. The rationale? Assumptions that older workers lag in adapting to rapid tech shifts, from cloud-native architectures to GenAI workflows. But this overlooks a goldmine: experience. Older professionals bring battle-tested judgment — spotting ethical pitfalls in AI deployments, architecting scalable systems from the mainframe era, or navigating stakeholder politics that sink 70% of digital transformations (per Gartner). The challenge has been proving their velocity matches the 20-somethings grinding 80-hour weeks. Enter AI. The Great Equalizer for Efficiency and Insight Generative AI democratizes productivity, erasing speed gaps that fuel age bias. Tools like GitHub Copilot, Claude, and Cursor now handle 40-55% of coding tasks, per GitHub's 2025 State of the Octoverse report — freeing humans for high-value work. For older developers, this means recapturing peak efficiency without the burnout of constant upskilling. Consider prompt engineering, AI's secret sauce. Seasoned pros excel here, leveraging contextual wisdom to craft precise instructions. A 2024 McKinsey study showed prompt-savvy users boost AI output quality by 30-50%; veterans' edge shines in nuanced scenarios, like generating secure microservices code or debugging legacy integrations. Example: A 58-year-old architect at a Fortune 500 firm used GPT-4o to prototype a Kubernetes-orchestrated app in hours, drawing on 30 years of deployment failures to refine prompts iteratively — output rivaled a junior team's weeks-long sprint. Beyond code, AI amplifies broader strengths: Knowledge Synthesis: Tools like Perplexity or Gemini summarize vast docs instantly, letting experts apply domain intuition without rote recall.Lifelong Learning Acceleration: Adaptive platforms (e.g., Duolingo for code via Replit AI) tailor training to experience levels, compressing years of ramp-up.Collaboration Boost: AI notetakers (Otter.ai, Fathom) and real-time copilots bridge generational gaps, turning mentorship into scalable superpowers. Real-world proof? IBM's 2025 pilot paired 50+ engineers with Watsonx; productivity surged 35%, with error rates dropping due to "insight-infused" prompts. Startups like Replicate report hiring 40+ talent post-AI, citing 2x faster innovation cycles. Why Companies Should Prioritize Older Pros: The Business Case Hiring gray hair isn't charity — it's strategy. Deloitte's 2025 Human Capital Trends flags "experience dividends" as key to AI-era resilience: older workers reduce project risks by 25% via foresight, per Harvard Business Review analysis. They mentor juniors effectively, curbing 40% turnover in Gen Z-heavy teams (Gallup data). Quantifiable wins include: AdvantageYounger WorkersOlder + AI WorkersBusiness ImpactProductivityHigh raw speedAI-amplified consistency20-40% faster delivery (McKinsey)InnovationBold ideasRefined, feasible execution30% higher success rates (Gartner)Risk MitigationTrial-and-error learningPreemptive issue spotting50% fewer production bugsRetentionHigh churn (25% annual)Loyalty (10-15% churn)$50K+ savings per roleDiversity ROIHomogeneous viewsCross-era perspectives19% higher revenue (BCG) Forward-thinking firms agree. Salesforce's 2026 hiring push targets 45+, armed with Einstein AI for seamless onboarding. "Experience compounds with AI," says CEO Marc Benioff. Governments echo this: EU's Digital Decade mandates age-diverse tech pipelines, backed by AI subsidies. Critics warn of resistance — older workers must embrace tools. Yet adoption rates rival youth: Stack Overflow's 2025 survey shows 62% of 50+ devs using AI daily, up from 12% in 2023. Embracing Meritocracy: Fair Chances for All Ages This vision is no zero-sum race pitting young against old. AI fosters true meritocracy, where talent triumphs regardless of age — evaluating contributions on impact, not calendars. Workplaces can and should host larger youth contingents for fresh dynamism, balanced by veterans' stabilizing force, creating multigenerational teams that outperform homogeneous ones by 20% in creativity (McKinsey). The goal: equitable opportunity, upskilling programs for all, and hiring that rewards proven value, ensuring tech's talent pool expands sustainably. A Reinvented Future: Long Live the Long-Living! AI doesn't replace wisdom; it resurrects it. By turbocharging efficiency and channeling time-won insights into prompts and strategy, it dismantles ageism's core myth: that tech demands perpetual youth. Companies ignoring this risk talent droughts amid 85 million AI-displaced jobs by 2030 (World Economic Forum). The call is clear: Tout older professionals as premium hires. Build AI-native roles celebrating their edge — Senior Prompt Architects, Insight Orchestrators. Tech's future belongs to the ageless: those who pair machine horsepower with human depth. As one 62-year-old CTO shared post-layoff reinstatement, "AI gave me my 30s back — and then some." Long live the long-living.

Reading is essential for everyone, and especially for software engineers. Our field centers on managing and advancing knowledge. As technologies and architectural paradigms evolve and challenges grow more complex, continuous learning becomes fundamental. In 2025, I read 34 books spanning philosophy, history, economics, and software engineering. While these subjects may seem unrelated to coding, they all aim to deepen our understanding of systems, whether in societies, economies, or software architectures. This article highlights six books that stood out for software engineers. Each offers lessons beyond technical implementation, covering strategy, leadership, learning, and design — skills that grow in importance as engineers progress in their careers. Some of these books are rereads. Revisiting valuable books often reveals new insights as our perspectives evolve. What once seemed theoretical may become highly practical when we encounter similar situations in real projects. Let’s start with a book that addresses one of the most misunderstood topics in engineering organizations: strategy. Crafting Engineering Strategy One of the most impactful books I read in 2025 was Crafting Engineering Strategy: How Thoughtful Decisions Solve Complex Problems by Will Larson. Many engineers assume their organization lacks an engineering strategy. In reality, most organizations already have one — it just might not be effective, explicit, or aligned with the company’s goals. Will Larson, also known for An Elegant Puzzle and as a staff engineer, provides a practical guide to navigating technical and organizational complexity through structured strategy. The book is especially valuable for senior engineers, architects, and engineering leaders who influence decisions beyond code. The author presents a repeatable process for building actionable engineering strategies, from diagnosing problems to communicating and implementing initiatives. Real-world examples from companies like Stripe, Uber, and Calm show how strategy shapes decisions on platform migrations, API deprecations, and infrastructure investments. Some of the most valuable lessons include: Building durable engineering strategies from first principlesApplying techniques such as Wardley Mapping and systems modelingLeading strategic initiatives as a staff+ engineer or engineering executiveLearning from real case studies across different industriesImproving long-term influence through structured thinking Engineering strategy is often seen as abstract or reserved for executives. This book clarifies that strategy is the structured alignment of technical decisions with long-term goals. While strategy and technical insight are essential, they are not the only factors in a successful engineering career. Often, the real differentiator is less technical. Emotional Intelligence Emotional Intelligence by Daniel Goleman offers an important perspective for software engineers: technical skills alone are not enough. In many organizations, engineers with strong technical capabilities are surprised when others — sometimes with less technical expertise — reach leadership positions faster. It is tempting to assume that the system is unfair. In reality, another factor is often at play: emotional intelligence. Daniel Goleman’s groundbreaking work explores how human behavior is shaped by two complementary systems: the rational mind and the emotional mind. While traditional intelligence (IQ) measures analytical ability, emotional intelligence (EI) includes qualities such as: Self-awarenessSelf-regulationEmpathySocial skillsMotivation These capabilities strongly influence collaboration, conflict resolution, communication, and leadership. Drawing on psychological and neurological research, Goleman explains why some with high IQs struggle professionally while others with moderate IQs succeed. Emotional intelligence shapes our ability to build trust, influence others, and navigate complex social environments — skills that grow in importance as engineers move into architectural or leadership roles. Another powerful insight from the book is that emotional intelligence is not fixed at birth. While childhood experiences shape it, EI can be developed throughout adulthood through reflection, feedback, and intentional practice. Recognizing this aspect of growth changes how we view engineering careers. The most successful engineers are not only technically strong but also understand people, teams, and organizational dynamics. This naturally brings us to the next topic: how engineering teams actually function and succeed in practice. Leading Effective Engineering Teams Leading Effective Engineering Teams by Addy Osmani is another standout book from my 2025 reading list. Drawing on over a decade with the Chrome team at Google, Osmani examines what makes engineering teams effective. The book addresses both individual contributors and engineering managers. One of the key themes of the book is the distinction between efficiency, effectiveness, and productivity — three concepts that are often used interchangeably but actually represent very different things. Efficiency focuses on doing tasks quickly.Productivity measures output.Effectiveness measures whether the work actually delivers meaningful impact. In engineering teams, optimizing the wrong metric can cause problems. Teams focused solely on productivity may generate large volumes of code without delivering real value. Osmani emphasizes that effective teams are built on trust, accountability, and clear communication. The book offers practical guidance on topics such as hiring, mentoring, career growth, and building sustainable engineering culture. Some highlights include: Traits of highly effective engineers and teamsTechniques for fostering trust and accountabilityStrategies to minimize friction in collaborationSystems thinking approaches for daily engineering decisions.Methods for improving visibility and recognition within organizations The most valuable lesson is that engineering excellence is rarely achieved alone. It almost always results from a healthy team culture. Once we understand how teams function, the next natural question becomes: how should we design the systems those teams build? This leads us to a topic that is often misunderstood in software architecture. Balancing Coupling in Software Design When software engineers first study architecture, one concept appears repeatedly: coupling. The message is almost always the same: coupling is bad. However, Balancing Coupling in Software Design by Vlad Khononov challenges this simplistic perspective. Coupling is not inherently bad. In fact, it is unavoidable. Every design decision we make introduces some form of coupling. The real challenge is understanding and controlling it. Khononov explores how coupling affects modularity, system evolution, and long-term maintainability. The book builds upon decades of research in software engineering while adapting those concepts to modern architectural practices such as microservices, domain-driven design, and distributed systems. Rather than treating coupling as something to eliminate, the book presents it as a design dimension that must be balanced. Some key insights include: Understanding different types of coupling in software systemsUsing coupling intentionally to manage complexityRecognizing trade-offs between modularity and system cohesionApplying design principles that support long-term evolution This perspective is especially valuable for architects and senior engineers who must balance flexibility, performance, and maintainability. Even the best design principles are ineffective if engineers cannot continuously learn and adapt. Given the rapid pace of change in our industry, learning is a core engineering skill. Ultralearning Ultralearning: The Essential Guide to Mastering Hard Skills and Future-Proofing Your Career by Scott H. Young focuses on one of the most critical abilities for modern professionals: learning efficiently. Software engineers constantly encounter new frameworks, languages, architectures, and methodologies. The challenge is not only learning new technologies but also deciding what is worth learning. Young introduces the concept of ultralearning, an intense and structured approach to mastering complex skills quickly. The book presents nine principles that help individuals learn deeply and effectively through self-directed education. Some of the ideas explored include: Direct learning through real projectsStrategic practice and feedback loopsRetrieval-based learning instead of passive readingExperimentation and adaptation of learning strategies The book highlights historical and modern ultralearners, such as Benjamin Franklin, Richard Feynman, and Judit Polgár, showing that structured self-learning has long driven mastery. For software engineers, this mindset is particularly valuable. The industry evolves rapidly, and those who learn efficiently gain a significant advantage over time. However, learning and design are only part of the equation. Without effective knowledge sharing, teams and organizations struggle to stay aligned. Docs Like Code Documentation remains one of the most underestimated aspects of software engineering. In many organizations, teams fall into one of two extremes. Either documentation is almost nonexistent, forcing engineers to rely on meetings and tribal knowledge, or there is an overwhelming amount of documentation that becomes outdated and ignored. Docs Like Code: Collaborate and Automate to Improve Technical Documentation introduces a more balanced approach. The core idea is simple: Treat documentation the same way we treat code. This means applying practices such as: Version controlCode reviewsContinuous integrationAutomated validationCollaborative workflows By integrating documentation into the development lifecycle, teams can ensure that knowledge evolves alongside the codebase. The result is documentation that remains relevant, maintainable, and useful, rather than becoming an abandoned artifact. For engineers focused on system design and long-term maintainability, this approach transforms documentation from a bureaucratic task into an essential engineering practice. Final Thoughts Reading remains one of the most powerful habits a software engineer can develop. The books highlighted here address various aspects of engineering growth: strategy, emotional intelligence, team dynamics, architectural design, learning, and documentation. Together, they offer a broader perspective on growing beyond coding to become a more complete engineer. Software engineering is not only about building systems. It also involves understanding complex environments, collaborating with others, making strategic decisions, and continuously learning. Sometimes, the best way to improve as an engineer is simply to start with a good book.

By Otavio Santana CORE

There's a particular kind of learned helplessness that settles into engineering organizations after a few years of rapid cloud growth. You ship a feature. The feature works. Latency looks fine, error rates stay quiet, on-call doesn't page. Then three weeks later someone from finance drops a Slack message — a screenshot of the AWS Cost Explorer with a jagged upward spike, annotated with a red arrow and a question mark. By then, the deployment that caused it has been buried under six more deploys. The engineer who wrote the change is mentally two features ahead. Nobody remembers. You run a postmortem on nothing. This is the default state for most shops. Not negligence, exactly. More like a structural information deficit: the feedback loop between code change and cost impact is measured in billing cycles, not seconds. Runtime FinOps is the attempt to collapse that latency. The core mechanical insight is embarrassingly simple once you see it. Cloud spend is ultimately a function of resource consumption, which is itself a function of workload behavior, which is directly caused by deployed code. The causal chain is unbroken. What's broken is the observability of that chain — the instrumentation stops at runtime metrics and never continues downstream into the dollar layer. Prometheus scrapes CPU and memory. Datadog tracks p99 latency. Nobody is emitting cost_per_request_dollars into the same time-series store. That gap isn't accidental. It reflects organizational archaeology — engineering tools were built by engineers who didn't own the bill, and finance tools were built by accountants who didn't understand deployment pipelines. The FinOps movement as a discipline has largely tried to paper over this by creating shared dashboards and monthly reviews. That's better than nothing. It is not remotely sufficient. What sufficient looks like: a Grafana panel, sitting next to your latency and throughput charts, showing dollars-per-minute in something close to real time. Not aggregated monthly, not delayed by the 24-to-48-hour lag that AWS billing data typically carries, but live. Or close to live. And critically, annotated — vertical lines at every deploy, tagged by Git SHA, so when the cost curve flexes upward you can see which change correlated with when. Tools like Kubecost and CloudZero attempt this for containerized workloads, mapping cluster resource consumption to workloads and namespaces with reasonable accuracy. The attribution model involves some approximation — particularly around shared infrastructure, node-level overhead, and storage that doesn't decompose cleanly to individual pods — and practitioners would be dishonest if they called it precise. It's directionally accurate. In FinOps, directionally accurate and fast beats precisely accurate and three weeks late every single time. The tagging problem deserves its own meditation, because this is where ambition usually fractures against operational reality. The idea is clean: every cloud resource carries tags — service, team, environment, git-sha, pr-number — and those tags flow through billing, letting you attribute cost to the unit of work that caused it. In theory, you can then answer "what did this pull request cost us in production over its first 72 hours of traffic?" In practice, tagging compliance in most organizations sits somewhere between 40% and 70% on a good day, because tags are set at resource creation and then drift, or get set inconsistently across Terraform modules, or simply aren't applied to resources provisioned through the console in a hurry. Data transfer costs — often a substantial portion of a distributed system's bill — aren't taggable in any meaningful way. RDS instance costs don't decompose to the query or calling service. The tag taxonomy you design in January will be partially obsolete by June when someone creates a new microservice and doesn't know the convention. None of this means tagging is futile. It means the feedback loop you build on top of tags is only as trustworthy as your tagging governance, and tagging governance requires someone to actually own it, which requires organizational will that frequently isn't there. The more robust pattern I've seen in practice: tag at the workload level (not the resource level), enforce it via CI/CD gate rather than relying on humans to remember, and accept that you'll have a residual "unattributed" bucket that you manage down over time rather than eliminating entirely. Tools like AWS Tag Editor and custom OPA policies for Terraform can close the loop on net-new resources. The legacy tail requires a different, less glamorous approach: manually audit, assign, iterate. The CI/CD integration story is where things get genuinely exciting, and also where practitioners should calibrate their expectations carefully. Infracost is the canonical example: it parses Terraform plan output, estimates the monthly cost delta of the proposed infrastructure change, and posts that estimate as a comment on the pull request. This is legitimately useful. A PR that adds three RDS read replicas and a NAT gateway should trigger a cost conversation before it merges, not after the bill lands. Engineers who see "this change will add ~$340/month" in their PR review interface learn, over time, a working intuition about what infrastructure costs. That intuition is rarer than it should be. The limitation is that Infracost and its peers estimate infrastructure cost — the static resource footprint — rather than operational cost, which includes data transfer, API calls, Lambda invocations, storage I/O, and everything else that scales with traffic and behavior rather than existence. A change that looks cost-neutral at the infrastructure level might double your CloudFront egress if it changes response payload sizes. It might triple your DynamoDB read units if it introduces a hot key. The tools don't know this. They can't, without runtime data. The more sophisticated version of this loop, which fewer teams have built, uses predictive cost modeling against actual traffic. You have a deployment. You have the last N days of traffic patterns. You can project forward: "given current traffic, this new resource configuration will consume approximately $X over the next 30 days." AWS Cost Explorer has a forecast API. Combining it with deployment annotation is not a huge engineering lift, but it requires someone to actually build and maintain the plumbing. Most teams haven't made that investment. Consider what an SRE-inflected cost culture actually demands. SRE borrow two concepts that apply almost without modification: error budgets and anomaly alerting. An error budget for cost would look like this: the service owns a monthly cost envelope, approved and visible, and the team tracks burn rate against it the way they track error budget burn against their SLO. When burn rate exceeds a threshold — say, the monthly budget will be exhausted in 20 days at current trajectory — that's an alert, the same severity as a latency SLO violation. Not a finance report. A PagerDuty ticket if you want to be maximalist about it, or at minimum a Slack alert that reaches the on-call engineer, not the VP of Engineering. AWS Cost Anomaly Detection does a serviceable version of this out of the box, using ML to detect spend patterns that deviate from the expected baseline and sending SNS notifications. It's underused. I suspect this is partly because the notification goes to whoever set up the billing alert (often a platform team, sometimes a finance person) rather than to the team that owns the service. The alert finds the wrong inbox and dies there. The organizational fix is unglamorous: route cost anomaly notifications to the same escalation paths as operational incidents. The same service catalog that maps an alert to an on-call rotation should map a cost anomaly to the team that owns the relevant tagged resource. This requires the tagging to work. Everything requires the tagging to work. There's an architectural pattern worth naming explicitly: cost as a flow control signal. In a well-instrumented system, you might have a service that responds to demand by scaling out — adding pods, provisioning more compute, whatever the autoscaling policy dictates. This is good. Autoscaling is good. But autoscaling policies are typically expressed in terms of CPU utilization or queue depth or request rate, never in terms of "we have now spent $X in the last hour and this is abnormal." A traffic spike from a misbehaving client, a scraper, an accidental infinite loop in a partner's integration — these can drive spend through the ceiling before any CPU-based autoscaler would even notice a problem. Dollar-rate alerting fills a different detection envelope than performance alerting. A pathological client that sends low-volume but expensive requests — each one triggering a chain of downstream API calls, S3 reads, expensive ML inference — might not move your CPU metrics at all. It will move your bill. If you're watching dollars-per-minute in Prometheus and the rate doubles, that signal is available to you immediately. Whether you act on it programmatically (rate limiting, circuit breaking, graceful degradation) or operationally (alert, investigate, remediate) is a choice, but you can't make it if you can't see it. The blameless postmortem for cost incidents is a concept that sounds slightly ridiculous the first time you hear it and becomes obviously correct about sixty seconds later. When a cost spike happens, the natural instinct in most organizations is either to ignore it (it's just money, nobody died) or to hunt for the responsible party and make an example of them. Both responses are bad. Ignoring it means the behavior repeats. Making an example of someone means engineers become risk-averse about infrastructure changes in ways that slow down the whole organization. The SRE approach to operational incidents — reconstruct the timeline, identify contributing factors, generate mitigations, share the learning broadly — transfers completely. What was the change that caused the spike? Was it a code change, a configuration change, an unexpected shift in traffic? Was it even caused by a change, or is it an emergent behavior of a system that was always going to fail this way under sufficient load? What could have caught it earlier? What will catch it next time? The output of that process is institutional knowledge and, eventually, changed defaults. The team that burns their cost budget on an accidentally O(n²) database query and runs a postmortem on it will write better queries afterward, not out of fear but because they now have a concrete understanding of what "better" means in dollar terms. Honestly, the biggest obstacle isn't technical. The tools exist. Kubecost, CloudZero, Infracost, CloudHealth, AWS-native cost tooling — the ecosystem is mature enough that you can build a meaningful runtime FinOps practice without writing much novel infrastructure. The pipeline from resource consumption to tagged cost attribution to developer-facing dashboard is navigable. What isn't navigable without organizational agreement is the question of who owns this. Finance owns the bill but not the code. Engineering owns the code but not the budget. Platform teams own the tooling but not the individual services. FinOps functions, where they exist, often sit in a liminal space that has advisory authority but not operational authority. None of these entities, alone, can close the feedback loop. The teams that actually do this well tend to have one thing in common: a clear owner at the service level. Not "the platform team will build cost dashboards for everyone" but "this service team owns a cost SLO, reviews it in their weekly ops meeting, and is the first call when a cost anomaly fires." That's a cultural stance, not a technical one. If you wanted to change something by Monday morning, the smallest high-signal move is this: find your last three significant cost spikes, look at the deployment timeline, and see whether you can identify the correlating change. Do this manually, in AWS Cost Explorer, cross-referenced against your deployment log. If you can correlate them — if the mechanism is visible in retrospect — you now have a concrete example to show your team of what a runtime cost signal would have caught in real time. That example is worth more than any amount of abstract advocacy for FinOps practices. Then ask yourself: what's the minimum instrumentation that would have surfaced this signal at deploy time? Maybe it's a CloudWatch alarm on spend rate. Maybe it's a Kubecost dashboard with a deployment annotation. Maybe it's just a Slack alert from Cost Anomaly Detection routed to the right channel. Start there. The elaborate CI/CD cost gates and per-Git-SHA bill-of-materials and predictive spend forecasting are all real and all worthwhile, but they're downstream of a simpler belief: that cloud spend is a system metric, not a finance report, and your observability stack should treat it that way. The rest follows.

For decades, software engineering followed a relatively predictable path: learn the language, master the tools, deliver results, and progress. That model is quietly breaking. Today, engineers are expected to do more than build systems — they are expected to influence decisions, communicate across teams, and demonstrate impact beyond their immediate environment. Yet most career advice still focuses solely on improving technical skills. This creates a gap. In this article, we explore how open source — especially through Jakarta EE — fills that gap, turning everyday engineering work into something visible, scalable, and career-defining. The Challenge of Modern Software Careers Once we accept that technical excellence alone is no longer enough, the next question becomes unavoidable: What actually sustains a software engineering career today? The industry has changed in subtle but significant ways. Stability has decreased, expectations have expanded, and the definition of value has shifted. Engineers are no longer evaluated only by their ability to deliver features, but by their capacity to influence decisions, communicate ideas, and operate beyond the boundaries of their immediate team. This creates a tension. Many engineers continue to invest heavily in technical preparation — learning frameworks, improving coding practices, studying architecture — yet still feel stuck. The issue is not always a lack of effort, but often a mismatch between effort and opportunity. Preparation, in isolation, does not scale if it remains invisible. Historically, engineering was never just about tools. The term itself comes from ingenium, referring to ingenuity, creative problem-solving, and the capacity to devise solutions under constraint. That older meaning matters because it reminds us that engineering is not simply technical execution; it is the disciplined application of judgment. But judgment alone does not guarantee opportunity. This is where Seneca becomes surprisingly modern. He is often paraphrased as saying that luck is what happens when preparation meets opportunity. Whether we call it luck, chance, or timing, the principle is the same: opportunity favors those who are already in motion. In the context of a software career, this means waiting to become visible only when the perfect opportunity arises is already too late. We need preparation, certainly, but also visibility and adaptability, because in practice these are what allow preparation to encounter opportunity at all. That is why the real challenge of the modern career is not only becoming good, but becoming discoverable, credible, and ready. And this is exactly where open source and open standards begin to matter. Open Source and Open Standards as Career Leverage Open source is often misunderstood. It is frequently treated as a side activity, something optional or even altruistic. But if we examine it more carefully, open source functions as a mechanism for making work visible at scale. It transforms private effort into public evidence. Instead of describing your experience, you expose it. Instead of claiming expertise, you demonstrate it. This distinction matters because traditional career signals — résumés, certifications, interviews — attempt to infer capability. Open source reduces that distance. It allows others to see how you think, how you collaborate, how you respond to criticism, and how you improve an idea over time. In that sense, open source becomes more than a technical activity. It becomes a form of preparation made visible. And that returns us to Seneca’s insight: Preparation without contact with the world remains incomplete. It is only when knowledge becomes visible and testable in public that it can truly meet opportunity. But open source alone is only part of the picture. To understand why it can have such a strong effect on a career, we need to add another concept: open standards. Historically, standards have been among the great enablers of civilization. Shared language allowed cooperation beyond small groups. Writing preserved thought across generations. Standard units of measure made trade, engineering, and science reliable. Human progress did not scale merely because people were talented; it scaled because meaning became shareable. Software is no exception. As systems become larger and more interconnected, a lack of standards leads to fragmentation, lock-in, and unnecessary complexity. Open standards address this by defining shared expectations independently of a single implementation. They create stability without demanding uniformity from vendors. When open source and open standards work together, something unusual happens. Open source creates transparency, collaboration, and visibility. Open standards create consistency, interoperability, and durability. One opens the door to participation; the other ensures that what is built can endure beyond a single company or framework. For software engineers, this combination is particularly powerful. It means that contributing is not only about fixing code or adding features. It is also about entering into a wider conversation about how systems should be designed, how technologies should evolve, and how collaboration can scale across organizations. This is why open ecosystems are so valuable for a career. They do not merely improve technical skill; they train judgment, communication, and long-term thinking. And few examples in enterprise Java illustrate this intersection as clearly as Jakarta EE. Jakarta EE: Where Open Source Meets Enterprise Reality Jakarta EE represents a convergence of these ideas in the Java ecosystem. At its core, it provides vendor-neutral APIs intended for long-lived enterprise applications. On the surface, that may sound like a technical description. In reality, it reflects a broader philosophy: software should evolve without forcing organizations into permanent dependency on a single implementation. This matters because enterprise systems are rarely short-lived. They are designed to survive years of evolving requirements, teams, and infrastructure. Without standards, this continuity becomes fragile. With them, systems gain a degree of resilience and predictability. That is why Jakarta EE is more than a framework discussion. It is an example of how open standards and open source can coexist to serve real business needs. It provides a shared foundation while still allowing multiple implementations, vendors, and runtimes to participate. This reduces fragmentation and makes enterprise Java more coherent over time. For engineers, engaging with Jakarta EE introduces a deeper layer of professional growth. The questions shift from local implementation details to broader design concerns. How should an API behave across environments? How do we preserve compatibility while allowing evolution? How do we create something that remains useful beyond the immediate preferences of one team or one company? These are not only coding questions. They are architectural and even philosophical questions, because they concern continuity, cooperation, and trade-offs over time. And that brings us back, quietly, to the same principle. If modern careers require preparation, visibility, and adaptability, then Jakarta EE offers a space where all three can be exercised together. It is certainly technical work, but it is also public, collaborative, and durable work. In other words, it is preparation in a form that has a real chance of meeting an opportunity. Still, understanding the value of such an ecosystem is one thing. Applying it to daily life is another. Applying This in Practice: From Knowledge to Career Movement Knowing that open source and standards can accelerate a career does not, in itself, change anything. The practical question is how to make this part of one’s professional life without turning it into burnout or abstraction. The first answer is consistency. Many engineers approach open source in bursts of enthusiasm, contributing intensely for a few days or weekends and then disappearing. But careers, much like reputations, are built less by intensity than by continuity. Seneca, in his Stoic way, repeatedly emphasized discipline over impulse. That applies here as well. A small, consistent contribution is often more transformative than an occasional heroic effort. The second answer is to treat open source as training, not as performance. At the beginning, the work may feel invisible or unpaid, and that can be discouraging. But this is precisely where long-term thinking matters. You are not only contributing code; you are learning to write clearly, to discuss ideas, to review systems critically, and to operate in public. These are career assets that compound. The third answer is communication. No meaningful open ecosystem works without it. Engineers must learn to explain decisions, respond respectfully, document clearly, and engage across cultures. This is one reason English becomes so important in practice. In software, English functions almost like musical notation in music: it is the medium through which participation becomes possible at scale. Learning it early is not merely a linguistic advantage; it is access to the broader conversation. The fourth answer is balance. A career is not strengthened by sacrificing everything to it. One of the oldest philosophical lessons, not only in Stoicism but in ethics more broadly, is that discipline without measure becomes self-destruction. Open source should expand your life, not consume it. Saying no, focusing on what matters, and accepting that no one can master the entirety of IT are signs of maturity, not weakness. And finally, there is the matter of visibility. Being skilled is essential, but it is not enough if your work never leaves the confines of the private sector. Visibility is not vanity. Properly understood, it is the process by which trust becomes possible. When people can see what you build, how you reason, and how you contribute, they have something concrete on which to base their confidence. Over time, this changes the nature of career opportunities. Instead of constantly needing to prove yourself from zero, your work begins to speak ahead of you. Conclusion: Preparation Meeting Opportunity If there is a single thread connecting all of this, it is the old Stoic insight we started with: Opportunity does not belong to those who merely hope for it, but to those who prepare in a way that allows chance to find them. That is why the modern software career cannot be reduced to technical competence alone. Preparation still matters, but it must now be visible and adaptable. Open source gives that preparation a public form. Open standards give it structure and durability. Jakarta EE shows how both can come together in a practical, long-lived, and globally relevant enterprise setting. The result is more than better code. It is credibility, trust, and a career foundation that extends beyond a single employer or moment in the market. In uncertain times, that may be the closest thing to stability we can build for ourselves. And perhaps Seneca would recognize the pattern immediately: we do not control when opportunity appears, but we can control whether we are ready when it does.

By Otavio Santana CORE

At moderate volumes, AWS Glue feels almost effortless. You increase workers. The job runs faster. You double the input size. Runtime roughly doubles. Everything behaves predictably. Then one day, it stops behaving that way. We had a job that ran in about 15 minutes. The dataset grew. Runtime climbed to 27. That made sense. We increased workers. It dropped to 22. We increased workers again. It dropped to 21. That was the moment it became clear we weren’t compute-bound anymore. What slowed the job down wasn’t CPU. It wasn’t memory. It wasn’t even S3 read time. It was shuffle. It was skew. And it was file behavior. This article walks through the experiments I ran to understand where Glue jobs really break at scale — and what actually fixes them. The Setup The pipeline is simple. Raw transaction-style data lands in S3. A Glue job transforms it, joins it to a small dimension table, aggregates it, and writes the result back to S3. Sometimes the output is plain Parquet. Sometimes it’s written to Iceberg. No streaming. No ML. No exotic orchestration. Just Spark running inside Glue. To keep this reproducible, I generated synthetic data inside Spark. Python from pyspark.sql import functions as F def generate_transactions(rows: int): return ( spark.range(rows) .withColumnRenamed("id", "txn_id_num") .withColumn("account_id", (F.col("txn_id_num") % 500000).cast("string") ) .withColumn("amount", (F.rand(42) * 500).cast("double")) .withColumn( "txn_ts", F.expr(""" timestampadd( MINUTE, cast(txn_id_num % 100000 as int), timestamp('2025-01-01 00:00:00') ) """) ) .withColumn("txn_date", F.to_date("txn_ts")) .withColumn("merchant_code", (F.col("txn_id_num") % 10000).cast("int") ) .drop("txn_id_num") ) This lets us scale from 5 million rows to hundreds of millions without introducing unknown variables. Where a Glue Job Can Actually Slow Down Before going deeper, it helps to anchor the discussion in how a Glue job actually executes. Most Glue job performance issues map cleanly to one of five phases: Read from S3TransformShuffleWriteCommit metadata (optional) When Scaling Still Works The first experiment was intentionally simple. Python df = generate_transactions(50_000_000) df_baseline = ( df.withColumn("amount_bucket", F.when(F.col("amount") < 50, "LOW") .when(F.col("amount") < 200, "MED") .otherwise("HIGH") ) .groupBy("txn_date") .agg(F.count("*").alias("txn_cnt")) ) At 5 million rows, the job was quick. At 50 million, runtime increased proportionally. At 200 million, it was slower but still predictable. This is what Spark does well: narrow transformations and simple aggregations scale cleanly. The problems start when the workload becomes wide. The Shuffle Shift Things changed as soon as I introduced a join and grouped by a higher-cardinality key. Python dim = ( spark.range(0, 10000) .withColumnRenamed("id", "merchant_code") .withColumn("merchant_category", F.concat(F.lit("cat_"), (F.col("merchant_code") % 50)) ) ) df_joined = df.join(dim, "merchant_code", "left") df_agg = ( df_joined.groupBy("txn_date", "merchant_category") .agg( F.count("*").alias("txn_cnt"), F.sum("amount").alias("total_amount") ) ) Runtime increased, but what mattered more was what the Spark UI showed. Most of the time was now spent inside shuffle stages. CPU wasn’t pegged. Executors weren’t maxed out. But the shuffle stage consumed the majority of runtime. That distinction matters. When a job is compute-bound, adding workers usually helps. When a job is shuffle-bound, the bottleneck shifts to data movement. Shuffle is not just another transformation. It is full data redistribution across the cluster. Rows are repartitioned by key, exchanged across executors, and often written to disk before being merged again. It is network-heavy. It is disk-heavy. And it is extremely sensitive to key distribution and imbalance. Once shuffle dominates runtime, adding workers produces diminishing returns. Why? Because you are no longer limited by raw compute. You are limited by how evenly data can be distributed across partitions. Skew: The Silent Runtime Killer To test skew, I ran: Python df.groupBy("account_id") \ .count() \ .orderBy(F.desc("count")) \ .show(10) A few keys had dramatically more rows than others. That explains long-running tasks at the tail of shuffle stages. Distributed systems are only as parallel as their most overloaded partition. One partition holding millions of rows can stall the entire stage. Salting as a Controlled Tradeoff One mitigation is salting: Python salted = df.withColumn("salt", (F.rand() * 10).cast("int")) salted = salted.withColumn( "account_salted", F.concat_ws("_", "account_id", "salt") ) This spreads large keys across partitions. It improves parallelism. It also increases shuffle complexity and requires careful downstream handling. Salting is a tradeoff, not a universal fix. The Partitioning Trap The most dramatic slowdown wasn’t from shuffle. It was from partitioning. Partitioning by txn_date behaved well. Partitioning by account_id looked logical. It wasn’t. Python df.write \ .mode("overwrite") \ .partitionBy("account_id") \ .parquet("s3://bucket/account_partition/") The result: File counts explodedWrite time increased significantlyAverage file size dropped sharply High-cardinality partitioning multiplies partitions and files. Each Spark task can write one file per partition. At scale, that becomes thousands of files. Small Files Are Not Harmless Small files affect: S3 object listingQuery planningMetadata operationsCompaction requirements The fix is not random repartitioning. It’s intentional shaping. Python df.repartition(200, "txn_date") \ .write.partitionBy("txn_date") \ .parquet("...") Aligning repartitioning with partition columns reduces file chaos. When Iceberg Enters the Picture Writing plain Parquet exposes file-level problems. Writing to Apache Iceberg adds a metadata layer. Each write creates: Data filesManifest entriesA snapshotA commit operation If file counts are high, commit time grows. If partitions are excessive, manifest lists expand. Creating the table is straightforward: SQL CREATE TABLE transactions_iceberg ( txn_id STRING, account_id STRING, txn_ts TIMESTAMP, amount DOUBLE ) USING iceberg PARTITIONED BY (days(txn_ts)); Writing is equally simple: Python df.repartition(200, "txn_date") \ .writeTo("catalog.db.transactions_iceberg") \ .append() The complexity shows up later: Slower planningGrowing snapshot historyMetadata overhead that scales with file count Iceberg doesn’t create performance issues. It amplifies poor file discipline. The Serverless Ceiling There is a point where: Shuffle dominates runtimeSkew stalls a subset of tasksFile creation dominates write timeCommit time becomes visibleIncreasing workers has minimal effect At that point, the scaling curve flattens. That’s the serverless ceiling. Adding more workers doesn’t help. Reshaping the workload does. Reducing shuffle width. Managing skew. Designing sane partition strategies. Controlling file size intentionally. Those changes moved runtime more than any worker increase did. Closing Thought Serverless removes cluster management. It does not remove distributed systems physics. Data movement still costs. Imbalance still hurts. Files still matter. Metadata still accumulates. Once you start thinking in terms of workload shape instead of raw compute, Glue scaling becomes much more predictable. And the next time a job jumps from 15 minutes to 40, you’ll know exactly where to look.