Career Development Resources

DZone's Featured Career Development Resources

Building a Zero-Cost Daily Job Alert Pipeline on GitHub Actions

By Mandar Chaudhari

I needed a job to run once a day, remember what it did yesterday, and cost nothing to operate. The obvious answer is a small VM with cron, or a Lambda plus DynamoDB. I did not want to pay for either, and I did not want a server to patch. So I pushed the whole thing onto GitHub Actions and used a JSON file committed back to the repo as the database. It has now run 139 times in production on the free tier, tracking just over 1,000 records, and the operating bill is still zero. Here is the part that took the most thought: keeping state across runs that are, by design, completely stateless. "The daily digest the pipeline sends, with new postings badged." The Constraint That Shapes Everything GitHub Actions gives you a cron trigger for free: Shell on: schedule: - cron: "0 16 * * *" # 09:00 EST daily workflow_dispatch: # manual button That solves scheduling. It does not solve memory. Every run starts on a fresh ubuntu-latest runner with a clean checkout. Anything you write to disk during the run is gone when the job ends. For my use case (a daily digest that must not re-send jobs it already sent), that is the entire problem. The script has to know what it saw yesterday. The standard fix is an external store. But for a workload that writes a few kilobytes once a day, standing up a database is more operational surface than the actual task. The repo is already there, the runner already has a checkout, and the workflow already has a token. So the store is the repo. Git as the Database The pattern is three lines at the end of the workflow: stage the state files, commit if they changed, push. Shell permissions: contents: write # the default token is read-only; you must opt in # ... run the script, which writes seen_links.json and job_history.json ... - name: Commit updated history files run: | git config user.name "GitHub Actions Bot" git config user.email "[email protected]" git add seen_links.json job_history.json 2>/dev/null || true git diff --staged --quiet || git commit -m "Update job history [skip ci]" git push One automated commit per day. The repo's own history is the database, and the audit log comes for free. Two details here are not optional, and I learned both the slow way. First, permissions: contents: write. The GITHUB_TOKEN handed to a workflow is read-only by default. Without this block, the git push fails with a 403, and the failure is at the very end of the run, after the real work succeeded, so it looks like everything worked until you check tomorrow and the state never persisted. Second, git diff --staged --quiet || git commit. This commits only when something actually changed. Committing an unchanged tree is an error, and a daily job that finds nothing new is a normal Tuesday. The || makes "nothing to commit" a no-op instead of a red X. The result is that the database lives in git history. Every state change is a commit. I can read yesterday's seen_links.json by checking out yesterday's commit. That is free audit logging I did not have to build. The Infinite-Loop Trap Here is the gotcha that will bite anyone who copies this pattern: a workflow that pushes a commit can trigger a workflow that runs on push, which pushes a commit, which triggers the workflow. The guard is the [skip ci] token in the commit message: git commit -m "Update job history [skip ci]" GitHub treats [skip ci] in a commit message as "do not start workflows for this commit." My scheduled workflow uses it. I also had a second, older workflow file in the repo whose commit message was a plain "Update seen links" with no skip token. Because that workflow only ran on schedule (not on push), it never actually looped, but it was one: push line away from a runaway. If your state-committing workflow has any push trigger, the skip token is the difference between a daily job and a billing incident. Put it in from the start. Decoupling "New" From "Still Worth Showing" The other decision I am glad I made early was separating two ideas that look like one: a record being new today, and a record being relevant today. A naive version sends only what is new since the last run. That breaks the moment a run finds nothing, or the moment the user skips a day. So state is two files with two jobs. seen_links.json is a flat set of every URL ever processed, used purely for deduplication. job_history.json is a rolling window: each entry carries a first_seen timestamp, and a record stays in the window for ten days regardless of how many runs happen in between. Shell def cleanup_old_jobs(history, max_days): today = datetime.now().date() cleaned = {} for category, jobs in history.items(): cleaned[category] = [] for job in jobs: first_seen = job.get("first_seen") seen_date = datetime.fromisoformat(first_seen).date() if (today - seen_date).days <= max_days: cleaned[category].append(job) return cleaned So "new" is computed per run (anything not in seen_links.json), and "relevant" is the trailing ten-day window. The daily output is never empty, nothing is ever sent twice, and a record ages out on a fixed schedule instead of vanishing the first quiet day. Two files, two responsibilities. Trying to make one structure do both is where this kind of project usually rots. The Dependency I Refused to Add The source data is two different table formats from upstream pages: one uses GitHub-flavored markdown tables, the other uses raw HTML tables inside the same document. The clean answer is a parsing library. I chose regex and the standard library instead, and I want to be honest about why and what it costs. The script tries markdown first, then falls back to HTML: Shell parsed_jobs = parse_markdown_table(text) if len(parsed_jobs) == 0: parsed_jobs = parse_html_table(text) # SimplifyJobs uses HTML The upside is a requirements.txt with exactly one line (requests), which means the install step on a cold runner is near-instant, and there is no transitive dependency that can break a 9 a.m. job. The downside is real, and I will not pretend otherwise: regex table parsing is brittle. When an upstream source changed its column layout, my parser silently returned zero rows for that source. It did not crash. It just quietly stopped finding jobs from one feed, which is the worst failure mode because nothing alerts you. For a personal tool with one user, that trade is fine: I notice within a day and patch a regex. For anything with real users, I would add a parser and, more importantly, a "parsed zero rows from a source that normally returns dozens" alarm. The lesson is not "regex bad." It is that a zero-result parse should be treated as a failure signal, not a valid empty result. Cheap Correctness Wins Two small filters do more work than their size suggests. Deduplication is a set membership check, which makes the whole pipeline idempotent. Running the workflow twice in one day produces the same output as running it once, because the second pass finds everything already in seen_links.json. For a cron job that you will inevitably trigger manually while debugging, idempotency is what lets you mash the button without consequences. Link quality is an allowlist of known applicant-tracking domains (Greenhouse, Lever, Workday, Ashby, and friends). Upstream rows mix real application links with company homepages and image badges. Filtering to known ATS hosts drops the noise without trying to validate every URL: Shell JOB_HOST_HINTS = ("greenhouse.io", "lever.co", "myworkdayjobs.com", "ashbyhq.com", "smartrecruiters.com", "icims.com", ...) def looks_like_job_link(url): return any(h in url.lower() for h in JOB_HOST_HINTS) An allowlist is the right default here because the failure mode is asymmetric. Letting through a dead homepage link wastes a click; an allowlist that occasionally drops a valid but unusual ATS is a one-line addition when I notice it. I would rather under-include than ship dead links. What it Actually Costs The numbers from production: 139 scheduled runs committed back to the repo, 1,062 unique links tracked in the dedupe set, three Python files, one runtime dependency, and one YAML workflow. Infrastructure cost is zero, because GitHub Actions' free tier covers a once-a-day job comfortably and Gmail's SMTP handles the delivery. There is no server, no database, no secret rotation beyond an app password, and nothing to wake up to at 3 a.m. When is This Pattern the Right Call? Reach for git-as-a-database when the write volume is low (you are committing on a human timescale, not a request timescale), the state is small and serializable, a single writer is doing the writing (the scheduled job), and you actively want the change history. A daily digest, a status snapshot, a slowly-changing config, a scoreboard: all good fits. Do not reach for it when you have concurrent writers (two runs racing to push will collide and one will fail the non-fast-forward push), when the state is large enough to bloat the repo, or when you need sub-minute reads or transactions. At that point you have outgrown the trick and a real datastore earns its keep. For everything in the first bucket, the calculus is hard to beat: the scheduler, the runtime, the storage, and the audit log are all things you already have for free. The only code you write is the part that does the work. More

Evolve or Automate: What It Actually Means to Be an AI-Native Data Engineer

By Janani Annur Thiruvengadam

CORE

The Moment It Gets Real At some point in the last year, every data engineer had the same experience. You opened a copilot tool, typed a rough description of what you needed, and watched it generate a working ETL pipeline in about thirty seconds. Not a skeleton. Not pseudocode. Actual, runnable PySpark with joins, transformations, and a DAG scaffold. And for a moment, the question that the industry had been treating as hypothetical became very concrete: if AI can do this, what exactly am I here for? That question deserves a serious answer — not the dismissive "AI is just a tool" reassurance, and not the catastrophist "engineers are obsolete" take. The honest answer is more nuanced, more interesting, and more actionable than either of those. What AI Can Actually Do Today Let's be precise about what has changed, because the hype runs in both directions. AI copilots in 2026 are genuinely impressive at a specific class of data engineering tasks. Give a well-prompted model a schema and a business requirement, and it will produce SQL that would have taken a competent engineer thirty minutes to write. Ask it to scaffold a dbt model with tests and documentation, and it delivers something you can actually work from. Point it at a slow query and ask for optimization suggestions, and it identifies the right indexes and join strategies most of the time. The work that once defined the day-to-day of data engineering — writing transformations, building pipeline boilerplate, generating unit tests, documenting schemas — is now legitimately acceleratable by an order of magnitude. That compression is real. A pipeline that took a week to build from scratch now takes a day. A day's worth of dbt model work now takes a morning. The cycle time has collapsed, and pretending otherwise is not a useful position. But Would You Actually Deploy It? Here is where the honest conversation has to happen. AI generates code that looks production-ready. It compiles. The DAG runs. The transformations return the right rows on the test dataset. And then you look closer. There are no retry semantics. There is no idempotency guarantee — run it twice, and you get duplicates. There are no data quality checks, no row count assertions, no schema drift detection. Observability is absent. The error handling catches exceptions and logs them to nowhere. Governance controls do not exist because the model has no idea what your data classification policies are. The code is impressively correct at the logic layer and completely unprepared for production reality. And that gap — between "AI generated it" and "it is actually deployable" — is not a small gap. It represents most of what makes data engineering genuinely hard. This is not a criticism of AI tooling. It is a precise description of where the boundary currently sits. And that boundary is exactly where the value of a skilled data engineer now concentrates. The Three-Bucket Reality Not all data engineering work is equally automatable, and the honest framework is to split it into three categories based on where AI sits today. What AI handles well. SQL and transformation generation, dbt model scaffolding, unit test generation, schema documentation, query explanation, code refactoring, and first-draft pipeline boilerplate. These tasks are high-volume, pattern-heavy, and well-represented in training data. AI performs them at a level that meets or exceeds what most engineers produce under time pressure. What AI assists but cannot own. Pipeline architecture decisions, root cause analysis on production failures, performance tuning for complex distributed jobs, and data modeling judgment for novel domains. AI is genuinely useful here as a thought partner and accelerant, but the decisions require context, business knowledge, and judgment that models do not reliably carry. What remains fundamentally human. Trade-off evaluation with real organizational constraints, governance and compliance decisions, architecture choices with long-term consequences, and anything requiring accountability. These require not just the right answer but the right answer for this company, this data, this regulatory environment, this team. That is irreducibly human work. The critical observation is that the boundary between these buckets is not static. Tasks that sat in the second bucket eighteen months ago have migrated into the first. The direction of travel is clear. Engineers who have concentrated their value entirely in automatable work are already exposed. Engineers who have built depth in judgment, architecture, and systems thinking are in an increasingly strong position. The Workflow Has Already Changed The before and after is not theoretical. It is visible in how high-performing data engineering teams actually operate today. The traditional workflow moved linearly through extraction, transformation, loading, and serving — each stage measured in hours to days, the full cycle measured in weeks. It was plagued by boilerplate, manual testing, documentation that was always out of date, and context-switching that fragmented deep work. The AI-enhanced workflow runs the same stages but with a fundamentally different time signature. StageTraditionalAI-EnhancedExtractHours — manual SQL, custom connectorsMinutes — AI-generated queries, auto connectorsTransformDays — dbt models, Spark jobsHours — AI-assisted modeling, auto schema detectionLoadHours — DAG authoring, schedulingMinutes — auto DAG generation, smart schedulingServeDays — dashboard building, documentationHours — auto documentation, natural language query The total cycle time compresses from weeks to days. That compression does not come from removing the engineer. It comes from removing the repetitive execution work so the engineer can focus on the decisions that actually require human judgment. What the Collaboration Actually Looks Like The AI-native data engineer workflow is not "prompt and deploy." It is a structured collaboration with a clear division of responsibility. AI accelerates the build. The engineer ensures it is correct, reliable, observable, and production-ready. The accountability for what ships belongs to the engineer, not the model. That accountability is not a burden — it is the source of professional value. The engineers who treat AI output as a draft to be critically evaluated and hardened will consistently outperform those who either ignore the tools entirely or treat generated code as finished work. Both of those failure modes are common. Neither is sustainable. The Skill Set Reorganizes, Not Disappears The skills required to be an excellent data engineer are shifting, but they are not evaporating. They are reorganizing around three pillars. Technical depth now centers on evaluating AI-generated code rather than writing all code from scratch. This requires strong fundamentals — you cannot spot the subtle join fanout in AI-generated SQL if you do not understand join semantics. It also means investing in observability, reliability engineering, and prompt crafting as first-class technical skills. A well-constructed prompt that produces deployable output in one iteration is genuinely more valuable than the ability to write the same code manually from scratch. Systems thinking becomes the primary differentiator. Architecture decisions, data modeling judgment, trade-off evaluation, and problem framing are tasks that compound in value as AI handles more execution work. The engineer who can look at a generated pipeline and immediately identify the three ways it will fail at scale is providing something no current model reliably provides. Engineering leadership expands to include guiding AI usage within a team, establishing review standards for AI-generated code, owning governance controls, and setting the quality bar that separates production-ready from impressive-looking. This is not a soft skill add-on — it is a core engineering responsibility in an environment where the output volume of any individual engineer has increased dramatically. The role is shifting from execution to judgment. That is an upgrade, not a downgrade, for engineers willing to make the transition deliberately. How to Actually Evolve The path forward is concrete, not abstract. Start by integrating AI into your daily work right now — not as an experiment but as a workflow change. Use it for SQL drafting, pipeline scaffolding, and test generation. Build the muscle of critically evaluating what it produces. Develop prompting habits that consistently get you to a usable first draft rather than something you have to rewrite from scratch. Level up by investing deliberately in the areas AI does not cover well. System design. Distributed systems fundamentals. Reliability and observability patterns. Data modeling for complex domains. These skills appreciate in value as AI handles more of the execution layer — the relative scarcity of strong systems thinkers increases as the supply of generated boilerplate becomes effectively infinite. Lead by taking ownership of AI quality standards on your team. Be the person who defines what "production-ready" means for AI-generated pipelines, who establishes review checklists, who sets governance guardrails. This is influence that compounds over time and is not replicable by a model. The Honest Bottom Line AI will not replace data engineers. But data engineers who treat their value as residing primarily in writing code — rather than in the judgment, architecture, and reliability thinking that makes code worth deploying — are taking a position that becomes harder to defend with each model release. The opportunity is real, and it is now. The engineers who learn to work with AI as a genuine collaborator, who develop the critical evaluation skills to close the gap between generated and production-ready, and who invest in the systems thinking that AI cannot replicate — those engineers are not threatened by this transition. They are the ones who define what data engineering looks like on the other side of it. Evolve deliberately. The alternative is not standing still — it is falling behind at an accelerating rate. More

Open Source as a Leadership Lab for Software Engineers

By Otavio Santana

CORE

The Real Skill Stack Behind Production-Ready AI Engineers

By Joshua Shelton

You Don’t Need To Be a Manager To Lead: Why Leadership Matters for Software Engineers

By Otavio Santana

CORE

Build Your Own Local AI QA Engineer With Docker, Ollama, LibreChat, and Playwright MCP

Artificial intelligence is rapidly transforming software testing by enabling QA engineers to generate test cases and test plans, automate browser interactions, analyze and debug failures, and execute complex testing workflows using simple natural-language prompts. While cloud-based AI assistants offer impressive capabilities, they often require subscriptions and sharing potentially sensitive application data with third-party services. Running an AI-powered testing assistant locally addresses these concerns by providing better privacy, lower operating costs, and complete control over the testing environment. In this tutorial, we’ll learn how to build our own local AI QA engineer using Docker, Ollama, Qwen3:8b, LibreChat, and Playwright MCP. It will allow us to perform browser automation and interact with web applications using natural language, all without relying on cloud-based AI services. Understanding the Architecture Every interaction begins with the user. For example, a user enters a prompt in LibreChat, such as “Open the Playwright website and click the ‘Get Started’ button.” LibreChat serves as the conversational interface through which users interact with the AI assistant. Rather than processing the request itself, it forwards the prompt to a locally hosted large language model, Qwen3:8b, running via Ollama. After receiving the prompt, Qwen3:8b interprets the user’s intent and generates a step-by-step execution plan. Instead of interacting with the browser directly, the model determines which tools are required and communicates those instructions using the Model Context Protocol (MCP). These MCP requests are handled by the Playwright MCP Server, which acts as the bridge between the language model and the browser. It translates the AI-generated instructions into executable Playwright commands. The Playwright MCP Server then launches a Chrome browser and performs the requested actions. Depending on the prompt, it can navigate to websites, click buttons, complete forms, extract text from web pages, capture screenshots, and execute a wide range of browser automation tasks. Once the browser completes the requested operations, the execution results are returned to Qwen3:8b. The language model analyzes the browser output and transforms the technical details into a clear, human-readable response. LibreChat then presents this response to the user. Instead of displaying raw Playwright logs, it provides a concise summary such as: “Navigation completed successfully. The Playwright website was opened, and the Get Started button was clicked successfully.” This architecture enables browser automation through natural language while ensuring that every component runs locally. As a result, we benefit from enhanced privacy, greater security, and complete control over the entire AI-powered automation workflow. Prerequisites Before getting started, ensure that the following software is installed on your machine: DockerNode.js 20 or higher versionGitOllama We’ll use Docker Desktop to run LibreChat, Node.js to install and run the Playwright MCP Server, Git to clone the required repositories, and Ollama to download and serve the local large language model. Having these tools installed beforehand will make the setup process smooth and straightforward. System Requirements Running a local AI-powered browser automation stack requires a reasonably capable machine. A system with 16 GB of RAM or more is recommended to run Docker containers and the language model efficiently. We’ll also need 20–25 GB of available disk space, preferably on an SSD, to accommodate Docker images and downloaded models. While a dedicated GPU can significantly improve model inference speed, it is entirely optional, and the setup works well on modern CPUs. For this tutorial, I’m using the following configuration: Operating system: macOS (M2 Pro)Memory: 16 GB RAM We can have the same setup on Windows and Linux, with only minor platform-specific differences in the installation steps. Setting Up the Environment for the Local AI QA Engineer Docker, Node.js, and Git are widely used development tools, and detailed installation guides for each are readily available online. Installing Ollama To install Ollama, either download the installer from the official website or use the installation command provided for your operating system. For macOS, it can also be installed using the following Homebrew command: Plain Text brew install ollama Once the installation is complete, it can be verified by running the following command in the terminal: Plain Text ollama --version Installing Qwen3:8b Qwen3:8b is chosen for this setup because it offers a strong balance of reasoning, code generation, and performance, making it ideal for Playwright TypeScript test generation, AI agents, MCP integration, and modern QA automation workflows while running efficiently on a local machine. However, other higher models can also be chosen if you know a better one. Another factor in choosing this model was the available system memory. Since my machine has 16 GB of RAM, some memory also needs to be reserved for other tools used in this setup, such as Docker, LibreChat, and Playwright. We need to start Ollama first by running the following command from the terminal. (It should be kept running in the background): Plain Text ollama serve Open a new terminal and run the following command to pull the Qwen3:8b model: Plain Text ollama pull qwen3:8b It should take some time to complete the pull, as the model is around 5.2GB. Once the download completes, we can check the model by running the command: Plain Text ollama list It should list the model downloaded. Next, we can quickly verify by running the model using the command: Plain Text ollama run qwen3:8b Once the model starts, it will prompt you to enter a query. To verify that everything is working correctly, try a simple prompt such as “What is 2 + 2?”. Observe how the model processes the request and generates its response. If the setup is successful, it should return the correct answer, 4, confirming that the model has been downloaded, installed, and is functioning properly. To stop the model, type “/bye” in the prompt, and it should exit. Qwen3:8b provides a good balance between performance and resource usage, making it a suitable choice for this hardware configuration. If more RAM is available, you can opt for larger LLMs that offer stronger reasoning and coding capabilities. Installing LibreChat With Docker LibreChat is an open-source AI platform that provides a unified and customizable interface for interacting with multiple AI models. It enables us to manage all our AI conversations from a single application while supporting features such as AI agents, Model Context Protocol (MCP) servers, custom tools, and integrations with both local and cloud-based LLMs. LibreChat acts as the front-end chat interface that communicates with the locally running Qwen3:8b model through Ollama. It allows us to execute AI-powered browser automation workflows entirely on our local machine. Follow the steps below to install LibreChat: Step 1: Clone the LibreChat GitHub Repository The repository can be cloned by running the following command: Plain Text git clone https://github.com/danny-avila/LibreChat After cloning the repository, navigate to the LibreChat folder, copy the .env.example file, and create a new .env file from it. Plain Text cd LibreChat cp .env.example .env Let's keep the .env file as it is, using the default values. Step 2: Connect Ollama to LibreChat Ollama can be connected to LibreChat by updating its configuration in the “librechat.yaml” file. The example file is already available in the cloned repo. Run the following command to copy librechat.example.yaml and create librechat.yaml. Plain Text cp librechat.example.yaml librechat.yaml Update the following configuration in the file to connect Ollama to LibreChat: YAML endpoints: custom: - name: "Ollama" apiKey: "ollama" baseURL: "http://host.docker.internal:11434/v1" models: default: - "qwen3:8b" fetch: true titleConvo: true titleModel: "current_model" summarize: false summaryModel: "current_model" modelDisplayLabel: "Ollama" Make sure that this configuration is added to the “custom” block, which falls under the “endpoints” block. This configuration adds Ollama as a custom AI endpoint in LibreChat. The baseURL tells LibreChat where to connect to the Ollama API, while the default model specifies that Qwen3:8b should be used by default. Since LibreChat is running inside a Docker container while Ollama is running directly on the host machine, we use http://host.docker.internal:11434/v1 instead of localhost. The special hostname host.docker.internal allows the Docker container to access services running on the host system, enabling LibreChat to connect to the locally running Qwen3:8b model through Ollama. Setting fetch: true allows LibreChat to automatically detect and display all models available in Ollama. The remaining options configure the user interface by generating conversation titles using the current model, disabling conversation summarization, and displaying the endpoint with the label Ollama in the LibreChat interface. Step 3: Mount the Configuration in the docker-compose-override.yml The docker-compose-override.yml can be copied and created in the same way as we did “librechat.example.yaml”. Plain Text cp docker-compose.override.yml.example docker-compose.override.yml The following block should be updated in the docker-compose.override.yml file. YAML services: api: volumes: - ./librechat.yaml:/app/librechat.yaml This file mounts the custom “librechat.yaml” configuration file into the LibreChat container. By mapping ./librechat.yaml to /app/librechat.yaml, Docker ensures that LibreChat uses the custom configuration each time the container starts. This approach allows us to modify settings, such as custom endpoints and AI models, without rebuilding the Docker image. Step 4: Start the LibreChat Application Using Docker Compose The LibreChat application can be started using the following command: Plain Text docker compose up -d It will take some time for the Docker images to download, and containers will start. Run the following command from the terminal to check the Container status: Plain Text docker ps -a This command displays the status of all Docker containers. If any container is unhealthy or encounters an issue, its status will be clearly indicated in the output. In case any container is unhealthy or encounters an issue, the following command can be run to check its logs: Plain Text docker logs <container name> Once all the containers are started successfully, open a new browser and navigate to http://localhost:3080 to start LibreChat. Since we are accessing LibreChat for the first time, we will be prompted to register and create a new user account. After completing the registration process, we can sign in and start using the application. Step 5: Selecting Ollama > Qwen3:8b Model By default, the gpt-5.5 model is selected. To select the Qwen3:8b model: Click on the gpt-5.5 modelSelect Ollama > Qwen3:8b Once the Qwen3:8b model is selected, we can verify if it is working by sending a simple prompt such as “What is 2+2?” Make sure the command “ollama serve” is already running in the terminal in the background, else the model Qwen3:8b won't work on LibreChat. Once we receive a successful response from the model, we can confirm that the Qwen3:8b model has been configured and integrated successfully with LibreChat. Install Playwright MCP Server Playwright MCP can be installed by running the following command in the terminal: Plain Text npx @playwright/mcp@latest \ --host 0.0.0.0 \ --allowed-hosts "*" \ --port 8931 \ By default, Playwright MCP listens only on localhost, which means applications running inside Docker (like LibreChat) cannot connect to it. Using --host 0.0.0.0 makes the server accessible from Docker containers, while --allowed-hosys "*" allows requests from host.docker.internal instead of restricting access to localhost. Once the Playwright MCP server is started, we can leave it running in the terminal. After the Playwright MCP server starts, it shows the following message at the bottom: “For legacy SSE transport support, you can use the /sse endpoint instead”. We will configure the Playwright MCP server using the SSE (Server-Sent Events) transport. Although Playwright MCP also supports the Streamable HTTP transport, LibreChat currently does not support connecting to it via the /mcp endpoint. Therefore, the SSE transport is used to establish a reliable connection between LibreChat and the Playwright MCP server. Configure Playwright MCP Server in LibreChat Playwright MCP server can be added to LibreChat by updating the following configuration in the “librechat.yaml” file. YAML mcpServers: playwright: type: sse url: http://host.docker.internal:8931/sse timeout: 120000 This configuration registers the Playwright MCP server with LibreChat. The type: sse setting specifies that the connection uses the Server-Sent Events (SSE) transport, while the url points to the Playwright MCP server running on the host machine. The hostname host.docker.internal allows the LibreChat Docker container to communicate with services running outside the container. The timeout: 120000 sets the request timeout to 120 seconds, giving the AI agent sufficient time to complete browser automation tasks before the connection expires. However, the timeout can be extended to 15–20 minutes or more, as there is no harm in doing that. YAML mcpSettings: allowedDomains: - 'host.docker.internal:8931' - 'localhost:8931' The mcpSettings configuration also needs to be added under the ‘actions’ block in the “librechat.yaml” file. The mcpSettings.allowedDomains section defines the list of trusted MCP server endpoints that LibreChat is allowed to connect to. By including both host.docker.internal:8931 and localhost:8931, LibreChat can establish a secure connection to the Playwright MCP server, whether it is accessed from within the Docker container (host.docker.internal) or directly from the host machine (localhost). Any MCP server not included in this list will be blocked, providing an additional layer of security. Restart the LibreChat app so it reads the newly configured Playwright MCP server: Plain Text docker compose restart That, or we can also shut down the already running LibreChat and start it again by using the commands below: 1. To shut down LibreChat: Plain Text docker compose down 2. To start it again: Plain Text docker compose up -d After restarting LibreChat, log in and navigate to the home page, and follow the steps below: Click on the MCP Settings menu on the left-hand menu panel.In the MCP Settings window, click on the “+” button to add MCP. Fill in the details for adding the Playwright MCP server; make sure to add the following settings: MCP server URL: http://host.docker.internal:8931/sseTransport: SSEAuthentication: NoneTick the “I trust this application” checkbox. Click on the “Create” button to save the details. Make sure that the Playwright MCP server is started and running on the terminal as discussed in the earlier section Click Connect for the newly created MCP server to establish the connection and begin using it. If everything is fine, a message should be displayed on successful connection. Understanding Model Context Protocol (MCP) By itself, a large language model (LLM) is limited to generating text. It can answer questions, explain concepts, write code, or summarize information, but it cannot directly interact with external systems or perform real-world actions. Model Context Protocol (MCP) changes this by enabling AI models to communicate with external tools and services through a standardized interface. Instead of simply providing suggestions, an AI model can execute tasks such as interacting with browsers, reading files, querying databases, or creating pull requests. Think of MCP as USB for AI A simple way to understand MCP is by comparing it to the USB standard. Before USB became the universal standard, every hardware manufacturer used its own proprietary connector. Printers, keyboards, cameras, and other peripherals all required different cables and custom software integrations. This made connecting devices unnecessarily complicated. USB solved this problem by introducing a common communication standard. Once both the computer and the device supported USB, they could communicate regardless of the device type. Whether you connected a keyboard, webcam, microphone, or external hard drive, the same protocol handled the communication. MCP brings the same level of standardization to AI systems. Without MCP, every AI application requires building and maintaining custom integrations for every external tool it wants to use. If we switch to a different AI application, those integrations often need to be recreated from scratch, resulting in duplicated effort and increased maintenance. A collection of awesome servers for the Model Context Protocol can be found at mcpservers.org. With MCP, tools expose a common interface that any MCP-compatible AI application can use. The AI model only needs to understand the MCP protocol, while the implementation details are handled by the individual MCP servers. Why MCP Matters for QA Automation For QA Automation Engineers, MCP unlocks the ability to automate complete testing workflows rather than isolated tasks. Consider the following request: “Read the Jira story, generate Playwright tests, execute them, analyze any failures, and create a GitHub pull request.” With MCP, the AI agent can coordinate multiple tools to complete the entire workflow. For example, it can: Read the user story from JiraAccess the application’s source code from GitHubGenerate Playwright TypeScript testsExecute the tests in a real browserCapture screenshots, logs, and execution reportsCommit the generated tests to GitHubUpdate the Jira ticket with the test results Each of these actions may be handled by a different MCP server, such as a Jira MCP server, GitHub MCP server, and Playwright MCP server. From the AI model’s perspective, however, every server is accessed using the same standardized MCP protocol. This standardization is what makes MCP so powerful. Rather than building custom integrations for every tool, AI systems communicate through a single, consistent protocol. As a result, MCP servers for Playwright, GitHub, databases, and many other services can be integrated and used in a uniform, scalable manner, significantly simplifying the development of AI-powered automation workflows. Creating an AI Agent With Playwright MCP Server in LibreChat for Automation Testing Let’s create a new AI Agent for browser automation testing with Playwright MCP using the steps below: Step 1: Click on the Agent Builder menu on the left-hand menu panel. Step 2: Enter the following mandatory details to create a new agent: Name: Provide a meaningful name to the agent.Category: Provide a category to the agent.Model: Select Qwen3:8bMCP Servers: Click on the Add MCP Server Tools button > Select the Playwright MCP Server that we created in the earlier section.Click on the Save button. Step 3: Update the model parameters. Clicking on the Model field, which has Qwen3:8b selected, should open the Model Parameters page. The following parameters can be set using this page: Provider: OllamaModel: Qwen3:8bTemperature: 0.2Top P: 0.85Frequency Penalty: 0.00Presence Penalty: 0.00Reasoning Effort: MediumReasoning Summary: Auto Click on the Save button to set the parameters. Step 4: Setting the instructions for the AI agent. The Following instructions can be pasted into the Instructions field in the Agent Builder window, or a “SKILL.MD” file can be created and uploaded using the Skills section of this agent. Markdown # Skills for the Local AI Agent for automation testing You are an expert QA Automation Engineer controlling a browser through Playwright MCP. Your goal is to execute browser actions safely and reliably. ## Tool Usage Rules - Do not run all MCP tools at the same time - Use only one Playwright MCP tool at a time. - Wait for the result of each tool before deciding the next action. - Never assume the page state. - Inspect the current page before interacting. - Do not start the next MCP tool unless the first one is complete ## Navigation Rules Treat the following actions as navigation-triggering actions: - Clicking Login, Submit, Continue, Save, Next, Checkout, etc. - Clicking any hyperlink. - Form submission. - Any action that changes the URL or reloads the page. - Wait until the page is fully loaded before making another tool call. After any navigation-triggering action: 1. Do not call any DOM inspection tool immediately. 2. Wait until the page has completely loaded. 3. Wait for the URL to stabilize if it changes. 5. Continue only after the new page is available. 6. Never inspect the previous page after navigation. ## Rules for locating web elements - Take a fresh snapshot to inspect the current page - Do not use XPath locator strategy - Use the same field name to locate elements, do not hallucinate and add prefix or suffix to field names - Use Semantic locator strategy: getByRole, getByText, getByLabel, getByPlaceHolder, getByAltText, getByTitle, getByTestId - Never use brittle CSS selectors such as .btn-primary, .container > div:nth-child(2), #content div span, or auto-generated classes. - Avoid nth() unless there is no unique locator. ## Interaction Rules - Verify and confirm that an element exists before interacting. ## Error Recovery If any Playwright tool fails: - Stop issuing new actions. - Inspect the current page. - Check Interaction Rules - Determine whether navigation has occurred. - Retry only if the page state confirms it is safe. - Do not repeat the same action more than once without confirming that the page state has not changed. Never repeat the same click more than once without checking the current page. ## Important If a click causes navigation, always assume the previous execution context has been destroyed. Do not read the DOM until the new page has fully loaded and a fresh snapshot has been obtained. Show a summary of test execution with the step count and pass or fail status - Run only the steps that are provided; do not hallucinate - Any deviation from these rules is not acceptable - Do not generate any additional steps - Always prioritize stability over speed. Providing instructions to an AI agent helps define its behavior, responsibilities, and the boundaries within which it should operate. These instructions act as persistent guidance, ensuring the agent follows consistent practices every time it performs a task instead of relying solely on the user’s prompt. For detailed setup instructions and troubleshooting guidance, refer to the GitHub repository. With these steps, the local AI agent is now ready to take commands. Running the AI Agent for Browser Automation To start using the AI Agent, click on New Chat.Click on the model name dropdown and select My Agents > The name of the agent that you created. Let’s use the following simple prompt and see how it works. Plain Text open http://playwright.dev verify the page title Once the prompt is submitted, we can observe the browser as the AI agent begins executing the task. The agent invokes the Playwright MCP server, which automatically launches a browser and performs the requested actions to navigate to the website and interact with the page. After the task is completed, Qwen3:8b analyzes the outcome and returns the results directly in the LibreChat conversation, demonstrating browser automation powered by Playwright MCP and Qwen3:8b. Let’s run another prompt for a login test scenario: Plain Text Navigate to https://parabank.parasoft.com/parabank/index.htm Locate "Username" field using "name=username" Enter "john" into the "Username" field. Locate "Password" field using "name=password" Enter "demo" into the "Password" field. Locator "Log In" button using "input[type="submit"] Click on the "Log In" button Verify that the "Accounts Overview" page is displayed This prompt also takes some time to understand the request before execution begins. It is important to note that the clearer and more specific the prompt, the more efficiently the AI agent can interpret and execute it. Well-structured prompts reduce ambiguity, minimize the chances of hallucinations, and typically result in faster execution and more accurate outcomes. As a best practice, break complex tasks into clear, sequential instructions whenever possible to improve the agent’s reliability and overall performance. As shown in the screenshot above, the AI agent invoked five tools from the Playwright MCP server to interact with the application and complete the requested workflow. It navigated to the website, located the username and password fields, entered the provided credentials, and submitted the login form. Finally, it verified that the login was successful by confirming that the “Accounts Overview” page was displayed. Since this setup runs entirely on a local machine, the AI agent takes approximately one minute to begin execution and around 4–5 minutes to complete a simple scenario. For more complex scenarios involving multiple steps, validations, or integrations, the AI agent is expected to take longer to analyze the request and complete the execution. But Execution time can be significantly reduced by running the setup on a machine with more powerful hardware, such as additional RAM, a faster CPU, or a dedicated GPU. Watch the step-by-step YouTube tutorial for Building your Local AI QA Engineer. Final Words Building a local AI QA engineer with Docker, Ollama, LibreChat, and Playwright MCP is an excellent way to explore the future of AI-powered software testing while keeping complete control over the data and infrastructure. By running everything locally, we eliminate recurring API costs, improve data privacy, and create a flexible environment for experimenting with AI-assisted browser automation using natural language. This setup is only the beginning of what’s possible. As we become more familiar with MCP and AI agents, the local QA assistant can be extended by integrating tools such as GitHub, Jira, databases, or custom MCP servers to automate even more of the testing workflow. Happy AI-powered testing!!

By Faisal Khatri

CORE

Top 10 Best Places to Prepare for Your Next Data Engineer Interview

Landing a data engineering role means clearing a gauntlet that no other software discipline has to face all at once: airtight SQL, production-grade Python, data modeling instincts, distributed-compute fluency (Spark, warehouses, ETL), and system design that has to survive real data volume. Generic coding prep barely scratches the surface, and "just grind LeetCode" advice falls apart the moment an interviewer asks you to model a slowly changing dimension or reason about a skewed join. So we did the work. We evaluated the resources data engineers actually use, judged on five things that matter: relevance to the DE interview loop, depth of practice, realism of the questions, feedback quality, and price. Below is the ranked list. A quick note on methodology: this ranking favors resources that target the data engineering loop specifically, not generic algorithm grinding. That bias is intentional, and it is why the order may surprise you. 1. DataDriven.io Most "interview prep" platforms were built for generic SWE roles and bolt on a SQL section as an afterthought. This one was built from the ground up for the data engineering loop. The catchphrase you will hear repeated in DE communities is that DataDriven.io is LeetCode for data engineers, and it fits: instead of inverting binary trees, you are writing window functions against realistic schemas, designing star schemas, debugging an ETL transform, and reasoning about partitioning, all in an in-browser SQL and Python sandbox that runs your query against real data and tells you exactly where it broke. It is also the rare place where the whole product is built for the job rather than adjacent to it, which is why datadriven.io is great for data engineer interview prep specifically: SQL practice that ramps to multi-CTE analytics, a deep set of Python practice problems, plus data modeling, dimensional modeling, PySpark, and system-design tracks, with execution-based feedback and a difficulty curve that reaches the staff-level questions that actually separate offers from rejections. Verdict: The most targeted, realistic data engineering interview practice available today. Earns the top spot. 2. "Cracking the Coding Interview" (the book, by Gayle Laakmann McDowell) A deserved classic, and intentionally a book rather than a website. CTCI is still the best single artifact for understanding how technical interviews are actually structured: how the conversation flows, how to think out loud so the interviewer can follow your reasoning, how to recover when you get stuck, and how to handle the behavioral and negotiation segments that strong candidates routinely fumble. Most people lose offers not because they could not solve the problem but because they could not show their work, and this book is the canonical fix for that. Where it falls short for our purposes is scope. It will not teach you windowed SQL, slowly changing dimensions, or how to design a lakehouse, and its algorithm focus skews toward generalist software roles rather than the data engineering loop. The data structures and big-O chapters are still worth a pass because algorithm screens do show up, but treat them as a refresher, not your main event. Read CTCI once early in your prep to fix your interview mechanics, internalize the communication patterns, then spend the rest of your time on hands-on, domain-specific platforms. Verdict: Essential reading for interview mechanics; not a substitute for domain practice. 3. "Designing Data-Intensive Applications" (the book, by Martin Kleppmann) If CTCI teaches you how to interview, "DDIA" teaches you what a data engineer is actually supposed to know. Replication, partitioning, consistency models, batch versus stream processing, storage engine internals, the failure modes of distributed systems: this is the conceptual backbone of nearly every data engineering system design round. When an interviewer asks why you would choose a log-structured merge tree over a B-tree, or how you would keep two datastores in sync without losing events, the answers live in these pages. It is dense, and it is emphatically not an interview drill book. You will not find practice questions, and you cannot cram it the night before. What it gives you instead is judgment: the candidate who has internalized DDIA answers "how would you design this pipeline" with the calm of someone who has already thought through the tradeoffs, names the failure cases before being prompted, and explains why a choice holds up under real data volume. Read it slowly over weeks, ideally early in your prep, and pair it with a hands-on platform so the concepts attach to actual queries and schemas rather than floating as theory. Verdict: The definitive conceptual reference. Read it slowly, alongside real practice. 4. LeetCode The default destination, and it earns its spot for one practical reason: the Database problem set is sizable, the algorithm catalog is enormous, and the platform's brand means a large share of companies still pull their initial coding screen straight from it. If your target company is known to run a generic algorithm round before the data-specific rounds, you need exposure here, and the sheer volume of problems plus community discussion means you will rarely be surprised by a pattern you have never seen. The catch for data engineers is that LeetCode was built for the algorithm interview, not the DE loop. Its SQL section is genuinely solid but secondary; the questions are puzzle-shaped rather than drawn from real schemas, and you will not find data modeling, ETL design, dimensional modeling, or Spark anywhere on the platform. There is also a real failure mode here: candidates over-invest in LeetCode because it is comfortable and gamified, then walk into a DE loop under-practiced on the things that actually decide it. Use it deliberately to clear the algorithm gate and to keep your raw coding sharp, then move the bulk of your hours to resources that target data engineering directly. Verdict: Necessary for the algorithm screen; thin for the data-engineering-specific rounds. 5. HackerRank HackerRank is where a surprising number of companies host their take-home and timed online assessments, so practicing in its environment carries a payoff most resources cannot offer: you get comfortable with the exact editor, the exact test-case runner, and the exact time-pressure UI you may actually be scored in. For an assessment you cannot retake, that familiarity is worth real points, because fighting an unfamiliar interface while the clock runs is a self-inflicted way to lose. Its SQL and problem-solving tracks are beginner-friendly, well-structured, and free to work through. The ceiling, though, is lower than you want for a senior DE loop. The problems lean academic and self-contained rather than job-realistic, the SQL rarely reaches the messy multi-table analytics that real interviews probe, and there is nothing on modeling, pipelines, or system design. The smart way to use HackerRank is as format rehearsal: run a few timed sets so the assessment environment feels routine, then build your actual depth somewhere that mirrors the work. Do not let a green checkmark on an easy problem set convince you that you are loop-ready. Verdict: Great for getting comfortable with the testing environment; limited depth. 6. SQLZoo A long-running, completely free interactive SQL tutorial that runs entirely in the browser with no signup, no setup, and no paywall. It walks you from SELECT basics through joins, grouping, subqueries, and window functions, with short hands-on exercises after each concept so you are writing real queries from the first lesson rather than just reading about them. For anyone whose SQL has gone rusty, or who learned it informally and has gaps they cannot quite name, it is the most painless way to rebuild muscle memory before stepping up to interview-grade problems. It is a teaching tool, not an interview platform, and you should treat it as exactly that. The problems stay introductory, the datasets are small and tidy, and there is nothing on data modeling, ETL, pipelines, or system design — the parts of the loop that actually separate data engineers from analysts. Its value is as a fast diagnostic and warm-up: work through the sections that feel shaky, confirm your fundamentals are solid, then graduate to harder, execution-based practice against realistic schemas. Linger here too long, and you will plateau well below where a real interview will push you. Verdict: A friendly free SQL primer; foundational rather than interview-level. 7. "Python for Data Analysis" (by Wes McKinney) Written by the creator of pandas, this is the reference for the kind of data-wrangling Python that shows up constantly in DE take-homes and pairing rounds: reshaping, grouping and aggregating, merging on imperfect keys, handling missing values, parsing dates, and cleaning the kind of messy tabular data that never looks like a tidy LeetCode input. Many data engineering interviews quietly assume this fluency, then hand you a notebook and a dirty CSV and watch how you move; if your Python is sharp on algorithms but clumsy on real data manipulation, this book is exactly the gap-closer. It is a library-and-technique book, not interview prep, and it will not touch SQL, data modeling, distributed compute, or system design. There are also no interview questions to grind, which is fine, because its job is to make the tools second nature so that during a timed exercise you are reasoning about the problem instead of fumbling for the right pandas idiom. Read the chapters on data loading, cleaning, and group operations, keep it nearby as a reference, then go apply the techniques in hands-on practice against problems that actually resemble the job. Verdict: The definitive practical Python reference for data work; not a drill book. 8. "Fundamentals of Data Engineering" (the book, by Joe Reis & Matt Housley) Another deliberate book pick, and the best single survey of the modern data engineering lifecycle: generation, ingestion, storage, transformation, and serving, plus the cross-cutting concerns like orchestration, data quality, and governance that interviewers increasingly probe. Where DDIA goes deep on systems internals, this book goes broad on how the pieces fit together into a working data platform, which is precisely the framing you want for the "walk me through how you'd build X" and "what would you consider before choosing this approach" portions of a loop. It is a framework-and-vocabulary book, not a practice book, and that is both its strength and its limit. It will give you the mental model and the shared language to discuss tradeoffs like a practitioner, which makes you sound, accurately, like someone who understands the field. But it contains no exercises, so reading it alone will not build the hands-on skill an interviewer also tests. Use it to organize everything you know into a coherent lifecycle, fill the conceptual gaps, then go write the queries and design the schemas somewhere that gives you real feedback. Verdict: The best lifecycle overview in print; conceptual, not hands-on. 9. Mode SQL Tutorial A free, well-regarded interactive SQL tutorial built by an analytics company, which shows in its framing: it teaches SQL the way analysts and engineers actually use it, oriented around answering real questions from data rather than solving abstract puzzles. It runs in the browser, takes you from the basics through intermediate analytics queries including aggregation and the early window-function territory, and the explanations are unusually clear about why a query is shaped the way it is. For someone shoring up SQL foundations before diving into harder problems, it is one of the cleanest no-cost on-ramps available. Like SQLZoo, it is a tutorial rather than an interview-prep platform, so it stops well short of the difficulty a real DE loop will throw at you, and it covers none of the modeling, pipeline, or system-design ground. It is best read as a companion to a hands-on platform: use Mode to internalize the analytical mindset and clean up your SQL fundamentals, then take that foundation into execution-based practice where the problems are harder, the schemas messier, and the feedback tells you exactly where your query went wrong. Verdict: A clean free SQL on-ramp; foundational rather than interview-level. 10. Pramp/Interviewing.io (mock interviews) Rounding out the list: peer and expert mock interviews. All the solo practice in the world cannot reproduce the specific pressure of explaining your reasoning out loud to a real human while a clock runs and someone is judging you, and that pressure is exactly where otherwise-prepared candidates fall apart. A handful of mock loops surface the weaknesses you cannot see in yourself: the long silences, the jumping to code before clarifying the question, the inability to narrate a tradeoff. Pramp pairs you with peers for free, while Interviewing.io connects you with experienced interviewers, often anonymously, for higher-fidelity feedback. The honest limitation is supply and specificity. Data-engineering-focused interviewers are scarcer than generalist software ones, so depending on availability, you may land in an algorithm or general system-design mock that only partially mirrors a true DE loop. That is still worth doing, because the communication skills, the structure, the clarifying questions, the calm narration, transfer directly regardless of the exact problem. Schedule one or two once your technical prep is underway, treat the feedback as data, and fix the delivery habits well before the interview that counts. Verdict: Best for rehearsing delivery and nerves; DE-specific matches can be hit-or-miss. How to Actually Use This List You do not need all ten. A focused plan beats a scattered one: Build the foundation. Skim CTCI for interview mechanics and start DDIA for concepts.Do the reps where it counts. Spend the bulk of your time on hands-on, DE-shaped practice that maps directly onto what you will be asked (see #1).Patch specific gaps. Use LeetCode for the algorithm screen, SQLZoo or the Mode tutorial to shore up SQL, and a mock interview or two to rehearse out loud. The candidates who get offers are not the ones who consumed the most content. They are the ones who practiced the actual job. Pick the resources that put you closest to it, start today, and write more queries than you read. Good luck with your loop.

By Rahul Han

Building an Idempotent Job Queue in Node. js That Never Runs the Same Task Twice

Today, in modern backends, you probably have those distributed job queues for everything, including sending emails, processing payments, generating reports, and syncing data to third parties. As soon as you add retries to handle transient failures, however, you inherit a hard problem: how do you ensure that when the network, worker, or broker can fail at any point, your job runs exactly once? The short answer is: "exactly once delivery" is a great concept, but in practice it's mostly fiction given the nature of distributed systems. What you really can make is at-least-once delivery + idempotent processing, yielding exactly once effects. This article demonstrates how to accomplish this in Node.js with a tangible, functioning implementation. The Problem: Retries Cause Duplicates Take a worker that charges the customer and then marks the job completed TypeScript async function processJob(job) { await chargeCustomer(job.customerId, job.amount); await markJobComplete(job.id); } This seems fine until you consider that the work crashes after chargeCustomer succeeds but before markJobComplete executes. Because the queue does not receive an acknowledgement, it redelivers the job. The customer gets charged twice. This is not a rare edge case. Do any significant amount of throughput and workers fall over, containers reschedule, network calls default after the server has already worked its way through them. If you have side effects in your job, then you can always assume any job may be delivered more than once. The Solution: Idempotency Keys The main concept is to give every job created a unique, deterministic idempotency key and log the output of processing that key. The worker only checks if a key has been processed before doing any work. If so, it simply returns the result that was saved and does not redo the work. Here is the schema for how we can keep track of processed jobs. TypeScript CREATE TABLE processed_jobs ( idempotency_key VARCHAR(255) PRIMARY KEY, status VARCHAR(20) NOT NULL, -- 'in_progress' | 'completed' result JSONB, created_at TIMESTAMPTZ NOT NULL DEFAULT now(), completed_at TIMESTAMPTZ ); The job must encode its sensitive payload and key, not randomly generated at enqueue time. Good keys will be things like charge:order_12345, which will hopefully be stable across retries of the same logical operation. A Working Implementation The trick is to acquire the key atomically before doing anything useful. To claim the job, we execute a single atomic operation in PostgreSQL, which is an INSERT... ON CONFLICT DO NOTHING: TypeScript const { Pool } = require('pg'); const pool = new Pool(); async function processIdempotent(idempotencyKey, work) { const client = await pool.connect(); try { // Step 1: Try to claim the key atomically. const claim = await client.query( `INSERT INTO processed_jobs (idempotency_key, status) VALUES ($1, 'in_progress') ON CONFLICT (idempotency_key) DO NOTHING RETURNING idempotency_key`, [idempotencyKey] ); // Step 2: If we did NOT claim it, someone else already did. if (claim.rowCount === 0) { const existing = await client.query( `SELECT status, result FROM processed_jobs WHERE idempotency_key = $1`, [idempotencyKey] ); const row = existing.rows[0]; if (row.status === 'completed') { return row.result; // Return the cached result — no double work. } // Still in progress elsewhere — let the queue retry later. throw new Error('JOB_IN_PROGRESS'); } // Step 3: We own the key. Do the actual work. const result = await work(); // Step 4: Record the result. await client.query( `UPDATE processed_jobs SET status = 'completed', result = $2, completed_at = now() WHERE idempotency_key = $1`, [idempotencyKey, result] ); return result; } finally { client.release(); } } Now the worker becomes: TypeScript async function processJob(job) { return processIdempotent(`charge:${job.orderId}`, async () => { const charge = await chargeCustomer(job.customerId, job.amount); return { chargeId: charge.id }; }); } The key is already completed, and whenever this job is delivered the second (or more) time it will return the chargeId that was previously stored without charging again. Handling the Stuck "in_progress" Case The last failure mode that remains is where a worker picks a key, sets it to in_progress, and then dies without completing. Now this key is stuck, and any retry gives JOB_IN_PROGRESS forever. The solution is an expiration-lease for the lease. 1. Add locked_until column, make expired lock reclaimable: TypeScript const claim = await client.query( `INSERT INTO processed_jobs (idempotency_key, status, locked_until) VALUES ($1, 'in_progress', now() + interval '5 minutes') ON CONFLICT (idempotency_key) DO UPDATE SET locked_until = now() + interval '5 minutes', status = 'in_progress' WHERE processed_jobs.status = 'in_progress' AND processed_jobs.locked_until < now() RETURNING idempotency_key`, [idempotencyKey] ); It only requires a lock if the circuit is in progress and its lease has timed out, which means that some worker abandoned it earlier. The completed jobs will never be reclaimed, because the WHERE excludes them. Why not simply use a distributed lock One of the most common instincts here is to grab ourselves a Redis lock (if not using redis-lock, do SETNX with a TTL). Despite a lock being a solution for mutual exclusion, they do not solve idempotency by themselves. The job is already done, but because it uses a lock to prevent two workers from running at once. If you only use a lock, the job will be reprocessed when the lock expires and a redelivery is attempted. What you need is a permanent record of completion and that is what the processed jobs table provides. Locks and idempotency keys address two separate problems, yet durable systems typically require both. Takeaways Assume at-least-once delivery; make each job handler idempotent.Use the intent from job to derive idempotency keys; ensure they are stable across retries.Store results and claim keys atomically with INSERT... ON CONFLICT so duplicates return the cached result.Lease with an expiration because crashed workers should not block a key forever. Idempotency is certainly not useful, but it helps the queue to be the difference between something you can trust and a facility that will silently double charge your customers because of load. Treat it as a first-class citizen, because adding it via retrofitting after failing is way worse.

By Bilal Azam

The 20 Software Engineering Laws

Most engineers learn these laws the hard way. When you try to rewrite something and it doesn’t deliver, or when a project is already late, adding engineers to the team will just make it fail faster. Sometimes, when you start using a metric to measure progress, the whole team will start trying to manipulate it. Then, six months later, someone mentions a 1975 law that addresses exactly what happened. I paid a price to learn this, too: I spent half my career learning these lessons the hard way, as many others probably did. The twenty laws listed below are the ones I refer to most often, although there are more (more on this later). Software development laws explain what is happening, what is about to happen, and what will not work no matter how hard you try. Some of these laws are sixty years old. They still apply to software development in 2026, and they will still apply in 2036 because they are not really about software. They are about people working together to build things under time pressure (basically, a lot of them are just laws of human nature). These laws are not rules that tell you what to do. They tell you what is already happening, but you still have to make the decisions. These laws just help you understand what is going on. Each of these laws made the list because I have experienced them myself. My book covers all fifty-six laws. If you only have time to remember twenty software development laws, these are the ones that I think are important. In particular, we will talk about the following laws: Gall’s Law: A complex system that works is always built from a simple system that worked first.KISS: Keep it simple. Anything beyond that is overhead.Conway’s Law: Organizations design systems that mirror their communication structure.Hyrum’s Law: With enough users, every observable behavior of your API becomes someone’s dependency, no matter what the contract says.CAP Theorem: A distributed system can guarantee only two of: consistency, availability, and partition tolerance.Zawinski’s Law: Every program expands until it can read mail. The ones that cannot are replaced by ones that can.Brooks’s Law: Adding people to a late software project makes it later.Ringelmann Effect: Individual output drops as team size goes up.Price’s Law: Half the work is done by the square root of the people.Dunning-Kruger Effect: The less you know about something, the more confident you tend to be.Hofstadter’s Law: It always takes longer than you expect, even when you account for Hofstadter’s Law.Parkinson’s Law: Work expands to fill the time available.Goodhart’s Law: When a measure becomes a target, it stops being a good measure.Gilb’s Law: Anything you need to quantify can be measured in some way that beats not measuring it.Knuth’s Optimization Principle: Premature optimization is the root of all evil.Amdahl’s Law: The speedup from parallelism is limited by the sequential part.Murphy’s Law: Anything that can go wrong will go wrong.Postel’s Law: Be conservative in what you send, liberal in what you accept.Sturgeon’s Law: 90% of everything is crap.Cunningham’s Law: The fastest way to get the right answer online is to post the wrong one. So, let’s dive in. How Systems Get Built 1. Gall’s Law A complex system that works is always built from a simple system that worked first. Systems do not work as well in real life as they do on paper because many problems do not surface until they hit the real world. These problems only appear when real users interact with systems, and by then, they either work or they do not. Every complex system that works got that way one step at a time. The systems that try to be perfect from the start usually fail. This is why most new versions of systems rewritten from scratch do not work out: teams keep all the features they had before, but lose the simple things that made the old systems good. Examples. Let’s take an example of Instagram. At the start, it was something else, but not a picture-sharing platform. The app was called Burbn, and it had: check-ins, gaming, photo sharing, all stuck together. Then, the founders cut everything except photo sharing, and the stripped-down core became the product. Google Wave went the other way. It launched with chat, email, a forum, and a document editor, all at once. Nobody could tell you what it was for, and it was dead in 15 months. 2. KISS (Keep It Simple, Stupid) Keep it simple. Anything beyond that is overhead. The KISS principle is a reminder that simplicity should be our key goal. If you can solve a problem with a 50-line script vs a complex 500-line solution, KISS favors the simpler solution because each line of code has the potential to cause an error. Why is simplicity so important? Software, in general, is complex to build and must be understood by humans. A simple design is much easier to maintain: new team members can get up to speed faster, bugs are easier to localize, and modifications cause fewer ripple effects. The KISS principle encourages developers to resist “clever” code that does too much at once, and to avoid architecting solutions that address future problems at the cost of current complexity. Example. Let’s say that we have a startup that needs a feature-flag system and decide to build a custom solution. They built it as a separate microservice with its own database, cache, admin UI, WebSocket notifications, and A/B testing support. It introduces a lot of complexity and takes a lot of time to build, which, if something goes wrong, can cause a lot of trouble. What they needed was a JSON config file. This would have taken an afternoon. 3. Conway’s Law Organizations design systems that mirror their communication structure. Your app architecture is already defined and essentially the same as your organization chart. For example, if you have four teams working on a project, you will probably end up with an app that has four parts. If the teams that work on the frontend, the backend, and the data do not communicate, your application will have three parts that do not work well together. If you rewrite your system without changing how your company is organized, you will still have the system, just written in a different language. The other way around works too. You can pick the architecture you want and then create teams that would naturally produce that kind of system. Amazon did this back in the 2000s. They broke their system down into smaller services managed by small teams, which changed how the system and the company worked together. This is called Inverse Conway’s Maneuver. Examples. Many modern AI organizations often split research from application engineering. Then, research optimizes benchmarks, while product ships apps against real users. The output is a model that scores well and a product that doesn’t work, because each side is optimizing for its own communication boundary. The pattern shows up at a small scale, too. A three-person team almost always ships a monolith because the cost of breaking it up is higher than the cost of keeping it together. 4. Hyrum’s Law With enough users, every observable behavior of your API becomes someone’s dependency, no matter what the contract says. The interface contract you wrote is not a proper contract. The real one is what your system actually does, including the parts you never expected to be important. For example, it could be timing, error message text, key order in JSON responses, and the exact bytes of a hash. Someone, somewhere, is depending on all of it. This is why backward compatibility costs so much in mature systems. This means that you actually don’t maintain the API you designed, but the accidental one. Examples. A good example is the SimCity game. I remember well that it had a use-after-free bug that worked fine on Windows 3.x because memory was never actually reclaimed. Then, Windows 95 reclaimed it, and SimCity crashed. Microsoft shipped Windows 95 with a special memory-allocator mode that was activated only when SimCity was running, so the bug would continue to work. Browsers do this at internet scale. Every quirk that web developers built into the platform effectively becomes part of it. The browser can’t change the quirk without breaking half the web. 5. CAP Theorem A distributed system can guarantee only two of the following: Consistency, Availability, and Partition tolerance. Networks fail. In a distributed system, that's not something you design around. It's something you accept. Once a partition happens, you have to pick: block writes to keep data consistent, or keep serving traffic and let replicas drift. Every distributed database makes this call. Most just don't tell you which one. They hide behind labels like "eventually consistent" or "highly available" and leave you to find out during an incident. Examples. MongoDB favors consistency, meaning that when a partition problem occurs, some MongoDB replicas will not accept any data until the entire system is working properly again. On the other hand, Cassandra will keep answering queries even when the replicas do not agree, and it will later fix the inconsistencies. Neither MongoDB nor Cassandra is wrong. They are just making choices about what your system can afford to lose. 6. Zawinski’s Law Every program expands until it can read mail. The ones that cannot are replaced by ones that can. Feature creep is not something that happens during the process. It is actually the process itself. When a tool is good at what it does, and people like it, they start using it all the time. The people in charge of the product want to keep the users engaged and stay on the platform. So the tool begins to take on tasks that are related to it. Over time, the tool becomes really slow and has a lot of unnecessary extra features. Then a new competitor comes along with a simpler version that does exactly the same thing. As the app's popularity grows, more and more unnecessary features are added. Examples. A famous example is Netscape, which started as a browser and ended as a suite with email, news, and a web editor. Firefox came as a fix and stripped it down, got popular, but then added plugins and a developer toolchain. We also remember Slack, which was launched to kill email and now has voice, video, bots, and an app directory. All of this is possible if the product doesn’t have the right north star metrics. How Teams Lose Speed 7. Brooks’s Law Adding people to a late software project makes it later. Software work is not easy to split among team members. When you bring someone new onto the project, it takes them a while to get up to speed, which means your experienced people have to stop what they are doing to help the new person learn. If your project is already behind schedule, adding more people won't make it go faster. It will just make things worse. Frederick P. Brooks said it well: you cannot have a baby in one month just because you have nine women pregnant. Software work is, like that, too. Software work does not get done faster just because you have people working on it. Example. Once, I was a team lead of eight people, and we were always behind schedule. My first thought was to hire two engineers to help us catch up. But in the meantime, while we were searching for new people, two people left us. It seemed that everything was now working better, communication was easier, and we managed to do more than before. So, obviously, the solution was to make the team smaller, not bigger. 8. Ringelmann Effect As teams grow, output per person falls. When many people pull on the rope, each person does not pull as hard. Some of this is because it is hard to work smoothly, and some of it is because people think someone else will do the part. Either way, this pattern is real. It is more extreme than most people think. Examples. A large GitHub study measured this directly. Developers on teams of 2-5 people averaged around 1,850 lines of code a month, while a team of 10 dropped to 1,200. At 50 or more, it was 450. Output per person fell 75%. This is why small teams ship faster than big ones, and why Amazon’s two-pizza rule holds true. It’s a defense against Ringelmann. This is especially true in today's AI-driven world, where productive teams have fewer members than before, as AI is driving up personal and team productivity. 9. Price’s Law Half the work is done by the square root of the people. In a group of 100 people, about 10 people actually do half of the work that matters. If you have a group of 16 people, it is likely that 4 people do most of the work. This is true for every creative field. The people in the group who do most of the work are really important, but the others are important too, because they do what needs to be done to support everyone else. They make sure everything runs properly (sometimes called glue work). So we need both groups, but the problem is that if the top people in your group leave, the group will lose a lot of its ability to get things done. Example. We all know that when Musk took over Twitter, it cut its staff by roughly 50%, and the site kept running. Price’s Law predicted that. What the law did not predict was what the layoffs removed: depth in trust and safety, SRE coverage, and incident response. The top performers kept the lights on. The organization lost the ability to handle the next hard problem, and Twitter quietly asked some laid-off people to come back. Why Plans Drift 10. Hofstadter’s Law It always takes longer than you expect, even when you account for Hofstadter’s Law. Let’s say you need to estimate how long something will take. You think four weeks is an estimate, but then you remember that your guesses are usually too optimistic, so you double it to eight weeks, just to be sure. But in the end, it takes sixteen weeks. Now you think, the next time you will be better, aren’t you? You think it will take sixteen weeks because that's what happened the last time. No, it now takes thirty-two weeks, because things you don’t know about surprise you. These are tasks such as unplanned integration issues or requirement changes. In practice, Hofstadter’s Law explains why techniques like padding estimates, awareness of Parkinson’s Law, and the use of historical data are essential, yet surprises still occur. Example. A good example of the Hofstadter law is the Berlin Brandenburg Airport project. The software integration process was taking much longer than expected, as it involved 75,000 sensors and 50,000 light fittings. The plan was to take 18 months to finish, but they later realized this was not possible and extended the timeline to 30 months. In the end, it took 7 years to complete, with a final cost of €7 billion. This was 2.5x higher than planned, and the airport opened 9 years late. 11. Dunning-Kruger Effect The less you know about something, the more confident you tend to be. Here is the uncomfortable part. The skill you need to do something is the same skill you need to judge how well you did the thing, and this is the problem. People who are not very good at something cannot see what they are doing wrong, so they think they are better at the thing than they really are. Yet, people who are good at it see all the things they are still getting wrong, so they think they are not as good at it as they really are. Examples. When asked when something will be done, new developers often give confident, precise estimates, while experienced developers give ranges (the famous “it depends” answer). The juniors aren’t wrong to be convinced. They simply don’t yet know what they don’t know (unknown-unknowns). People usually get really excited about new technology at first. This is because they have not used it a lot yet. We are seeing this happen with artificial intelligence now. The people who say AI can do anything are usually the ones who do not use it every day, like managers. 12. Parkinson’s Law Work expands to fill the time available. If you give a developer two weeks to do a task that can be done in two days, it will take two weeks to finish. This does not mean the developer is lazy or puts things off. People tend to fill up the time they have. Over the two weeks, the developer will likely spend time making plans, trying things, and adding extra tasks that do not need to be done (gold-plating). But if there was a deadline to have this done in a day, it would probably be done on that day. The thing about Parkinson’s Law is that it says if you give people a certain amount of time to do something, they will probably take all the time to do it. So, teams should set clear and realistic time limits (aka deadline-driven development). However, managers must use it judiciously, combining Parkinson’s insight with realistic scheduling. If you compress timelines too much, you risk running into Hofstadter’s Law, which reminds us that work often still takes longer than expected, even with buffers. Examples. A developer given two months for a one-week task will spend a month prototyping alternatives, another week on architecture debates, and the last three weeks polishing details nobody asked for. If we give the same task, but this time with a clear one-week deadline, it will be shipped in one week. How Metrics Distort Work 13. Goodhart’s Law When a measure becomes a target, it stops being a good measure. We can use many different ways to measure our work, e.g., number of bugs closed, number of incidents, test coverage, or team velocity. When we start measuring people's performance based on these things, they will focus on making those numbers look good instead of actually doing good work. The numbers will go up, but the work will not get any better. This is because when we give people incentives, they will do what gets them the reward, not what we really want. When we measure the wrong thing, people will do the wrong thing to get ahead. Examples. I watched a team get rewarded for lines of code written at the start of 2000, and the number of PRs created some years later. Developers started copy-pasting instead of extracting shared logic. Some created PRs for almost every commit they made. The modern version is AI tokens consumed per engineer (called tokenmaxxing). More tokens are being treated as a sign of productivity. 14. Gilb’s Law Anything you need to quantify can be measured in some way that beats not measuring it at all. Gilb's Law is like the side of the coin to Goodhart’s Law. You can say, when looking at Goodhart’s Law, that having metrics is bad, but that is actually not true. Not having any metrics is even worse than that. If something is important to you, you should try to find a way to measure it, because we cannot improve what we don’t measure (as Peter Drucker famously said). Example. Developer productivity is usually a hard thing to measure, and it always has been. We had many bad metrics, from lines of code to token consumption. But deployment frequency and change lead time give you a signal (as in the DORA metrics for DevOps) as a proxy. What Breaks Under Load 15. Knuth’s Optimization Principle Premature optimization is the root of all evil. Most performance work happens too early and in the wrong place. Teams optimize code paths that never become hot, introduce complexity they never need, and burn time solving a scale problem they may never earn. So the best way is to write the code that works, then check its performance. If there is a problem, a tool will show you where it is. If not, just move on. Examples. I worked at a startup once, where we spent a lot of time setting up Kubernetes. The thing was that we did it to handle millions of users, and we didn’t even have 10 users yet. We were making our infrastructure ready for a load that didn’t exist. Our product features were not even finished. One of my colleagues said that we should make sure 100 people even want our product before we worry about handling millions of users. He was right. We still launched late. 16. Amdahl’s Law The speedup from parallelism is limited by the sequential part. If 10% of your work has to be done in a sequential way, the work will only go 10x faster, no matter how many computers you use. If 50% of the work has to be done one thing at a time, the work will only go twice as fast. The same thing happens with people. If one group of people has to say yes to every decision, about how something is built, that limits how fast your team can work, no matter how many engineers you have. If you add engineers, but they all have to wait for the same group of people to say yes, the line of people waiting just gets longer. Your team of engineers will still be slow because the group of people making decisions is a bottleneck. The work of your team of engineers will only go as fast as the group of people making decisions. Examples. Scaling web traffic by adding more app servers helps until every request hits one shared database or authentication service. Then adding more horizontal scaling doesn’t help. The conversation about AI productivity is hitting the roof now. AI makes coding faster, but you still have to think, check, fix errors, and work together on those steps that can’t be done simultaneously. This sets the limit on how much you can gain in the end. That’s why some engineers see their work speed up by 10 times, and others see a 1.2 times increase. 17. Murphy’s Law Anything that can go wrong will go wrong. In software, Murphy’s Law is often mentioned to explain bugs and production incidents: whatever can go wrong in code (a null pointer, a race condition, a network outage) will eventually manifest, especially in large user bases or at the worst possible time (Friday evening). In practice, this law encourages developers to write more defensive code. This means checking for nulls, handling exceptions, validating inputs, and failing gracefully when errors occur. It also reminds DevOps teams to anticipate failures by implementing monitoring, enabling rollbacks, and maintaining contingency plans. Example. On July 19 2024, CrowdStrike made a change to the Falcon Sensor settings. This change caused a memory issue on Windows machines. It made 8.5 million Windows machines stop working and show a screen. To fix this problem, someone had to log in to each machine and apply the fix, because those machines could not start up. This could be done remotely. And this happened on a Friday morning when no IT staff members were working. It caused problems for airlines, hospitals, and banks. Everything that could go wrong did go wrong on the day, just like Murphy’s Law says. 18. Postel’s Law Be conservative in what you send, liberal in what you accept. This law says that if your server sends HTTP responses, it should format headers exactly per spec. But if your server receives an HTTP request with an uncommon header order or an unusual format, you should still process it rather than drop the connection, as long as you can interpret it safely. Browsers do this at a scale. Most of the HTML on the web is not written correctly, but modern browsers still render it. If they were strict, half the internet would not be found. But there is one thing to consider. Being too liberal has a cost: if everyone accepts anything, problems will never be corrected. There will be just more mess. In security-sensitive code, tolerating input can make it easier for attackers to find. So, the basic idea still holds. You need to use judgment, as being lenient is not the same as being permissive. Example. In APIs, say your service expects a timestamp. If it receives a timestamp without a time zone, instead of rejecting, maybe you assume UTC or try to parse it anyway, being liberal in acceptance. But when your service returns data, you always include the time zone to ensure the output is conservative and precise. How to Judge Better 19. Sturgeon’s Law 90% of everything is crap. Most things we make will go unused, and most of the code we write is not good. Most projects we start do not deliver the value that we thought they would. This is not a bad thing per se. This is how things are when we are trying to create something new. If we pretend everything is great, we will treat every project the same, which will make things too complicated. The projects that really matter are the ones, like 10% of them. Finding these projects and getting rid of all the others is what really takes skill. Example. WordPress has roughly 57,000 plugins in its directory. Over 34,000 haven’t been updated in the past 2 years, and nearly 19% have zero active installs. A small number of well-maintained plugins powers 40%+ of the public web. That distribution is Sturgeon’s Law in one screenshot. 20. Cunningham’s Law The fastest way to get the right answer online is to post the wrong one. When you ask a question on some online forum, you usually get no response. If you post something that is clearly incorrect, people will jump in to correct you. They might just walk by if they see a question, and then cannot help themselves when they see something that is wrong. You can actually use this to your advantage. If you are having trouble with something, do not ask how you should do it. Instead, propose a solution you know is not very good, or share a draft, and then see what happens. The right answer might come to you without you even asking for it. Note that this trick only works when the people around you know what they are talking about. If you are in a group where everyone’s just as confused as you are, then a wrong answer can actually cause more harm than good. In that case, the wrong answer can just become information that people start to believe. Example. The whole bet of wikis, and later Wikipedia, runs on this insight. People correct errors faster than they write articles from scratch. The bet paid off on a civilization-scale. Conclusion In this article, I shared some of the most impactful laws I saw in my career. You do not have to memorize all of them. The top five or six laws will help you solve most of your issues. The rest are there for when a new problem arises. What is more important is knowing when a law applies and when it does not. These twenty laws often conflict with each other. Knuth says do not optimize early. Amdahl says find and fix the part of your project that is slowing everything down. Both are correct at times. The key is to know which one to use now. Also, this list is my list. Your list will be different. The laws that have caused you problems will be more important to you than the ones that have not. Over time, you will add your laws. Write them down when you notice them. One line per project, incident, or rewrite. Which law helped you? Which law gave you advice? What changed? Your personal list will be more helpful to you than any list I can give you. Frameworks, platforms, and deployment models have changed since Brooks wrote his book in 1975. These laws have not changed. They describe the one thing that has not changed: humans building things together under constraints they do not yet fully understand. That is why they are worth learning before the project, not after it causes problems.

By Milan Milanovic

CORE

The New Senior Developer Job Description: Half Engineer, Half AI Systems Architect

She had everything on the list. Eight years of experience. Strong systems design. Distributed architecture under her belt. The panel interview went well — one of the hiring managers later described it as the best technical conversation they'd had with a candidate all quarter. The team passed on her. Two weeks later, during a casual conversation with that hiring manager, the reason came out. It wasn't her architectural skills or her communication. It was a question someone had slipped in near the end: "Walk us through how you'd set up an AI-assisted code review pipeline for a team that ships twelve microservices." She described doing it manually. The other finalist described standing up an orchestration layer with context-aware models, configuring fallback thresholds, and building observable feedback loops that trained the team's prompt library over time. Same job title. Completely different mental model of what the job now involves. That story isn't unique. It captures something that's been happening gradually over the past eighteen months and then very suddenly in the last six: the senior developer role has quietly split into two jobs. One of them is the job we all trained for. The other is the job that a meaningful portion of your working week now actually requires. And the gap between developers who've accepted that and developers who haven't is becoming very hard to explain away in performance conversations. The Split That Happened Without a Memo Let's be specific about what the "AI Systems Architect" half of the role actually means, because people either over-mystify it or undersell it. It doesn't mean you become a data scientist. It doesn't mean you're fine-tuning models or writing PyTorch. Those are real jobs — they're just different jobs. What it means is something more operational and less glamorous: you are now responsible for designing, maintaining, and improving the systems of AI assistance that your team works inside of, not just the code that the team produces. That sounds abstract until you break it into daily decisions. Which tasks should be fully AI-generated versus AI-assisted versus AI-reviewed only? Where are your model's blind spots for your specific codebase, and how do you account for them in code review? When a junior developer on your team gets a plausible-but-wrong architectural suggestion from an AI assistant, what's the escalation path? How do you measure the quality of your team's prompting over time? These aren't rhetorical questions — they're operational ones that live teams are answering right now, often badly, because no one assigned anyone to own them. Senior developers are getting assigned to own them. Not officially. Not with updated job descriptions. Just through the ordinary mechanism of "this problem needs solving, and you're the most experienced technical person in the room." What "AI Systems Architect" Actually Means Day to Day The phrase sounds bigger than the practice. What it actually breaks down to is four interconnected responsibilities that are now landing on senior developers, whether they want them or not. First: workflow design. Someone has to decide which parts of the development cycle use AI assistance, at what level of autonomy, and with what human checkpoints. At most companies, this currently happens by accident — everyone develops their own habits, and nobody compares notes. The developers who are stepping into the architect half of the role are the ones making that deliberate, rather than emergent. Second: model selection and configuration. Not fine-tuning, but product-level decisions: which models for which tasks, what context window strategy, how to handle codebases that exceed context limits, what fallback behavior looks like. These are practical engineering decisions that live in the space between "developer tool choice" and "infrastructure decision." They belong to senior engineers. Third: quality governance. AI-generated code introduces a new failure mode: plausible-looking outputs that are subtly wrong. The patterns of wrongness are specific and learnable. Senior developers who have mapped the failure modes of their AI tooling — the kinds of edge cases it consistently misses, the naming convention assumptions it gets backward, the security patterns it handles confidently and incorrectly — are providing a form of institutional knowledge that is genuinely hard to replace. Fourth: team prompting culture. This is the one nobody talks about at conferences yet, but engineering managers across the industry have been mentioning it consistently over the past six months: the quality variance in how different team members prompt their AI tools is enormous, and it compounds. Senior developers who build and maintain shared prompt libraries, who do prompt review the way they do code review, who can diagnose why a colleague got a bad output — those developers are operating as a force multiplier for the entire team, not just themselves. The Job Description Before and After: A Concrete Comparison This is worth making explicit. Analysis of actual senior engineer job postings — anonymized, from companies between 80 and 1,200 employees — shows a clear shift when comparing what the role requirements looked like in early 2023 versus what's being written now. The change is real and measurable. The pattern across all of it: the what of the role hasn't changed so much as the how and the governance around it. Senior developers are still responsible for the same categories of work. They're now also responsible for the design of the AI-assisted systems that help a team do that work, and for the failure modes those systems introduce. The New Core Competency Stack Here's what the competency model looks like in practice when you lay it out. The traditional side should feel familiar. The AI architecture side probably contains a few items you haven't formally owned yet — but if you've been doing this job for more than two years and paying attention, you've been building these skills without realizing it. The Salary Premium Is Already Real Compensation data lags reality by about eighteen months, so take specific numbers here with appropriate skepticism. What industry reporting suggests is that a clear pattern is emerging: developers who can demonstrably operate in both halves of the new role — not just use AI tools personally, but architect AI-assisted workflows for a team — are commanding a premium that's running somewhere between 18% and 31% above their single-track counterparts at the same years-of-experience mark. That range is wide. The premium is highest in companies that have recently invested in AI transformation initiatives and learned, the hard way, that "everyone uses Copilot" is not the same as "we have a coherent AI engineering strategy." Those companies are specifically recruiting for systems architect skills because they've already paid for the gap. How to Build the Second Half of the Job Nobody teaches this in a course yet. There are some good books and a growing number of blog posts, but the skills are mostly developed through deliberate practice and iteration. Based on teams that have successfully made this transition, here's what works. The starting point is mapping your team's current AI-assisted work honestly. Not aspirationally — honestly. Which tasks are you and your team currently doing with AI assistance? Where does the output go without sufficient review? What are the categories of error you've caught, and what categories might you be missing? This audit, done once and updated quarterly, is the foundation of a governance practice. From there, the most leveraged thing most senior developers can do is build a shared prompt library for their most common task types. Not a personal one — a shared one, with a versioning and review practice attached. The discipline of reviewing a colleague's prompt and explaining why it produced a wrong output is one of the fastest ways to build the mental model you need for the governance half of the role.

By Dinesh Elumalai

CORE

Amazon Quick: AWS's Agentic Workspace, Explained for Engineers

AWS has been building agentic infrastructure for some time now — Bedrock, AgentCore, Strands — mostly aimed at engineers who want to build their own agent systems from scratch. Amazon Quick is a different layer of the same bet: a ready-to-use agentic workspace that targets teams directly, without requiring custom orchestration code. This article walks through what Quick is, how its components fit together technically, how the MCP integration model works with real code, and where it sits relative to the rest of AWS's agent stack. What Amazon Quick Is Amazon Quick is an AI assistant for work that connects to your existing tools — Slack, Microsoft Teams, Outlook, CRMs, databases, and local files — and gives a unified layer for querying, automating, and acting across them. It launched in preview at AWS's "What's Next with AWS" event on April 28, 2026. The product is aimed at teams, not just individual users. One person can build a custom agent scoped to a specific dataset or workflow, and the whole team benefits from it. Responses from Quick agents are grounded in your actual business data, not the underlying model's training distribution. Under the hood, Quick is built on Amazon Bedrock AgentCore and uses the Model Context Protocol (MCP) as its standard for connecting to external tools. It runs on AWS IAM and VPC, which means it inherits the same security and compliance posture as the rest of your AWS workloads. Components Quick bundles five distinct capabilities. It helps to understand each one separately before thinking about how they compose. ComponentWhat it doesSpacesCollaborative workspaces where teams pool files, dashboards, and data sources. Agents in a Space are grounded in that Space's data.AgentsCustom, domain-scoped agents built on your team's specific data. One person builds, everyone uses.ResearchMulti-source synthesis across internal data, the public web, and third-party datasets. Produces structured reports.Visualize (Quick Sight)Integrated BI layer. Conversational access to dashboards, charts, and forecasting — no separate BI tool required.Automate (Quick Flows)Workflow automation from simple daily tasks to complex multi-step processes with cross-app action execution. Each component is available through the web app, mobile, and a native desktop app (currently in preview for macOS and Windows) that can read local files and calendar context without requiring browser access. Where Quick Sits in the AWS Agent Stack AWS is building in two directions at once. AgentCore is the infrastructure layer for engineers who want to compose their own agent systems — runtime, memory, gateway, observability — with any model and any framework. Quick is the product layer on top: opinionated, team-facing, and deployable without writing orchestration code. The practical implication: if you're an engineer building internal tools or automation pipelines, you'll likely interact with both layers. AgentCore for the infrastructure wiring; Quick as a surface where non-technical teammates interact with the agents you build. The Integration Architecture The core question for any engineer evaluating Quick is: how does it actually connect to external systems, and what does the request path look like? Quick uses MCP (Model Context Protocol) as its primary integration standard. This is significant because MCP is an open protocol — it means Quick agents are not locked into AWS-specific connectors, and any MCP-compatible server can be registered as a tool source. High-Level Request Flow The sequence below shows the full lifecycle of a single agent-triggered tool call — from the moment Quick receives a prompt through to the response returning from a downstream API. Quick acts as the MCP client. Your MCP server exposes tools via listTools and callTool. Quick discovers them at registration time and makes them available to any agent or automation in the workspace. Authentication flows through OAuth 2.0, with support for Dynamic Client Registration (DCR) so Quick can register itself automatically without manual credential setup. Building an MCP Server for Quick Here is a minimal Python MCP server using the mcp SDK that exposes two tools Quick can invoke — get_ticket and list_open_tickets. This pattern works whether you host the server yourself or run it on AgentCore Runtime. Install Dependencies Python pip install mcp[server] httpx uvicorn Server Implementation Python # server.py from mcp.server import Server from mcp.server.sse import SseServerTransport from mcp.types import Tool, TextContent import httpx import json from starlette.applications import Starlette from starlette.routing import Route app = Server("jira-quick-integration") JIRA_BASE_URL = "https://yourorg.atlassian.net" JIRA_TOKEN = "Bearer <your-token>" # in production, load from AWS Secrets Manager @app.list_tools() async def list_tools() -> list[Tool]: return [ Tool( name="get_ticket", description="Retrieve details for a single Jira ticket by issue key.", inputSchema={ "type": "object", "properties": { "issue_key": { "type": "string", "description": "The Jira issue key, e.g. ENG-1234" } }, "required": ["issue_key"] } ), Tool( name="list_open_tickets", description="List open Jira tickets assigned to a given user.", inputSchema={ "type": "object", "properties": { "assignee": { "type": "string", "description": "The Jira username or email of the assignee" } }, "required": ["assignee"] } ) ] @app.call_tool() async def call_tool(name: str, arguments: dict) -> list[TextContent]: headers = {"Authorization": JIRA_TOKEN, "Content-Type": "application/json"} async with httpx.AsyncClient() as client: if name == "get_ticket": key = arguments["issue_key"] resp = await client.get( f"{JIRA_BASE_URL}/rest/api/3/issue/{key}", headers=headers ) resp.raise_for_status() data = resp.json() summary = data["fields"]["summary"] status = data["fields"]["status"]["name"] return [TextContent(type="text", text=f"{key}: {summary} [{status}]")] elif name == "list_open_tickets": assignee = arguments["assignee"] jql = f"assignee={assignee} AND status != Done ORDER BY updated DESC" resp = await client.get( f"{JIRA_BASE_URL}/rest/api/3/search", headers=headers, params={"jql": jql, "maxResults": 20} ) resp.raise_for_status() issues = resp.json().get("issues", []) results = [ f"{i['key']}: {i['fields']['summary']}" for i in issues ] return [TextContent(type="text", text="\n".join(results) or "No open tickets found.")] raise ValueError(f"Unknown tool: {name}") # Wire up SSE transport for Quick compatibility sse = SseServerTransport("/messages/") async def handle_sse(request): async with sse.connect_sse( request.scope, request.receive, request._send ) as streams: await app.run(streams[0], streams[1], app.create_initialization_options()) starlette_app = Starlette( routes=[Route("/sse", endpoint=handle_sse)] ) if __name__ == "__main__": import uvicorn uvicorn.run(starlette_app, host="0.0.0.0", port=8080) A few design constraints to be aware of when building for Quick: Each MCP tool call has a 300-second hard timeout. Operations that exceed this fail with HTTP 424. Keep individual tool calls narrow and fast.The tool list is treated as static after registration. If you add or remove tools on the server, the Quick admin must re-establish the connection to pick up changes.Quick supports both Server-Sent Events (SSE) and streamable HTTP as transports. Streamable HTTP is preferred for new implementations. Registering the MCP Server in Quick Once your server is running and publicly reachable over HTTPS, registration in Quick takes the following path: Shell Quick Console → Integrations → Add Integration → MCP Fields: Server URL: https://your-mcp-server.example.com/sse Auth type: OAuth 2.0 (or Service, or None) Client ID: <from your identity provider> Authorization URL: https://auth.example.com/oauth/authorize Token URL: https://auth.example.com/oauth/token If your identity provider supports OAuth Dynamic Client Registration, Quick will auto-register and you skip the manual client ID step entirely. Quick sends an initial unauthenticated request to the MCP server; if it receives a 401 with a WWW-Authenticate header containing a resource_metadata URL, it fetches the metadata document and proceeds with DCR automatically. Once registered, Quick calls listTools at startup and exposes every discovered tool to agents and automations in the workspace. The AgentCore Gateway Option For teams that don't want to write and operate an MCP server from scratch, Amazon Bedrock AgentCore Gateway provides a managed alternative. You point Gateway at a Lambda function or an OpenAPI spec, and it handles the MCP wrapping, auth, logging, and semantic tool discovery automatically. If you use it, Quick never calls your internal APIs directly — everything flows through Gateway's auth and routing layer, as shown in the sequence diagram above. The semantic search capability is worth noting specifically. When an agent has access to dozens or hundreds of tools, passing the full tool list on every turn wastes context and causes the model to pick the wrong tool. Gateway's built-in x_amz_bedrock_agentcore_search tool lets Quick find the right tool by semantic similarity rather than scanning the entire registry each turn. Practical Considerations A few things worth keeping in mind before integrating: Tool scope matters. When agents are given too many tools simultaneously, selection accuracy degrades — the model reasons over too many options per turn and picks incorrectly more often. Keeping each agent or MCP server to a focused set of 3–5 tools produces better results than exposing everything through one endpoint. This is a known pattern in multi-agent architectures and applies equally to Quick agents. The 300-second timeout is real. Design each tool call to complete a single, bounded operation. Avoid chaining multiple downstream API calls inside a single tool invocation. If you need a multi-step workflow, model it as separate tools and let the agent orchestrate the sequence. Local context on the desktop app. The desktop app reads local files and calendar events directly, without upload. For engineers who work primarily in terminals and local editors, this is a meaningful integration point — meeting context, local documentation, and recent file changes are all available to the assistant without any configuration. MCP interoperability. Because Quick uses MCP as the standard, the same MCP server you build for Quick can also be consumed by Claude Code, Amazon Q Developer, and other MCP-compatible clients. The integration contract is portable. References Amazon Quick — Product overview and featuresIntegrate external tools with Amazon Quick Agents using MCP (AWS ML Blog, Feb 2026)MCP integration — Amazon Quick User GuideAmazon Bedrock AgentCore — Overview and documentationIntroducing Amazon Bedrock AgentCore Gateway (AWS ML Blog)Top announcements of the What's Next with AWS, 2026 (AWS News Blog, Apr 2026)

By Jubin Soni, FBCS

CORE

Why Your QA Engineer Should Be the Most Stubborn Person on the Team

There is a common stereotype that software testing is just a dull exercise in checking what should already work. In reality, the cost of a missed bug in a serious product is far higher than a minor visual glitch or a button shifting out of place. It can lead to failures in critical workflows, data loss, service outages, and major financial damage for the business. People often say that QA is just there to check developers’ work. That is a superficial view. The role of QA is not simply to confirm that the code works, but to try to uncover every scenario in which it can fail. QA engineers have to think differently from the people who built the system. They need to think like real users, including those who will inevitably follow unexpected paths or use the product in ways no one originally planned. The myth that QA is routine work has lasted for years. Yes, regression testing is part of the job. But if you force an engineer into rigid checklist-driven testing, you will miss bugs. Strong QA does not follow a script mechanically. It explores the product, looks for edge cases, and finds new ways it can fail that no one considered during planning. QA Tasks and Approaches In most organizations, there is a distinction between QA (Quality Assurance) and QC (Quality Control), and both need to be considered together. It is not enough to verify the quality of the product being shipped. Teams also need to analyze the development and testing process itself, identify weaknesses, and look for ways to improve the product over time. Tasks break down into two major streams: Bugfix and regression. It is not enough to confirm in a ticketing system that a developer has fixed a bug. The goal is to make sure it does not come back a month later. That is why any critical fix should be covered by a regression test. This becomes a safeguard against product degradation.Feature testing. The work starts long before the build. Documentation and requirements are studied not just to understand how the feature is supposed to work, but to anticipate how it might break neighboring components. Depending on complexity, the right testing approach is chosen: from simple checklists to full test cases and automation where it makes sense. Good teams aim for high automation coverage, but they also stay realistic. Test coverage varies across modules: legacy code and experimental features require different approaches. In some areas, developer-owned unit tests may be enough. In others, QA needs to validate functionality through more complex integration scenarios. What Good QA Looks Like and How to Build It Good QA means stable tests that provide sufficiently complete coverage of a feature or product. It is also critical that QA and development teams actively collaborate. For example, when handing a product off to development, QA provides reports on what was tested and how, what risks remain, and what can be improved in coverage and overall product quality. It is also good practice to write tests alongside the code. For example, developers may write TAP-based functional tests for PostgreSQL or unit tests for Go code. At the same time, QA engineers prepare to test the finished product: they analyze documentation, talk to business analysts and technical product managers, and clarify expected behavior. Based on that information, QA specialists create test cases and checklists, automate them when appropriate, and add them to CI. It also helps to have a DevOps engineer who can take infrastructure work off QA’s plate. If there is no DevOps engineer, that responsibility often falls to the QA engineer as well. Manual vs. Automated Testing Automation is invaluable for repetitive tasks where manual re-checking leads to attention fatigue. In these cases, automated tests improve test quality and reduce errors caused by human fatigue or inattention. Exploratory manual testing complements automation. It helps uncover product behavior, edge cases, and unusual scenarios that automated tests may not cover. When building automated tests, test design techniques can be applied deliberately. For manual testing, the engineer's own experience matters most: you need to look at the product broadly and systematically, analyze what affects its behavior, and try to break it from angles nobody thought to consider. For example, one of my colleagues once tried to bring up a cluster in a nonstandard way by skipping one step, and the system failed. A user may follow their own path, and the developer simply may not have accounted for it. From an exploratory testing perspective, it is valuable to have custom tooling for fuzzing and generating random input combinations. For example, an in-house combinator can shuffle files with SQL queries to produce unexpected execution paths and input combinations. ASan is commonly used for detecting memory errors, while Valgrind is widely used for memory debugging, leak detection, and profiling. For web application testing, Selenium Grid supports cross-browser execution at scale. For automated tests, Pytest can serve as the test execution framework, while Allure Report generates CI-integrated reports from the test results. TestRail handles test case management and coverage tracking at the team level. Testing Against Open Source and Third-Party Components QA becomes more complex when a product is built on top of an open-source core or depends heavily on external components. In that case, the team is not testing only its own code. It is also testing how upstream changes, extensions, integrations, and internal modifications behave together. For example, a product based on PostgreSQL may include its own commercial logic while still pulling changes from the upstream project. That changes the QA process. If a bug is found in upstream code, quietly patching it only inside the commercial product is not always the right approach. In many cases, the issue should be reported back to the community, especially when the defect affects the original project and not only the commercial layer. The same logic applies to extensions and external components. Before adding a third-party component, QA should treat it as untrusted until it has proven stable in the target environment. That means exploratory testing, compatibility checks, regression coverage, review with SCA tools, and verification under realistic workloads. If bugs are found, they should be reported to the maintainers, and the component should be added only after the fix is available and the behavior is stable enough for production use. Upstream bugs are often harder to investigate than internal code defects. With internal development, the team usually has full visibility into every commit. If something breaks, tools such as git bisect can quickly point to the change that introduced the regression. With upstream code, changes often arrive in larger batches. When something fails after a merge, QA and developers may need to analyze a much larger set of external changes before they can isolate the root cause. Communication works differently, too. With internal code, the developer who owns the change may be one message away. In an open-source project or a third-party component, the feedback loop is longer. Reports need to be reproducible, technically precise, and useful to maintainers. This is where strong QA becomes more than bug reporting. It becomes engineering communication. AI and Vibe Coding: Hype or Genuine Value? Vibe coding is having a moment: you describe a task to an AI, and it writes the code for you. In QA, this can work, but with some important caveats. The most practical current use cases are generating sets of basic test cases (for example, pairwise combinations) and writing small helper scripts. This helps remove the “blank page” problem and saves engineers from some routine work. But it is important to understand the boundary. AI is like a junior engineer who never gets tired but often hallucinates. It cannot reliably anticipate deep architectural specifics of a product or subtle edge cases. At some point, a human must step in. Otherwise, the vibe can end with a critical bug in production. How to Become an Effective QA Engineer and Grow in the Profession At university, testing is typically covered as part of a programming curriculum. That is enough to learn the basics and understand whether the field interests you. After that, you can continue learning independently or take dedicated QA courses. For many serious QA roles, strong Python skills are one of the most useful foundations for automation, along with a solid understanding of Linux. This is especially true in systems development: you simply cannot test a complex product effectively if the terminal intimidates you or you do not understand how the environment works. But hard skills alone are only tools. The key soft skills for QA are systems thinking and the ability to make a clear technical argument. Finding a bug is not enough. You also need to explain why it should be fixed and make the case to the developer. This is not about avoiding conflict. It is about clear, constructive communication in a shared technical language. Where Can QA Engineers Grow? Technical track: QA Architect or Lead SDET. This path means continuous growth in coding, test architecture, and CI/CD. You become the expert who can build a quality engineering infrastructure from the ground up. Management track: Team Lead or Head of QA. Here, the focus shifts to processes, hiring, and mentoring. The idea that managers do not need deep technical knowledge is dangerous. To lead a team of engineers, you need to understand their pain points and the complexity of their work. And yes, some QA engineers eventually move into development or analysis. Testing experience becomes a real advantage there: engineers with a testing background tend to write cleaner code because they can see in advance where it is likely to break. Useful Resources A short list of resources to help you get started in QA and stay current with what matters: How Google Tests Software by James Whittaker, Jason Arbon, and Jeff Carollo: A useful book on how testing can be organized at scale, with strong ideas around risk, ownership, automation, and engineering culture. Lessons Learned in Software Testing by Cem Kaner, James Bach, and Bret Pettichord: A foundational text covering practical testing wisdom that holds up regardless of stack or methodology.ISTQB: Not every strong tester needs certification, but ISTQB is very useful for learning shared testing terminology, test design basics, and the difference between QA, QC, verification, validation, test levels, and test types. ISTQB materials are especially helpful for beginners who need structure. Ministry of Testing: One of the best-known global testing communities, with articles, discussions, courses, events, and practical material on quality engineering, exploratory testing, automation, and modern QA careers. Test Automation University: A practical resource for learning automation, API testing, UI testing, CI integration, and related engineering skills. Final Thoughts QA requires a lot of patience and focus, but knowing when to switch context matters just as much. Sometimes that is exactly what helps you solve a problem after spending a week chasing an intermittent failure with no clear result. A solution can almost always be found. Do not forget physical activity or anything else that lets your brain step away from the task for a while. It gives you the energy to come back and look at the problem from a different angle.

By Alex Vakulov

CORE

You Learned AI. So Why Are You Still Not Getting Hired?

You learned prompt engineering. You built a chatbot. You finished a course. You added “GenAI” to your LinkedIn headline. And still, the interviews go nowhere. That does not always mean the AI job market is fake. More often, it means your signal is weak. Most candidates are showing that they can use AI tools. Employers are trying to hire people who can make AI useful inside a messy business, with real customers, bad data, edge cases, budgets, and risk. That is a very different standard. I have spent more than two decades working on systems where loose thinking becomes expensive very quickly. In enterprise platforms, weak architecture creates operational pain. In AI systems, weak judgment creates confident mistakes. That is why companies are not just hiring “people who know AI.” They are hiring people who can make AI dependable, measurable, and worth the cost. AI and big data top the list of fastest-growing skills. Source: World Economic Forum, “The Future of Jobs Report 2025.” The opportunity is real. But the winning profile is narrower than most people think. The Real Gap Is Not Learning. It Is Proof. A lot of job seekers think the market wants more certifications. It usually does not. What employers actually want is proof that you can take a vague business problem and turn it into a reliable AI workflow. That might mean improving support response quality. It might mean extracting fields from invoices. It might mean enriching product data. It might mean helping internal teams search better across thousands of documents. In every case, the question is the same: Can you turn AI from a cool demo into useful work? That is the hiring filter. Prompting Is Not the Skill. Precision Is. People still talk about prompt engineering as if it is a magic trick. It is not. The real skill is writing clear instructions for messy real-world work. Prompt engineering is the process of writing effective instructions for a model. Source: OpenAI, “Prompt engineering.” That sounds basic, but most candidates still operate at the level of vague intent. For example, “build a support bot” is not a serious instruction. A stronger version sounds like this: Handle password resets, order status checks, and return requests. Escalate angry customers. Do not invent policy. Log the reason for every escalation. Use only approved support content. That is not fancy. It is clear. And clear is valuable. This is one reason many smart people struggle to land AI jobs. They have learned how to ask AI interesting questions. They have not yet learned how to define work so precisely that a machine can do it safely and repeatably. That skill matters in AI engineering, AI product management, AI operations, AI consulting, and AI strategy. If you can define success clearly, you immediately become more hireable. Pretty Output Is Not the Same as Correct Work AI has a dangerous habit. It often sounds right before it is right. That is why evaluation matters so much. Evaluations (evals) are a way to test your AI system despite this variability. Source: OpenAI, “Evaluation best practices.” In plain English, this means one polished answer is not proof of quality. A summary can read well and still miss the legal risk. An invoice extractor can look accurate and still miss tax values. A product recommender can sound helpful and still suggest the wrong item. This is where many candidates lose credibility. They show outputs. They do not show checks. A stronger portfolio piece does not stop at “here is my AI app.” It says: Here is the task. Here is how I measured success. Here is where the system failed. Here is what I changed. Here is what still needs human review. That instantly feels more senior. If you want to stand out in AI hiring, start reviewing AI output as if your name is on it. Because in production, it will be. A Good AI Builder Can Break Work into Steps Another missing skill is decomposition. Can you take a big, fuzzy workflow and split it into smaller steps that AI can handle well? That is what real projects need. Take product catalog enrichment. A weak candidate says, “I built a product content generator.” A strong candidate says: First, classify the product. Then pull trusted attributes. Then draft the copy. Then check factual consistency. Then route uncertain cases to human review. That is a very different level of thinking. The same is true in support, search, compliance, reporting, and internal tooling. Employers are not paying for random prompt collections. They are paying for people who can structure work. And this is good news for job seekers. Because decomposition is not reserved for machine learning researchers. It is also a skill used by architects, product managers, QA leads, technical writers, analysts, and operations people. Many people are closer to AI work than they think. Trust Is Part of the Job Now There is another reason companies hesitate. Some AI tasks are easy to reverse. Others are not. A bad draft can be edited. A bad wire transfer cannot. A weak product description may annoy a customer. A wrong financial or medical recommendation can do real damage. That is why responsible deployment is now part of the skill set. Help manage the many risks of AI and promote trustworthy and responsible development and use of AI systems Source: NIST, “Artificial Intelligence Risk Management Framework (AI RMF 1.0).” The candidates who look strong in this market are the ones who naturally ask questions like: What is the cost of error? How often can this fail? Can we verify the result? Where should a human approve the outcome? What should the model never be allowed to do alone? This is not just a compliance issue. It is a product skill. It is an engineering skill. It is a leadership skill. And it is one of the clearest signals that someone understands production AI rather than just experimental AI. If the Economics Fail, the Idea Fails One more thing separates a hireable candidate from an enthusiastic learner. Business math. Not every workflow deserves the biggest model. Not every AI feature deserves to exist. Model choice and usage patterns directly affect cost, and official API pricing pages make that tradeoff visible across tiers and token categories. That means a serious AI professional should be able to say: This task is simple, so use a cheaper model. This task is high-stakes, so pay for a stronger one. This workflow runs at scale, so measure the cost before rollout. This use case saves time, but not enough money to justify production. That kind of thinking changes how hiring managers see you. Now you are not just someone who can build with AI. You are someone who can make a sound decision with AI. What Hiring Managers Actually Want to See If you have learned AI but have not landed a job, stop asking, “What course should I take next?” Start asking, “What evidence would make an employer trust me?” A strong answer usually includes one real workflow and five kinds of proof: A clear business problem. A precise task definition. A simple evaluation method. A sensible review process for risky cases. A rough explanation of cost and value. That is enough to make a portfolio piece feel real. Not flashy. Real. And real wins. The AI job market is not only looking for people who can talk to models. It is looking for people who can think clearly, reduce ambiguity, catch mistakes early, and connect technical work to business outcomes. That is why many candidates feel stuck. They are training for “using AI.” Employers are hiring for judgment. Once you understand that, the path changes. Build fewer demos. Show more decision-making. That is how you stop looking like someone chasing AI jobs, and start looking like someone ready to do one. For more practical insights on AI careers, software architecture, and building production-ready systems, connect with Faisal Feroz on LinkedIn and read more on his blog.

By Faisal Feroz

Stop Using the ATM-Didn’t-Kill-Jobs Story to Reassure Developers About AI

The ATM Didn't Kill Bank Tellers' Jobs There's a story economists love to tell about ATMs and bank tellers. You've probably heard it. When ATMs were introduced in the 1970s, everyone predicted they would eliminate teller jobs. They didn't. By the 2000s, there were actually more tellers than before the ATM existed. The story became a load-bearing parable for anyone who wanted to argue that technology doesn't kill jobs, cited by economists like Daron Acemoglu and David Autor, by tech executives like Eric Schmidt, and more recently by politicians reaching for reassuring historical analogies when asked about AI. David Oks recently published a sharp piece that complicates this parable. His conclusion: ATMs didn't kill bank tellers, but the iPhone did. Teller employment entered prolonged decline in the 2010s because mobile banking made the branch itself irrelevant. Once customers stopped coming in, the institutional context that gave the teller role its value simply ceased to exist. Oks draws a clean distinction from this: it is paradigm replacement, not task automation, that actually displaces workers. The ATM tried to fit capital into a labor-shaped hole. The iPhone changed the shape of the hole entirely. It's a compelling argument. And for software developers watching coding agents write functions, generate tests, and draft pull requests, it offers a certain comfort: you're not being replaced, just assisted. The real disruption, if it comes, will look completely different. I think that comfort is premature. Oks argument in itself is not wrong, but it is incomplete and the part it leave out matters a lot. The iPhone Argument Has a Gap Oks is right that the iPhone, not the ATM, killed the bank teller job. But his broader thesis that paradigm replacement is the mechanism of displacement, not task automation, isn’t historically accurate. Manual weavers weren't displaced by a paradigm shift. The power loom did the same thing they did, faster and cheaper, and they became redundant. The technology simply fit itself into a labor-shaped hole and the labor disappeared. Scribes, typing pools, telephone switchboard operators, elevator operators, etc. none of these occupations required a paradigm shift to be eliminated. In each case, task automation was sufficient. The job existed because a specific set of tasks had economic value. When those tasks could be performed more cheaply by a machine, the role contracted and eventually disappeared. What Oks has actually identified is a specific condition under which task automation fails to eliminate a role. That condition has two components: the automated task must be embedded in a broader human interaction that has independent value, and the cost reduction from automation must enable expansion of the overall activity. Both conditions held for bank tellers. The teller's residual tasks such as relationship management, complex service interactions and cross-selling financial products were genuinely valuable and hard to automate. And cheaper branches meant more branches, which meant more tellers. Demand expanded to absorb the efficiency gains. When those conditions don't hold, task automation alone is sufficient to eliminate a profession. The question for software development is which set of conditions applies. Where Software Development Sits Right Now At this moment, the conditions for software developers look more like bank tellers than manual weavers. But only just, and the gap is closing. Current coding agents are genuinely capable at specific set of tasks: writing boilerplate, generating tests, documenting code, drafting pull requests, handling well-specified implementation problems. These are real and substantial parts of software work. But they remain components of something larger. The tasks that define senior software development include understanding what problem is actually worth solving, navigating the organizational and technical tradeoffs in an architectural decision, holding the accumulated context of a system's history and making decisions about about second-order consequences. Current AI tools handle poorly these activities. This is why the most experienced engineers are the heaviest users of these tools. Staff+ engineers, according to recent survey data from The Pragmatic Engineer, use AI agents at higher rates than any other level. They're not threatened by the tools; they're amplified by them. Their judgment is the thing the tool can't replace, and the tool makes their judgment more productive. In Oks' terms: the automated task is still a component of a richer service relationship, and the remaining human tasks still have genuine value that can’t be easily automated. There's a second reason the current moment feels stable, and it's more structural. Almost everything about how software development is organized today was designed for humans working at human speed. The way we gather and surface production issues. The way we structure pull request reviews. The way we run sprint planning, write tickets, conduct incident postmortems. These workflows were built around human cognitive limitations and human communication patterns. Dropping a coding agent into them is like replacing a factory's steam engine with a single large electric motor and keeping the drive shaft: you get some efficiency gains, but you're not anywhere near the real potential of the technology. This means we're still in early transition. The friction isn't just technical capability; it's organizational inertia. And as long as workflows are designed for humans, the human remains load-bearing in ways that aren't purely about cognitive ability. Why the Trajectory Matters More Than the Current State Here's where the bank teller analogy breaks down in a way that should concern developers. ATMs had a hard ceiling. They could handle cash withdrawals, deposits, and balance checks. They couldn't have a conversation, make a judgment call, or handle anything outside their programmed transaction set. The teller's residual role was structurally protected by what the machine couldn't do. AI coding tools don't have an obvious ceiling. The tasks they handled poorly six months ago they handle better today. The tasks that seem safely human (system design, architectural judgment, understanding business context) are precisely what the next generation of tools is being explicitly built to address. The boundary between "what AI can do" and "what requires human judgment" is moving. The debugging agent is a useful case study here, because it illustrates something more significant than incremental improvement. Traditional observability and debugging workflows were designed around humans: a log file, a Slack thread, an on-call engineer piecing together a causal chain under pressure at 2am. The first generation of AI debugging tools tried to fit into that workflow: read the logs, suggest a fix, slot into the existing process. What's emerging now is different. Debugging agents designed from the ground up to collect data in machine-readable formats, pre-correlate signals before a human ever sees them, deduplicate noise, and surface a ranked hypothesis rather than a raw stream of events. This is not task automation slotted into an existing workflow. This is the workflow being redesigned around what machines need rather than what humans can do. It's the first concrete step toward something that would have sounded like science fiction three years ago: systems that close the loop on their own failures, with humans in an oversight role rather than a diagnostic one. The human's job shifts from "figure out what broke and why" to "decide which failure modes matter and whether the agent's response is appropriate." That's a real and valuable role. It's also a much smaller one. This pattern will repeat. PR reviews are being rethought not as human checkpoints with AI assistance, but as automated verification layers with human escalation paths. Architectural decision-making is starting to be approached with tools that can compare the tradeoffs made across a codebase. The demand elasticity argument (there's always more software to build, so developers will always be needed) deserves a direct response, because it's the most common counterargument and the most misleading. It's true that the demand for software is enormous and largely unsatisfied. New markets are opening: small businesses that couldn't afford custom software at previous price points now can. There is genuinely more to build than we're currently building. But new demand mostly benefits new categories of developers serving new markets: the solo developer building custom tools for local businesses, the non-specialist who can now direct agents without deep technical knowledge. It doesn't necessarily sustain employment at current levels in the enterprise and product development tiers where most professional developers currently work. When a team of five engineers with AI agents can do the work that previously required fifty, the question is whether organizations respond by building ten times as much software or by not rehiring the other forty-five. The historical pattern in most industries, including banking after the iPhone, is the latter, at least in the short to medium term. What This Means The structural conditions for developer employment look reasonably healthy for the next several years. The remaining human tasks are real and valuable. The workflows haven't been redesigned yet. The organizational inertia is substantial. But the trajectory is not ambiguous. The tasks that AI handles poorly today are the explicit targets of the tools being built right now. The workflows are starting to be redesigned from the ground up rather than retrofitted. The "richer service relationship" that protects the developer's role is narrowing as the agent takes on more of what previously required human judgment. The compression threat is more immediate and more certain than the paradigm-replacement scenario, but both are real. In the near term, the result will be the same output from smaller teams. In the longer term, "software development" as a distinct full-time profession organized around the ability to write and make decisions about code may dissolve into something else: a capability that technical knowledge workers exercise incidentally, the way managers today use Excel without being spreadsheet professionals. Oks ends his piece with a distinction that's worth noting: the ATM substituted tasks, the iPhone made them irrelevant. The people reassuring developers by citing the ATM story may not have noticed that we are already in the iPhone moment. Change in technology-driven labor markets follows a familiar pattern. It happens gradually, incremental improvements, partial automation, roles that adapt and absorb, and then all at once, when the paradigm finally shifts and the institutional context that made a role economically legible simply ceases to exist. We're in the gradual part. The "all at once" is not scheduled, but it's on the roadmap.

By Thomas Johnson

CORE

AI Didn't Replace Seniors; It Just Made Them the Bottleneck

Two to three years ago, the narrative surrounding AI in software engineering was quite simple: Inevitably, LLMs and AI tools would improve so much that they would replace most of the engineering workforce, including senior engineers. LLMs would write production-grade code, and organizations would need little more than a couple of product owners, a carefully crafted prompt, and a deploy button. Entire conferences were held around this concept, LinkedIn and Tech YouTube were drowning in "doomsday" posts. This prediction aged quite interestingly. What actually happened was both surprising and uncomfortable. Yes, AI tools can generate fairly good code at an extraordinarily fast pace, but the bottleneck in software delivery did not disappear. It shifted onto the shoulders of senior engineers, and the compounding effects of that shift are only now becoming visible. The Bottleneck Moved Here's the uncomfortable arithmetic of AI-assisted development in 2026. AI coding tools now write roughly 41% of all new commercial code, according to recent industry analyses. Feature velocity is at an all-time high, and yet experienced developers report spending more time debugging, more time in code review, and more time untangling architectural decisions that no human actually made. A 2025 study by METR found a striking disconnect: Developers using AI tools felt approximately 20% faster, but their measured task completion time was actually 19% slower on real-world codebases. The gap between perception and reality is nearly 40 percentage points. Two cognitive biases explain this: automation bias (where we overtrust automated output) and the effort heuristic (where less typing feels like less work). The code generation part was never really the hard part. Understanding what you have built, why it behaves the way it does, and what will break when you change it — that was always the bottleneck. AI made the fast part faster, but it also made the slow parts dramatically slower. Cognitive Debt: The Invoice Tech Companies Are Ramping Up There is a term gaining traction in engineering circles that captures this phenomenon precisely: cognitive debt. Unlike technical debt, which lives in the codebase and can be measured with linters and static analysis, cognitive debt lives in the minds of the developers working on the system. It is the growing gap between the amount of code that exists and the amount any human genuinely understands. Addy Osmani described this as comprehension debt: the hidden cost that does not show up in velocity metrics. The codebase looks clean, tests are green, and PR counts are up, but underneath those reassuring dashboards, parts of your system are running in production that no one on the team can explain. Either because that code was written by a non-human, or because the human that written it got laid off and replaced by an LLM. To be totally fair, this phenomenon existed even in the "pre-AI era." I have seen this firsthand in large enterprise environments — some microservices deployed to production where, due to team restructuring and layoffs, the people who developed the code are gone, and the people left maintaining it never built a mental model of how it works. The current tools are making this far worse. The organizational assumption that reviewed code is understood code no longer holds- engineers are approving code they did not fully understand, and that approval now carries implicit endorsement, thus quietly distributing liability. Margaret-Anne Storey, whose February 2026 research helped popularize the term, put it plainly: A program is not its source code. A program is a theory that lives in the minds of the developers. When AI generates the implementation and humans merely review it, that theory can fragment or disappear entirely. And when it does, even simple changes become dangerous. The Review Bottleneck Is a Senior Engineer Bottleneck So, who is left holding the system’s mental model together? The senior engineers. The ones who remember why that architectural decision was made under pressure eight months ago, who can look at a diff and immediately know which behaviors are load-bearing and which are cosmetic, who can tell the difference between a safe refactor and one that will quietly shift something users depend on. These engineers are now the scarce resource the entire organization depends on, and here is the irony: the same wave of AI adoption that increased the volume of code requiring review also triggered the layoffs that thinned the ranks of the people capable of reviewing it. Between 2023 and early 2026, the tech industry shed hundreds of thousands of jobs. Over 245,000 globally in 2025 alone, with 2026 on pace to exceed that figure. But the layoffs were not evenly distributed across seniority levels. Companies cut junior and mid-level roles aggressively, believing AI tools could absorb the work. The remaining senior engineers did not get a lighter workload. They got a heavier one, with fewer people to delegate to. This is what burnout looks like in 2026 — a slow erosion. Engineers who stop pushing back in design reviews because they do not have the energy, code reviews that become rubber stamps, and architectural choices made by default rather than deliberation. The people most likely to burn out are the people hardest to replace. And it gets worse.... The Broken Pipeline: Where Are Tomorrow’s Seniors Coming From? This brings us to the part of the story that keeps me up at night. If the current senior engineers are overwhelmed and burning out, or worse, laid off, the natural question is: who replaces them? The answer, increasingly, is nobody. Entry-level developer hiring has collapsed. Ravio’s 2025 Tech Job Market Report found that entry-level hiring dropped 73% year over year, while overall hiring dipped only 7%. This is a deliberate strategic decision playing out across the industry. In 2019, new graduates represented 32% of Big Tech hires. By 2026, that number has cratered to roughly 7%. The pipeline narrows invisibly. The reasoning from a CFO’s perspective is straightforward: why pay a junior developer $80–100K plus six months of ramp-up when a senior engineer with AI tools can cover triple the output? The math makes sense on a quarterly earnings call; it is catastrophic on a five-year horizon. Schools do not (yet) produce senior engineers - experience does. Debugging someone else’s code teaches you how systems fail, while writing boilerplate teaches you how systems are structured. Reviewing pull requests, even if stressful at first, teaches you how other people think about problems. Every one of those learning opportunities is a task that AI now handles, or that simply does not happen because there is no junior on the team to do it. A significant reduction in junior hiring between 2024 and 2026 means a proportional reduction in candidates for senior roles between 2031 and 2036. The industry is eating its seed corn. The Easy Button and the Erosion of Sharpness There is another dimension to this that extends beyond organizational hiring strategy and into individual skill development. When a tool exists that can do your job (at least the visible, measurable parts of it) for you, people will use it. It is human nature to be lazy. But consider what happens at the individual level. A mid-level engineer who has leaned heavily on AI code generation for two years stops building the neural pathways that come from working through problems manually. They lose the 30 seconds of working memory where they would have wired together the algorithm, considered edge cases, and built a mental anchor for that pattern. Multiply that across hundreds of completions per week, and the atrophy becomes significant. Luca Rossi describes two cognitive modes that matter here: create mode, where you actively build mental connections between ideas, and review mode, where you assess existing work with lower cognitive engagement. AI tools push developers from create mode into review mode by default. You stop solving problems and start evaluating solutions someone else produced. Review mode feels productive - you are reading code, spotting issues, making edits, but you are not building the mental model that lets you reason about the system independently. All of this is happening while the bar for what it means to be a qualified engineer is rising. FAANG interviews in 2026 have shifted from pure algorithmic puzzles toward scenario analysis, debugging exercises, and system reasoning under realistic constraints. Companies are looking for signals that cannot be autocompleted: the ability to read logs, investigate a performance regression, and explain why a request path suddenly slows under load. The interview process is selecting for exactly the kind of deep understanding that routine AI-assisted work erodes. The paradox is stark. The tool that was supposed to democratize software engineering is simultaneously making it easier to produce code and harder to develop the judgment needed to produce it well. A Humble Prediction: The Market Will Bifurcate If current trends continue (and every indicator suggests they will accelerate), the software engineering talent market is heading toward a painful bifurcation. On one side, a shrinking pool of senior engineers with genuine system understanding, architectural judgment, and the ability to reason about code they did not write. These people will command premium compensation and face relentless demand. They will also face relentless cognitive load because the organizational layers that used to absorb complexity beneath them are gone. On the other side, a growing population of developers who entered the profession during the AI era, who are proficient at prompting and reviewing but who never built the foundational mental models that come from years of hands-on struggle with real systems. They will be productive in narrow contexts and fragile in novel ones. They will pass AI-assisted coding assessments and struggle in incident rooms. The gap between these two groups will widen because nothing in the current incentive structure encourages closing it. What Can Be Done I do not pretend to have a complete playbook for this. But I believe the conversation needs to start in a few specific places. First, engineering organizations need to stop measuring AI adoption purely through velocity metrics. If your team is shipping 40% more code but your senior engineers are rubber-stamping reviews because they are overwhelmed, you have not improved; you have accumulated invisible debt that will come due at the worst possible moment.Second, the industry needs to redefine what a junior developer role looks like in 2026. The entry-level work is no longer writing boilerplate, but reviewing AI output, testing edge cases, writing better prompts, and building the judgment that AI cannot provide. The junior developer of 2026 looks different from the one we hired in 2018, and our job descriptions, onboarding, and expectations need to reflect that. But the role MUST exist. Eliminating it is organizational and market-wide amnesia in the long run.Third, individual engineers (at all levels) need to be honest with themselves about whether their daily workflow is building understanding or just building output. If you cannot explain a function to a colleague without referencing the prompt that generated it, you do not understand it well enough to own it, and that gap will inevitably catch up to you. AI tools are revolutionizing the software development space, but contrary to popular belief, AI did not replace senior engineers; it made them irreplaceable, overloaded, and increasingly alone. Organizations and engineers need to be aware of the hidden effects AI-assisted coding has and adapt accordingly.

By Abgar Simonean

Career Development

DZone's Featured Career Development Resources

Monthly Top Career Development Experts

The Latest Career Development Topics