DZone Spotlight

You’re debugging a bug related to inflated ratings in the /books/{id}/summary endpoint. You drop a breakpoint in BookService.getAverageRating(String), step through the code, and inspect the reviews list in the Variables view. Everything looks fine… until you spot a suspicious entry, a review for the same user added more than once. You pause and think: “Hmm… maybe these duplicate entries are causing the issue. Should I be using a Set instead of a List?” So, you try to locate where this reviews variable is declared. And that’s when it hits you, the code isn’t exactly minimal! 1. Navigate to Variable Declaration - “Which Variable Is This? I’ve Seen That Name 5 Times Already..” The code... The method uses the same variable name “review” multiple times. And to make things worse, reviews is also a parameter or variable name in other methods “Wait, which review am I actually looking at right now?” “Where exactly was this declared?” Instead of scrolling through code or trying to match line numbers, you can now use Eclipse’s Navigate to Declaration. Right-click the review variable in the Variables view and choose "Navigate to Declaration". Eclipse takes you straight to the correct line, whether it’s in the loop, the lambda, or the block. No more confusion. No more guessing. Note: This is different from Open Declared Type, which takes you to the class definition (eg. java.util.List for reviews variable), not the actual line where the variable was introduced in the code. 2. Collapse Stack Frames - “Where’s My Code in This Sea of Spring and Servlet Stack Traces?” While debugging that review variable earlier using Navigate to Declaration, you probably noticed something else: the stack trace was huge You might have thought: “Do I really need to see all these internal Spring and servlet calls every time I hit a breakpoint?” - Not anymore. Eclipse now gives you a Collapse Stack Frames option and it’s a lifesaver when working in frameworks like Spring Boot! In the Debug view, go to More options -> Java -> Collapse Stack Frames Now that’s clean! 3. Intelligent Stacktrace Navigation - “Do I Really Need the Full Class Name to Find This Method?” In a typical microservice or multi-module setup, you’ve probably done this: You added two new services for different book operations, like importing books and analytics. Each of them has its own BookService class (same name, but different methods). Everything works fine… until a test fails, and you’re left with a vague stack trace: BookService.formatSummary(Book) line: 44 In older Eclipse versions, unless you had the fully qualified class name, it would either show no matches or list every class named BookService which forced you to dig through them all. Like this…. Now you’re left wondering: “Which BookService is this coming from?” Not anymore… :D Now Eclipse Understands the Signature! If the stack trace includes the method name and signature, Eclipse can now disambiguate between classes with the same name. So on latest Eclipse (v4.35 onwards) you will be redirected to the indented class. “Yeah that’s the right one!” 4. JDK-Specific Navigation - “Why Am I Seeing StringJoiner From the Wrong JDK Version?” You’re debugging something deep.. maybe a library or JDK class like StringJoiner. You get a stack trace with versioned info like java.util.StringJoiner.compactElts([email protected]/StringJoiner.java:248) In older Eclipse versions, you’d get the list of available JDK source, maybe from Java 22,21 & 23 which doesn’t match what you’re running. Now, Eclipse understands the version specified in the stack trace and opens exactly the correct class from JDK 22.0.2, no mismatches, no confusion—pure results. 5. Auto-Resuming Trigger Points - “I Only Care About This One Flow, Why Do I Keep Stopping Everywhere Else?” You want to debug the BookService.formatSummary(Book) method that builds a book's display string. The issue is this method is used in multiple flows like /books/{id}/summary, /books/all, and /books/recommended. You care only about the /recommended flow, but every time you debug, Eclipse hits breakpoints in unrelated code flows. You keep hitting Resume again and again just to reach where you want to stop in the expected flow. Let Eclipse do the skipping for you… Here's how: Set a trigger point on the method getRecommendedSummaries() in BookController. Enable "Continue execution on hit" in the trigger then place a regular breakpoint or method entry breakpoint inside BookService.formatSummary(Book) Eclipse will skip the trigger point, fly past all the noise, and pause directly where your focus is, inside BookService.formatSummary(Book). You can even add a resume condition to the resume trigger (only resume when a specific condition is true else pause the execution similar to other breakpoints) for even more control. 6. Primitive Detail Formatter - “I Just Want to See This Value Rounded, but I Don’t Want to Touch the Code” On inspecting avgRating variable which of primitive double type in BookService.formatSummary(Book). Its showing something like 4.538888… but you want to see it rounded to two decimal places, just to understand what the final value would look like when shown to customers. Now with Primitive Detail Formatter support you can define a New Detail Formatter for double type in variables view directly or from Debug settings! Configure your formatter : Use “this” to represent the primitive Now, instead of showing the raw value, Eclipse will show you the adjusted value with two decimals... 7. Array Detail Formatter - “There’s an Array. I Only Care About One Value. Let Me See That.” In our temperature analytics method “BookService.getTemperatureStats()”, we're working with a hardcoded primitive array of temperature readings temps[]. But only the 6th value (index 5) matters. Say it represents the temperature recorded at noon. Usually, you'd scroll through the array or log specific indexes. But now, you can simply focus on exactly what you need. Simply add a Detail Formatter for arrays similar to previous one.. Use this[] to represent arrays And Voilà.. Perfect when you’re dealing with huge datasets but only care about a tiny slice… 8. Compare Elements - “I Wrote Three Different Ways to Generate Tags - But Which One Is Actually Doing a Good Job”? You’re developing a feature that adds tags to books like "java", "oop", or "bestseller". You’ve written three different methods to generate them, and now you just want to see if their results are consistent, complete, clean, and in the right order. You could print all the lists and manually check them, but when the lists are long, that gets slow and error-prone. Instead, you can now use Eclipse’s Compare Elements to instantly spot the differences. Simply select all three generated lists then Right-click → Compare Eclipse tells you what’s missing in each list when compared to the others. Note : You can compare Lists, Sets, Maps, Strings, and even custom objects - just make sure you're comparing the same type on both sides. Let’s take another scenario.. “These books look the same, but the system treats them differently - why?” You recently made Set<Book> allBooks to avoid duplicate book entries being listed but several customers have reported seeing "Clean Code" listed twice. And on debugging the source, you realize that every Book has different reference ids but still having duplicates in final result.. You suspect the issue is because filtering based on reference id’s, but it's hard to tell by glancing the objects, especially if there are many similar books. So for analysing you can Select the suspicious Book objects and then Right-click → Compare On comparing with Eclipse it is confirmed that object’s contents are same and it’s their reference id are different! – Bug confirmed! So now you can fix the bug by re-writing the logic to filter based on author and title! You also noticed that some books have identical title and author but different IDs, which sparks another idea…. maybe sometime in the future, duplicates should be identified based on content... 9. Disable on Hit - “I Only Want This Breakpoint to Hit Once - Not 100 Times.” You’re debugging a method like getAverageRating() that runs in a loop or across multiple books. You’re only interested in checking the first call, not stepping through all of them. You used to: Let it hit once then disable the breakpoint manually… Now Eclipse handles that for you. Click on a breakpoint and enable Disable on hit option Now on hitting the breakpoint it will automatically disabled. This is super handy for loops, Scheduled jobs & frequently called methods. 10. Labelled Breakpoints - “Wait… What Was This Breakpoint for Again?” After stepping through different flows, using resume triggers, collapsing stack frames, comparing variables, and disabling breakpoints on hit and modifying formatters of variables you now have 10+ breakpoints scattered across your project. One checks for null titles another one’s for catching duplicates one is testing trigger flow a few were just for one-time validations.. You open your Breakpoints view, and you’re met with this.. :O And now you're thinking: “Umm... which one was for verifying the tag issue again?Did I already add a breakpoint for the rating check?” No need to guess anymore. Eclipse now lets you label your breakpoints with meaningful descriptions. It’s like adding sticky notes directly to your debugging process. Right click on a breakpoint -> Label Once clicked provide your custom label :D Now your breakpoint will be highlighted with your custom label – finally! Debugging is one of those things we all do - but rarely talk about until it’s painful. And yet, when the right tools quietly guide you to the root cause, it feels effortless. That’s the kind of experience the Eclipse debugger aims to provide—thoughtful improvements that show up right when you need them most... :) If you run into an issue, spot something unexpected, or have an idea for improving these features (or even a brand new one!), feel free to raise it under here: https://github.com/eclipse-jdt/eclipse.jdt.debug Thanks for reading and until next time! Happy debugging!

Every BI engineer has been there. You spend weeks crafting the perfect dashboard, KPIs are front and center, filters are flexible, and visuals are clean enough to present to the board. But months later, you discover that no one is actually using it. Not because it’s broken, but because it doesn’t drive action. This isn’t an isolated issue, it’s a systemic one. Somewhere between clean datasets and elegant dashboards, the *why* behind the data gets lost. Business Intelligence, in its current form, often stops at the surface: build reports, refresh data, and move on. But visuals aren’t enough. What matters is decision utility, the actual ability of a data asset to influence strategy, fix problems, or trigger workflows. Dashboards without embedded insight aren’t intelligence. They’re decoration. When Clean Dashboards Mislead: A Quiet BI Failure A few years ago, a cross-functional product team rolled out a new feature and relied on a dashboard to track its impact. The visual was sleek and the conversion funnel appeared healthy. But something didn’t add up, the executive team wasn’t seeing the anticipated growth downstream. After a deep dive, it turned out the dashboard logic had baked in a rolling 30-day window that masked recent drop-offs. Worse, the metric definitions didn’t account for delayed user activation. The outcome? Teams doubled down on a strategy that was actually bleeding users. This incident wasn’t a failure of tools, it was a failure of interpretation, feedback, and context. That’s what happens when dashboards operate in isolation from stakeholders. Let’s break this down using a simplified SQL example. Here's what the flawed logic might have looked like: SQL SELECT user_id, event_date, COUNT(DISTINCT session_id) AS sessions FROM user_activity WHERE event_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) GROUP BY user_id, event_date; While technically valid, this logic excludes late activations and smooths over key behavioral shifts. A corrected version includes signup filters for active users: SQL WITH active_users AS ( SELECT user_id FROM user_events WHERE event_type = 'signup_confirmed' AND DATE_DIFF(CURRENT_DATE(), signup_date, DAY) <= 90 ) SELECT a.user_id, a.event_date, COUNT(DISTINCT a.session_id) AS sessions FROM user_activity a JOIN active_users u ON a.user_id = u.user_id GROUP BY a.user_id, a.event_date; This difference alone changed the trajectory of the team’s product decisions. From Reports to Results: The BI Gap No One Talks About The modern BI stack is richer than ever, BigQuery, Airflow, dbt, Tableau, Qlik, you name it. Yet, despite technical sophistication, too many pipelines terminate at a Tableau dashboard that stakeholders browse once and forget. Why? Because most BI outputs aren't built for real decisions. They’re built for visibility. But decision-making doesn’t thrive on static data points. It thrives on context, temporal trends, cohort shifts, anomaly detection, and most importantly, actionable triggers. Let’s consider a simple cohort segmentation approach that helps drive real outcomes: SQL SELECT user_id, DATE_TRUNC(signup_date, MONTH) AS cohort_month, DATE_DIFF(event_date, signup_date, DAY) AS days_since_signup, COUNT(DISTINCT session_id) AS session_count FROM user_sessions WHERE event_type = 'session_start' GROUP BY user_id, cohort_month, days_since_signup; This segmentation allows teams to observe how user engagement evolves across cohorts over time, a powerful signal for retention and lifecycle decisions. The Engineering Behind Useful BI A clean dashboard means little without a clean backend. Strong data engineering practices make all the difference between a flashy chart and a trustworthy business signal. Let’s look at two common building blocks. 1. Deduplicating Events: Deduplicating repeated user events ensures downstream metrics aren't inflated. Here's how that logic is typically implemented: SQL WITH ranked_events AS ( SELECT *, ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY event_timestamp DESC) AS rn FROM raw_events ) SELECT user_id, event_type, event_timestamp FROM ranked_events WHERE rn = 1; 2. Modeling Business KPIs in dbt: Business-level KPIs need consistent, traceable definitions. In dbt, we might define a revenue-per-cohort model as follows: SQL -- models/revenue_per_user.sql SELECT cohort_month, SUM(revenue) / NULLIF(COUNT(DISTINCT user_id), 0) AS revenue_per_user FROM {{ ref('cleaned_revenue_data') } GROUP BY cohort_month; And the accompanying schema tests help enforce data trust: YAML version: 2 models: - name: revenue_per_user tests: - not_null: - cohort_month - revenue_per_user - accepted_values: column_name: cohort_month values: ['2024-01', '2024-02', '2024-03'] Treat BI Like a Product: Users, Feedback, Iteration When BI is treated like a living system, not a static output, teams start optimizing for usage, clarity, and iteration. For instance, to track dashboard adoption: SQL SELECT dashboard_id, COUNT(DISTINCT user_id) AS viewers, AVG(session_duration) AS avg_time_spent, MAX(last_accessed) AS last_used FROM dashboard_logs GROUP BY dashboard_id ORDER BY viewers DESC; This data informs which assets should be retired, split, or iterated upon. When usage drops, it’s often a signal that the dashboard no longer answers the right questions. Mindset Over Toolset Ultimately, tooling alone doesn’t drive impact, clarity, iteration, and alignment do. This mindset shift is essential for any modern BI engineer. To support that, we regularly audit our metric catalogs: SQL SELECT metric_name, COUNT(*) AS usage_count, MAX(last_viewed_at) AS recent_use FROM metrics_metadata GROUP BY metric_name HAVING usage_count < 10; This simple query often uncovers stale metrics that confuse rather than clarify. The Architecture of Context: A Visual Walkthrough Here’s how well-structured BI pipelines tie it all together: Plain Text Data Sources ↓ ETL (Airflow, SQL) ↓ Semantic Layer (dbt, Python) ↓ Reporting Layer (Tableau, Qlik) ↓ Alerts & Feedback (Slack, Email) Let’s imagine you want to monitor funnel health. The detection logic might look like this: SQL SELECT funnel_step, COUNT(user_id) AS users FROM funnel_data GROUP BY funnel_step HAVING funnel_step = 'checkout' AND COUNT(user_id) < 1000; Once an anomaly is found, triggering an alert through Airflow keeps stakeholders in sync: Python from airflow.operators.email_operator import EmailOperator alert = EmailOperator( task_id='notify_low_checkout', to='[email protected]', subject='Checkout Drop Alert', html_content='User drop detected at checkout stage.', dag=dag ) The Future of Bi Is Invisible, but Influential BI is increasingly becoming modular, declarative, and headless. Metric layer tools like Cube.dev allow teams to define reusable KPIs that work across multiple surfaces. YAML cubes: - name: Revenue measures: - name: totalRevenue sql: SUM(${CUBE}.amount) This promotes consistency, reduces duplication, and enhances governance across teams. That’s the future of BI. Not just visual. Not just functional. But consequential.

Aside from those who have ignored technology trends for the last twenty years, everyone else is aware of — and likely working with — service-based architectures, whether micro, domain-driven, modulith, integration, data, or something else. From service-based, we’ve evolved to API-First, where APIs are first-class deliverables around which all solutions are built: front-end, back-end, mobile, external integrations, whatever. The APIs are intended to be implemented before other development work starts, even if the initial implementation is stubbed out, dummy code that allows other work to begin. API-First revolves around the contract. “Amelia in Code” by donnierayjones is licensed under CC BY 2.0. We software engineers focus on the way, too many different ways APIs may be implemented, but not our consumers: consumers’ concerns are that the published API contract is fulfilled as defined. Other than innate curiosity, consumers do not care about nor need to know the blood, sweat, and tears you poured into its implementation. Their concerns are fit for purpose and overall correctness. Organizations often expend considerable effort defining their API standards and applying these standards against their APIs to increase their chances of succeeding in an API-First world. I view API standards as a data contract for the consumer, akin to data standards defined to (hopefully) improve data handling practices within an organization. Yes, API functionality often extends beyond simple CRUD operations and shares many characteristics, but it is created and maintained by different groups within an organization. What Are Data Standards? “OMOP (IMEDS) Common Data Model (version 4)” by Wuser6 is licensed under CC BY-SA 4.0. Data standards, also known as data modeling standards, define an organization’s methodology for defining and managing mission-critical data to ensure consistency, understanding, and usability throughout the organization, including such roles as software engineers, data scientists, compliance, business owners, and others. Each organization has different goals and non-functional requirements important to their business, therefore no universal, all-encompassing standard exists. Methodology changes in the last 2+ decades have empowered software engineering teams to model their data solutions just in time, often with reduced involvement of the de facto data team (if one exists). Regardless, when organizational data standards are created (and enforced), its components often include: Naming: The term or vernacular used for consistency and ease of understanding when describing a subject area and its individual data elements, which may be industry standards, business-specific, or generic. Acceptable abbreviations may be part of naming standards. Occasionally, standards define names for internal database elements such as indices, constraints, and sequence generators.Data domains: Most data elements are assigned a data domain or class that defines its specific data implementation, for example: an id is the UUID primary key; a code is five alphanumeric characters; a percent is a floating point number with a maximum of four decimal points; a name contains between 5 and 35 alphanumeric characters.Structure: Determined by backend database — i.e., third-normal form in relational vs. data accessed together is stored together in document-orientedNoSQL — the persisted data can be dramatically different, though localized decisions still exist: business vs. system-generated primary keys; arrays, keys or sets for JSON subdocuments; data partitioning for performance and scaling; changes for development optimization. One solution implemented its own data dictionary on top of a relational database, simplifying customer configurability by sacrificing engineering simplicity.Validations: Data checks and verifications beyond what data types or data domains provide, often business- or industry-specific, such as A medical facility composed of one or more buildings, or a registered user must consist of a verified email address, corporate address, and job title. When implemented correctly, data validations improve data quality and increase the value of data to the organizations. This is not the all-inclusive list, as organizations with more mature data practices often have more components or areas covered. Unfortunately, in my opinion,, current favored methodologies often de-emphasized defining/applying formal data standards during feature development; however, that is a discussion for a different time. Digging Deeper It’s easiest to demonstrate the similarities of API and data standards with a real-life example, so let’s use the GitHub endpoint. Create a commit status for demonstration purposes: Users with push access in a repository can create commit statuses for a given SHA. Note: there is a limit of 1000 statuses per sha and context within a repository. Attempts to create more than 1000 statuses will result in a validation error. Fine-grained access tokens for “Create a commit status” This endpoint works with the following fine-grained token types: GitHub App user access tokensGitHub App installation access tokensFine-grained personal access tokens The fine-grained token must have the following permission set: URI The URI and its pathname contain both terminology (names) and data: a POST is primarily used when creating data, in our example, attaching (creating) a commit status to an existing commit;;the standard abbreviation for repository is repos;repos and statuses are plural and not singular (often requiring much discussion and consternation};{owner}, {repo}, and {sha} are values or path parameters provided by the caller to specify the commit being updated by this call;{owner} and {repo} are defined as case-insensitive. Notice that owners is not explicitly called out, such as hypothetically /owners/{owner}/repos/{repo}/statuses/{sha}. Because a GitHub repository is always identified by its owner and name. GitHub engineers likely concluded that including it was unnecessary, extraneous, or confusing. Headers The caller (consumer) declares the API version to use by specifying it in the HTTP header as a date. Alternatively, some API standards specify the version as a sequence number in the URI; presumably, other formats exist, of which I am unaware. Headers may also include data supporting non-functionality requirements, such as observability. Request Body The request body provides the data supporting this API call, in this case, the data required for creating a commit status. Points to make: only a top-level JSON document with no subdocuments;state is an enumerated value with four valid values;target_url is suffixed with a data type, _url implying a valid URL is expected;description is a short description, though what constitutes short, is unspecified: is it 20, 100, or 1024 characters? Request bodies are usually provided for HTTP POST, PUT, and PUT calls with an arbitrary size and complexity, based on the call's requirements. Call Response All APIs return an HTTP status code, and most return data representing the outcome (DELETE an expected exception). A successful API call for our demonstration purposes would be the created commit status. Unsuccessful API calls (non-2xx) often include error details that assist with debugging. The response for our API clearly shows the standards being applied: consistent _url data domain for the various URLs returned by GitHub, i.e., target_url, avatar_url, followers_url, etc;_id data domain for unique identifiers, i.e., node_id, gravatar_id, and standalone _id;creator subdocument which is the (complete?) user object;more plurals: followers_url, subscriptions_url, organizations_url, received_events_url. Query Parameters Our demonstration API does not have supporting query parameters, but APIs retrieving data often require these for filtering, pagination, and ordering purposes. Final Thoughts API standards and data standards are more similar, with similar issues and goals than many engineers wish to admit. Organizations would benefit if these standards were aligned for consistency rather than creating each in a vacuum. Though use cases supported by APIs often encompass more than base CRUD operations — and some APIs don’t result in persisted data — the API consumers view the contracts as data-driven. Therefore, applying well-known principles of data standards to your API standards increases the consistency and completeness of the APIs, reduces development time, and reduces errors. Originally published at https://scottsosna.com/2025/07/07/api-standards-are-data-standards/.

By Scott Sosna CORE

When building applications that rely on databases (which is almost every application, right?), one of the biggest challenges developers face is testing how their code handles various error scenarios. What happens when the database returns a HTTP 400 error? How does your application respond to throttling? Will your retry logic work as expected? These questions are crucial because, in production, errors are inevitable. This holds true for Azure Cosmos DB as well. The database's distributed nature means that errors can arise from various sources, including network issues (503 Service Unavailable), request timeouts (408 Request timeout), rate limits (429 Too many requests), and more. Therefore, robust error handling and testing are essential to maintain a reliable application that handles these gracefully rather than crashing or losing data. Testing edge cases and different permutations of error scenarios traditionally requires a lot of effort and can be quite complex. Common approaches include: Triggering real errors in your development environment (unreliable and hard to reproduce)Mocking entire SDK responses (complex and may not reflect real behavior)Waiting for errors to happen in production (definitely not ideal!) This is where error simulation can come in handy. By intercepting HTTP requests and selectively returning error responses, we can test specific scenarios in a controlled, repeatable manner. Simulating Errors Using Pluggable Mechanisms There are multiple ways to test error scenarios — from integration testing with real services to comprehensive mocking frameworks. This particular example uses the Go SDK for Azure Cosmos DB to illustrate how to simulate HTTP error conditions (403 Forbidden) for a specific operation (such as ReadItem). Note that this is just one technique in a broader toolkit of testing strategies. Follow the next steps if you want to run this as a standalone Go application and see how it works in practice. Or, feel free to skip to the next section for a walkthrough. Step 1: Copy the code below into a file named main.go: Go package main import ( "context" "errors" "fmt" "io" "net/http" "strings" "github.com/Azure/azure-sdk-for-go/sdk/azcore" azlog "github.com/Azure/azure-sdk-for-go/sdk/azcore/log" "github.com/Azure/azure-sdk-for-go/sdk/azcore/policy" azruntime "github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime" "github.com/Azure/azure-sdk-for-go/sdk/azidentity" "github.com/Azure/azure-sdk-for-go/sdk/data/azcosmos" ) func init() { azlog.SetListener(func(cls azlog.Event, msg string) { // Log retry-related events switch cls { case azlog.EventRetryPolicy: fmt.Printf("Retry Policy Event: %s\n", msg) } }) // Set logging level to include retries azlog.SetEvents(azlog.EventRetryPolicy) } // CustomTransport403Error implements policy.Transporter to simulate 403 errors only for ReadItem operations type CustomTransport403Error struct{} func (t *CustomTransport403Error) Do(req *http.Request) (*http.Response, error) { // Check if this is a ReadItem operation (typically a GET request with an item id in the path) // ReadItem URLs look like: /dbs/{db}/colls/{container}/docs/{id} isReadItemOperation := req.Method == "GET" && strings.Contains(req.URL.Path, "/docs/") if isReadItemOperation { fmt.Printf("CustomTransport403Error: Simulating 403 error for ReadItem operation: %s\n", req.URL.String()) // Create a simulated 403 response with sub-status 3 header := make(http.Header) header.Set("x-ms-substatus", "3") header.Set("x-ms-activity-id", "readitem-test-activity-id") header.Set("x-ms-request-id", "readitem-test-request-id") header.Set("Content-Type", "application/json") response := &http.Response{ StatusCode: 403, Status: "403 Forbidden", Header: header, Body: io.NopCloser(strings.NewReader(`{"code": "Forbidden", "message": "Simulated 403 error for ReadItem with sub-status 3"}`)), Request: req, } // Return both the response and the error so the SDK can handle it properly responseErr := azruntime.NewResponseError(response) return response, responseErr } // For all other operations (like account properties), use a fake successful response fmt.Printf("CustomTransport403Error: Allowing operation: %s %s\n", req.Method, req.URL.String()) // Create a fake successful response for account properties and other operations header := make(http.Header) header.Set("Content-Type", "application/json") header.Set("x-ms-activity-id", "success-activity-id") header.Set("x-ms-request-id", "success-request-id") response := &http.Response{ StatusCode: 200, Status: "200 OK", Header: header, Body: io.NopCloser(strings.NewReader("")), Request: req, } return response, nil } // RetryLoggingPolicy logs error details during retries type RetryLoggingPolicy struct{} func (p *RetryLoggingPolicy) Do(req *policy.Request) (*http.Response, error) { // fmt.Println("RetryLoggingPolicy: Starting retry with request URL:", req.Raw().URL.String()) // Call the next policy in the chain resp, err := req.Next() // If there's an error, log the details if err != nil { var azErr *azcore.ResponseError if errors.As(err, &azErr) { subStatus := azErr.RawResponse.Header.Get("x-ms-substatus") if subStatus == "" { subStatus = "N/A" } fmt.Printf("RetryLoggingPolicy: ResponseError during retry - Status: %d, SubStatus: %s, URL: %s\n", azErr.StatusCode, subStatus, req.Raw().URL.String()) } else { fmt.Printf("RetryLoggingPolicy: Non-ResponseError during retry - %T: %v, URL: %s\n", err, err, req.Raw().URL.String()) } } else if resp != nil && resp.StatusCode >= 400 { // Log HTTP error responses even if they don't result in Go errors subStatus := resp.Header.Get("x-ms-substatus") if subStatus == "" { subStatus = "N/A" } fmt.Printf("RetryLoggingPolicy: HTTP error response - Status: %d, SubStatus: %s, URL: %s\n", resp.StatusCode, subStatus, req.Raw().URL.String()) } return resp, err } func main() { opts := &azcosmos.ClientOptions{ ClientOptions: azcore.ClientOptions{ PerRetryPolicies: []policy.Policy{ &RetryLoggingPolicy{}, // This will log error details during retries }, Transport: &CustomTransport403Error{}, // Use the selective transport to simulate 403 errors only for ReadItem }, } creds, _ := azidentity.NewDefaultAzureCredential(nil) client, err := azcosmos.NewClient("https://i_dont_exist.documents.azure.com:443", creds, opts) if err != nil { fmt.Printf("NewClient Error occurred: %v\n", err) return } // Test the ReadItem operation container, err := client.NewContainer("dummy", "dummy") if err != nil { fmt.Printf("NewContainer Error occurred: %v\n", err) return } partitionKey := azcosmos.NewPartitionKeyString("testpk") _, err = container.ReadItem(context.Background(), partitionKey, "testid", nil) handlerError(err) } func handlerError(err error) { if err != nil { fmt.Println("ReadItem Error occurred") // Debug: Print the actual error type fmt.Printf("Error type: %T\n", err) // fmt.Printf("Error value: %v\n", err) var azErr *azcore.ResponseError if errors.As(err, &azErr) { fmt.Println("Successfully unwrapped to azcore.ResponseError using errors.As") fmt.Printf("error status code: %d\n", azErr.StatusCode) subStatus := azErr.RawResponse.Header.Get("x-ms-substatus") if subStatus == "" { subStatus = "N/A" } fmt.Printf("error sub-status code: %s\n", subStatus) } } } Step 2: Use the following commands to run the application: Go go mod init demo go mod tidy go run main.go Lets break this down and understand how it works. 1. Custom Transport Layer for Injecting Errors CustomTransport403Error is a custom HTTP transport that intercepts requests before they reach Azure Cosmos DB. This transport examines each request and decides whether to simulate an error based on the operation type (e.g., ReadItem). If the request matches the criteria, it returns a simulated error response; otherwise, it allows the request to proceed normally. It returns a 403 Forbidden HTTP response with a specific sub-status code (3 - WriteForbidden), and its wrapped in an azcore.ResponseError to closely mirror what Azure Cosmos DB would actually return. The x-ms-substatus header is particularly important for Cosmos DB applications, as it provides specific context about why an operation failed. It's included to make sure that error handling code processes responses exactly as it would in production. You can customize the error simulation logic as needed. Go type CustomTransport403Error struct{} func (t *CustomTransport403Error) Do(req *http.Request) (*http.Response, error) { isReadItemOperation := req.Method == "GET" && strings.Contains(req.URL.Path, "/docs/") if isReadItemOperation { fmt.Printf("CustomTransport403Error: Simulating 403 error for ReadItem operation: %s\n", req.URL.String()) // Create a simulated 403 response with sub-status 3 header := make(http.Header) header.Set("x-ms-substatus", "3") header.Set("x-ms-activity-id", "readitem-test-activity-id") header.Set("x-ms-request-id", "readitem-test-request-id") header.Set("Content-Type", "application/json") response := &http.Response{ StatusCode: 403, Status: "403 Forbidden", Header: header, Body: io.NopCloser(strings.NewReader(`{"code": "Forbidden", "message": "Simulated 403 error for ReadItem with sub-status 3"}`)), Request: req, } // Return both the response and the error so the SDK can handle it properly responseErr := azruntime.NewResponseError(response) return response, responseErr } // For all other operations, return a successful response // ... (successful response creation code) } The Go SDK for Azure Cosmos DB retries requests that return certain error codes, such as 403 (Forbidden), and others. This custom transport allows you to simulate these conditions without needing to actually hit the database. 2. Observing SDK Retries Using Custom Policies I covered Retry Policies in a previous blog post, "How to Configure and Customize the Go SDK for Azure Cosmos DB." In this example, a custom policy is used to provide visibility into the retry behavior. It logs detailed information about each retry attempt: Go type RetryLoggingPolicy struct{} func (p *RetryLoggingPolicy) Do(req *policy.Request) (*http.Response, error) { // Call the next policy in the chain resp, err := req.Next() // If there's an error, log the details if err != nil { var azErr *azcore.ResponseError if errors.As(err, &azErr) { subStatus := azErr.RawResponse.Header.Get("x-ms-substatus") if subStatus == "" { subStatus = "N/A" } fmt.Printf("RetryLoggingPolicy: ResponseError during retry - Status: %d, SubStatus: %s, URL: %s\n", azErr.StatusCode, subStatus, req.Raw().URL.String()) } } return resp, err } This is really useful for understanding how your application behaves under error conditions. When applications encounter errors, the SDK automatically retries those requests. They might ultimately succeed after a few attempts, but you may want to have visibility into this process. You can plug in a custom RetryLoggingPolicy to intercept these retries and log relevant information, such as the status code, sub-status code, and the URL of the request. This helps you understand how your application behaves during error conditions. 3. Integration and Error Handling Verification The main function ties everything together. It sets up the Cosmos DB client with the custom transport and retry policy, then performs a ReadItem operation. If an error occurs, it uses the handlerError function to extract and log the status code and sub-status code from the error response. The custom transport and retry policy are integrated into the application as part of ClientOptions. The custom transport is configured as the Transporter which represents an HTTP pipeline transport used to send HTTP requests and receive responses.The retry policy is added to the PerRetryPolicies list. Each policy is executed once per request, and for each retry of that request. Go func main() { opts := &azcosmos.ClientOptions{ ClientOptions: azcore.ClientOptions{ PerRetryPolicies: []policy.Policy{ &RetryLoggingPolicy{}, // This will log error details during retries }, Transport: &CustomTransport403Error{}, // Use the custom transport }, } // ... client creation and ReadItem operation ... _, err = container.ReadItem(context.Background(), partitionKey, "testid", nil) handlerError(err) } func handlerError(err error) { if err != nil { var azErr *azcore.ResponseError if errors.As(err, &azErr) { fmt.Printf("error status code: %d\n", azErr.StatusCode) subStatus := azErr.RawResponse.Header.Get("x-ms-substatus") if subStatus == "" { subStatus = "N/A" } fmt.Printf("error sub-status code: %s\n", subStatus) } } } Conclusion By making error scenarios easy to reproduce and test, you're more likely to build applications that handle them gracefully. This pattern can be extended to test various scenarios, such as different HTTP status codes (429 for throttling, for example), network timeouts, intermittent failures that succeed after retries, circuit breaker patterns, fallback mechanisms, and more. To begin with, you can focus on combination-specific operations (like read, or write) and error types that are most relevant to your application. You can gradually expand this to cover more complex scenarios, such as simulating throttling, server errors, or even network timeouts.

By Abhishek Gupta CORE

Java Collections components (such as Map, List, Set) are widely used in our applications. When their keys are not properly handled, it will result in a memory leak. In this post, let’s discuss how incorrectly handled HashMap key results in OutOfMemoryError. We will also discuss how to diagnose such problems effectively and fix them. HashMap Memory Leak Below is a sample program that simulates a memory leak in a HashMap due to a mutated key: Shell 01: public class OOMMutableKey { 02: 03: static class User { 04: 05: String name; 06: 07: User(String name) { 08: this.name = name; 09: } 10: 11: @Override 12: public int hashCode() { 13: return name.hashCode(); 14: } 15: 16: @Override 17: public boolean equals(Object obj) { 18: return obj instanceof User && name.equals(((User) obj).name); 19: } 20: } 21: 22: public static void main(String[] args) { 23: 24: Map<User, String> map = new HashMap<>(); 25: int count = 0; 26: 27: while (true) { 28: // Step 1: Create a key 29: User user = new User("Jack" + count); 30: map.put(user, "Engineer"); 31: 32: // Step 2: Change the key *after* insertion 33: user.name = "Jack & Jill" + count; 34: 35: // Step 3: Try to remove using the mutated key 36: map.remove(new User("Jack" + count)); // does not remove the record 37: map.remove(new User("Jack & Jill" + count)); // does not remove the record either 38: 39: if (++count % 100_000 == 0) { 40: System.out.println("Map size (leaked): " + map.size()); 41: } 42: } 43: } Before continuing to read, please take a moment to review the above program closely. In line #5, ‘User’ class is defined with the ‘name’ as the member/instance variable. This class has a legitimate ‘hashCode()’ and ‘equals()’ method implementation based on the ‘name’ variable.In line #27, this program goes on an infinite loop (i.e., ‘while(true)’) and creates new ‘User’ objects. In line #29, ‘name’ variable of the ‘User’ object is set to value ‘JackX’. In line #30, ‘User’ object is added to the ‘HashMap’. In line #33 ‘name’ of the user object is changed to ‘Jack & JillX’. Basically, the key of the ‘HashMap’ is mutated (i.e. changed). In line 36, ‘JackX’ ‘User record is removed, and in line #37 ‘Jack & JillX’ user record is removed from the ‘HashMap’. But both of the removals will silently fail, i.e., the user object will not be removed from the ‘HashMap’. Thus, when the program is executed, HashMap will start to grow with infinite user records and eventually result in ‘java.lang.OutOfMemoryError: Java heap space’. Why Does Mutable Key Result in OutOfMemoryError? Fig: HashMap Implementation In order to understand why the above program will result in OutOfMemoryError, we need to understand how HashMap’s are implemented. In a nutshell, HashMap internally contains an array of buckets. Inside each bucket, it has a list of records. HashMap uses the ‘hashcode()’ method of the key object to determine in which bucket the record should be stored. Once the bucket is determined, the record will be placed in the appropriate list of that bucket.When we use the ‘get()’ method to retrieve the record, HashMap uses the same ‘hashcode()’ method of the key object to determine the bucket in which the record should be searched. Once the bucket is determined, the ‘equals()’ method is invoked on all the record keys in the list of that bucket to retrieve the appropriate record. Equipped with this knowledge, let’s discuss what happens when the first ‘Jack1’ record is inserted into the ‘HashMap’. Based on the ‘hashcode()’ implementation in the User object, let’s say the ‘Jack1’ record gets inserted into the list in bucket#1. Once the record is stored, then the actual name is changed to ‘Jack & Jill1’ in the ‘HashMap’. So after the insertion, user record in bucket #1, contains ‘Jack & Jill1’ as the key and not ‘Jack1’ Now let’s answer the question, Why ‘map.remove(new User(“Jack” + count))’ doesn’t remove the record? Based on the ‘hascode()’ implementation of this ‘Jack1’ user object, HashMap will determine that the record is stored in bucket#1. Now HashMap will invoke the ‘equals()’ operation on all the keys that are present in list of bucket #1. ‘equals()’ operation will return ‘false’, because the actual name of this user object that is present in the list ‘Jack & Jill1’ and not ‘Jack1’ Now let’s answer the question, Why map.remove(new User(“Jack & Jill” + count))’ doesn’t remove the record? The ‘hashcode()’ implementation of ‘Jack & Jill1’, will return a different value, which will cause the HashMap to look up the record in a different bucket, let’s say bucket 3.Since in bucket #3, the record is not present, it will not be removed from the HashMap. Tricky, isn’t it? How to Diagnose a Mutable Key Created by Memory Leak? You want to follow the steps highlighted in this post to diagnose the OutOfMemoryError: Java Heap Space. In a nutshell, you need to do: 1. Capture Heap Dump You need to capture a heap dump from the application, right before the JVM throws an OutOfMemoryError. In this post, eight options to capture the heap dump are discussed. You might choose the option that fits your needs. My favorite option is to pass the ‘-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=<FILE_PATH_LOCATION>‘ JVM arguments to your application at the time of startup. Example: Shell -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/tmp/heapdump.hprof When you pass the above arguments, JVM will generate a heap dump and write it to ‘/opt/tmp/heapdump.hprof’ file whenever OutOfMemoryError is thrown. 2. Analyze Heap Dump Once a heap dump is captured, you need to analyze the dump. In the next section, we will discuss how to do a heap dump analysis. Heap Dump Analysis Heap Dumps can be analyzed through various heap dump analysis tools such as HeapHero, JHat, JVisualVM… Here, let’s analyze the heap dump captured from this program using the HeapHero tool. Fig: HeapHero flags memory leak using ML algorithm The HeapHero tool utilizes machine learning algorithms internally to detect whether any memory leak patterns are present in the heap dump. Above is the screenshot from the heap dump analysis report, flagging a warning that ‘main’ thread’s local variables are occupying 99.92% and most objects are occupied in one instance of ‘HashMap’. It’s a strong indication that the application is suffering from a memory leak, and it originates from the ‘java.util.HashMap’ object. Fig: Largest Objects section highlights ‘main’ Thread The ‘Largest Objects’ section in the HeapHero analysis report shows all the top memory-consuming objects (refer to the above screenshot). Here, you can clearly notice that the ‘main’ thread is occupying 99.92% of memory. Fig: Outgoing Reference section of ‘main’ Thread The tool also gives the capability to drill down into the objects to investigate their content. When you drill down into the ‘main’ Thread object, reported in the ‘Largest Object’ section, you can see all its child objects. From the above figure, you can see it contains 3.38 million User records. Basically, these are the objects that got added and never removed from the HashMap. Thus, the tool helps you to point out the memory-leaking object and its origin source, which makes troubleshooting a lot easier. How to Fix Mutable Keys Memory Leaks You can declare the key of the record to be final so that it can be changed once it’s initialized. Example: Shell 03: static class User { 04: 05: final String name; Conclusion From this post, we can understand that the mutated key in the Collections has the potential to bring down the entire application. Thus, by not mutating the key and using tools like HeapHero for faster root cause analysis, you can protect your applications from hard-to-detect outages.

By Ram Lakshmanan CORE

A previous article [Resource 1] provided general insights regarding Model Context Protocol, more exactly, it outlined how MCP can act as an universal adapter that allows AI assistants to securely access external systems in order to bring in new context that is useful to the interacting LLMs. The current article continues this analysis and exemplifies how a dedicated MCP server that is able to access a database can enable LLMs to inspect them and offer their users useful pieces of information. Users on the other hand, are now given the opportunity to automatically obtain actual business insights inferred directly from the existing data by using just the natural language. MCP servers in general expose three primitives — tools, resources and prompt templates. The experiment described further makes use of the first one. “The actors” involved are the following: Claude Desktop as the AI assistantClaude Sonnet 4.0 as the Large Language ModelThe user as the person interested in “having a conversation” to the databasePostgreSQL Server as the database server that holds the business dataPostgreSQL MCP Server as the leading actor that exposes the MCP tool Use Case Let’s assume a user can access the database and is interested in the following aspects: Concerning the database structure and the particular tables’ structureConcerning the data inside a schemaBusiness intelligence wise, that provides real business insights in a user-friendly manner When it comes to inquiries from the first two categories, these could be responded to easily by writing and executing the needed SQL queries directly against the database. Concerning the third category, one could still extract such pieces of information, but to accomplish that, a little bit more programming is required. What about not having to do any of these? The “distribution” above helps us achieve all only via configuration and by using the natural language. As already stated before, the ability to clearly formulate a question on a topic of interest and to write it down grammatically correct is still a useful (if not compulsory from a decency point of view) precondition. Preliminary Set-Up During this experiment, I used a Windows machine, thus the applications to install are destined for this platform, but the discussion is almost similar for the others as well. Database Set-Up Considering the PostgreSQL Database Server is up and running, one may create the simple schema that is used in this use case. Let it be called mcpdata. SQL create schema mcpdata; The experimental database models items from the same domain. Everything is simplified so that it is easier to follow and thus, the entities involved are the following: Vendor – designates a service provider – e.g. VodafoneService – represents a type of telecom service – e.g. VOIPStatus – a state an invoice may have at a certain moment – Under Review, Approved or PaidInvoice – issued for a service from a vendor, at a specific date, having a certain number and amount due The database initialization can be done with the script below. SQL drop table if exists vendors cascade; create table if not exists vendors ( id serial primary key, name varchar not null unique ); drop table if exists services cascade; create table if not exists services ( id serial primary key, name varchar not null unique ); drop table if exists statuses cascade; create table if not exists statuses ( id serial primary key, name varchar not null unique ); drop table if exists invoices cascade; create table if not exists invoices ( id serial primary key, number varchar not null unique, date date not null, vendor_id integer not null, service_id integer not null, status_id integer not null, total numeric(18, 2) default 0.0 not null, constraint fk_invoices_services foreign key (service_id) references services(id), constraint fk_invoices_statuses foreign key (status_id) references statuses(id), constraint fk_invoices_vendors foreign key (vendor_id) references vendors(id) ); Next, some experimental data is added. Although not much, it is more than enough for the use case here, nevertheless, one may add more or make modifications, as appropriate. SQL do $$ declare v_ver int; v_vdf int; v_org int; v_att int; s_voip int; s_eth int; s_tf int; s_mpls int; s_lo int; st_rev int; st_app int; st_paid int; begin insert into vendors (name) values ('Verizon') returning id into v_ver; insert into vendors (name) values ('Vodafone') returning id into v_vdf; insert into vendors (name) values ('Orange') returning id into v_org; insert into vendors (name) values ('ATT') returning id into v_att; insert into services (name) values ('VOIP') returning id into s_voip; insert into services (name) values ('Ethernet') returning id into s_eth; insert into services (name) values ('Toll Free') returning id into s_tf; insert into services (name) values ('MPLS') returning id into s_mpls; insert into services (name) values ('Local') returning id into s_lo; insert into statuses (name) values ('Under Review') returning id into st_rev; insert into statuses (name) values ('Approved') returning id into st_app; insert into statuses (name) values ('Paid') returning id into st_paid; insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('ver-voip-rev-1', '2025-06-02', v_ver, s_voip, st_rev, 151); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('ver-eth-rev-1', '2025-06-03', v_ver, s_eth, st_rev, 240); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('ver-tf-paid-1', '2025-06-04', v_ver, s_tf, st_paid, 102.44); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('ver-mpls-app-1', '2025-06-01', v_ver, s_mpls, st_app, 42.44); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('ver-lo-paid-1', '2025-06-05', v_ver, s_lo, st_paid, 113.44); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('vdf-lo-paid-1', '2025-06-10', v_vdf, s_lo, st_paid, 85.44); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('vdf-mpls-app-1', '2025-05-10', v_vdf, s_mpls, st_app, 80.44); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('vdf-tf-rev-1', '2025-05-20', v_vdf, s_tf, st_rev, 10.44); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('org-voip-paid-1', '2025-04-10', v_org, s_voip, st_paid, 50.81); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('org-voip-paid-2', '2025-05-10', v_org, s_voip, st_paid, 50.81); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('org-voip-paid-3', '2025-06-10', v_org, s_voip, st_paid, 50.81); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('att-eth-app-1', '2025-06-13', v_att, s_eth, st_app, 100); insert into invoices (number, date, vendor_id, service_id, status_id, total) values ('att-mpls-paid-1', '2025-06-20', v_att, s_mpls, st_paid, 98); end; $$; AI Host and MCP Server Set-up Since Claude Desktop will be used, it is first installed locally [Resource 3]. For the purpose of the experiment described in this article, signing up to a Claude AI free plan is enough. Once the application is started, it is possible to begin interacting with the available LLMs — Claude Sonnet 4.0 in this case. Without connecting it to a PostgreSQL MCP Server, the AI assistant has obviously no clue about the data that resides in the database. It lacks the particular context. To overcome this, PostgreSQL MCP Server is plugged into the AI client via configuration. In Claude Desktop, go to File –> Settings. Select Developer, then click Edit Config. Once the button is pressed, the user is indicated a JSON file the below snippet shall be written to. In my case, this is C:\Users\horatiu.dan\AppData\Roaming\Claude\claude_desktop_config.json. JSON { "mcpServers": { "postgres": { "command": "npx", "args": [ "-y", "@modelcontextprotocol/server-postgres", "postgresql://postgres:a@localhost:5432/postgres?currentSchema=mcpdata" ] } } } The PostgreSQL connection string at line 8 is the important one here and it shall be adapted from case to case. Nevertheless, the previously created mcpdata schema is used. Before being able to use this MCP server inside Claude Desktop, there is one more prerequisite – the server needs to be run. In this experiment, Node.js is used and consequently, it must be also installed and made available in the local path. If needed, [Resource 1] might help as it provides the configuration steps in detail. At this point, if all above steps have been accomplished as advised, after restarting Claude Desktop, PostgreSQL MCP server should be available and ready to be used. “Discussing” With the Database In this section, several interactions are analyzed, in accordance with the categories mentioned before. As the pieces of information requested are about the mcpdata database schema, Claude client now uses the MCP server integration to find the useful details. Whenever the tool considers whether it should use the server, it asks for permission. If allowed, it continues towards compiling the response. If interested in the current database structure, one may ask the AI assistant — Generate an entity-relationship diagram of the mcpdata schema. Depending on the intermediary results and the conclusions inferred, the model might perform several interactions with the database, as it considers fit. The result is the aimed ER diagram: Additionally, a great deal of useful explanations are returned as well, Plus a one paragraph conclusion: “This is a classic invoice management system design where the INVOICES table serves as the central fact table, with three lookup/dimension tables (VENDORS, SERVICES, STATUSES) providing normalized reference data. The relationships ensure data integrity and eliminate redundancy.” Another question in the same category might be — What tables are there in the ‘mcpdata’ schema and how many records are in each? The results are accurate, as expected, Claude Desktop triggered two queries to respond: JSON { `sql`: `SELECT table_name FROM information_schema.tables WHERE table_schema = 'mcpdata' ORDER BY table_name;` } and JSON { `sql`: `SELECT 'invoices' as table_name, COUNT(*) as record_count FROM mcpdata.invoices UNION ALL SELECT 'services' as table_name, COUNT(*) as record_count FROM mcpdata.services UNION ALL SELECT 'statuses' as table_name, COUNT(*) as record_count FROM mcpdata.statuses UNION ALL SELECT 'vendors' as table_name, COUNT(*) as record_count FROM mcpdata.vendors ORDER BY table_name;` } Moving further, a questions with slightly more business flavor might be — How many paid invoices are in June? The response is again correct and the way the AI assistant approached the problem is detailed below. It first checked the structure of the invoices and statuses tables: JSON { `sql`: `SELECT column_name, data_type FROM information_schema.columns WHERE table_schema = 'mcpdata' AND table_name = 'invoices' ORDER BY ordinal_position;` } { `sql`: `SELECT * FROM mcpdata.statuses;` } Then, it inferred that the unique identifier of the ‘Paid’ Status entity is 3. Since the user did not specify a year in the prompt, the AI assistant looked up all existing years: JSON { `sql`: `SELECT DISTINCT EXTRACT(YEAR FROM date) as year FROM mcpdata.invoices ORDER BY year;` } And as there are only invoices from the year of 2025, it counted these: JSON { `sql`: `SELECT COUNT(*) as paid_invoices_in_june FROM mcpdata.invoices WHERE status_id = 3 AND EXTRACT(MONTH FROM date) = 6 AND EXTRACT(YEAR FROM date) = 2025;` } Which resulted in the exact number of 5. Continuing the conversation, maybe the user is interested in what the total amount to be paid is and thus it asks — How much do we have to pay for these? The response is accurate again, concluding — “The total amount for the 5 paid invoices in June 2025 is $450.13.”. The AI assistant also makes an assumption, which might be true, depending on the design of the invoice statuses — “Since these invoices have a status of “Paid”, this represents money that has already been paid out, not money that still needs to be paid.” Basically, the user and the AI host are now engaged in a conversation held in natural language. Last but not least, if the user aims for more when it comes to the current state of the invoices in the system, a visual representation might be requested to the AI Assistant. “Generate a visual representation of the data in the invoices table” is the highlight request in this experiment. Once the pieces information needed are gathered with the help of the used MCP Server, a comprehensive visual and interactive dashboard is created. Details are also outlined as text. Visually, the generated graphs and reports look pretty good and might help one get a glance of the current state. “Total Amount by Vendor”, “Amount by Status”, “Monthly Invoice Trends” and “Invoice Details” are just a few examples the AI Assistant decided to compile and display to the user. No doubt it could do more, as the request here was quite vague. Conclusion This article exemplified a straight-forward, but insightful use case where a dedicated PostgreSQL MCP Server helped enriching the context of an AI application, in this case Claude Desktop and allowed the user quicky discover pieces of information inferred directly from the database, in a natural language conversational manner. SQLite MCP Server is another example of a solution which may be used to accomplish similar tasks. In addition to read-only tools, this one though allows creating / modifying database tables and writing data, thus the interaction may be even more insightful. In the particular case of this article, a user might even most probably be capable of adding invoices, which is indeed pretty spectacular. All in all, the imagination can fly freely as a lot of ideas are just a few “words” distance to being put into practice by using an AI assistant (Claude Desktop) and several MCP servers. Going even further, if the currently available MCP servers are not enough, one can develop its own brand-new ones that satisfy their needs and plug it into the assistant and Voila! Resources [Resource 1] – Enriching AI with Real-Time Insights via MCP[Resource 2] – PostgreSQL MCP Server[Resource 3] – Claude Desktop is available here[Resource 4] – SQLite MCP Server[Resource 5] – Anthropic – Developer Guide – MCP[Resource 6] – the picture is of a painting from ‘Harry Potter Warner Bros. Studios’, near London

Look, I've been in cybersecurity for over a decade, and I'm tired of seeing the same preventable disasters over and over again. Cloud security breaches aren't happening because of some sophisticated nation-state actor using a zero-day exploit. They're happening because someone forgot to flip a switch or left a door unlocked. The numbers are frankly depressing. According to Verizon's latest Data Breach Investigations Report, misconfiguration errors account for more than 65% of all cloud-related security incidents. IBM puts the average cost of a misconfiguration-related breach at $4.88 million. But here's what really gets me — these aren't just statistics. Behind every one of these numbers is a real company that had to explain to its customers why their personal data was sitting on the internet for anyone to grab. Remember when Capital One got hit in 2019? One misconfigured web application firewall. That's it. 100 million customer records exposed because of a single configuration error. Toyota's breach earlier this year? Cloud misconfigurations that went unnoticed for almost ten years. These companies aren't small startups — they have entire security teams, compliance departments, and millions of dollars in security budgets. The thing is, cloud platforms have made it incredibly easy to spin up infrastructure. AWS, Azure, Google Cloud — they've all designed their platforms to get you up and running as quickly as possible. Click, click, deploy. But here's the catch: the default settings are optimized for functionality, not security. And when you're racing to meet a deadline or fix a production issue, security configurations tend to get overlooked. I've seen it happen countless times. A developer needs to deploy something quickly, so they take the path of least resistance. A DevOps engineer is troubleshooting an access issue at 2 AM, so they temporarily broaden permissions "just to get things working." A database administrator needs to share data with a partner, so they make a bucket publicly readable "for now." These temporary fixes have a funny way of becoming permanent. So, let's talk about the seven cloud misconfigurations that I see most often in my work — the ones that make hackers' jobs ridiculously easy and, more importantly, how you can fix them before they cost you millions. 1. Public S3 Buckets and Storage Blobs: Digital Doors Left Wide Open This one's the classic. AWS S3 buckets, Azure Blob containers, and Google Cloud Storage buckets — configured with public read or write access when they absolutely shouldn't be. It's like leaving your front door open with a sign that says "valuable stuff inside." Here's how simple this attack is: hackers run automated tools that scan for publicly accessible buckets. They'll try common naming patterns like "companyname-backups" or "appname-database" or "projectname-logs." When they find one that's publicly accessible, it's game over. They download everything — customer data, financial records, source code, internal documents, and API keys. Everything. The Accenture breach in 2017 is a perfect example. Four Amazon S3 buckets containing incredibly sensitive data were left publicly accessible for months. We're talking about passwords, secret keys, decryption certificates, customer information — the whole nine yards. Nobody at Accenture even knew about it until a security researcher happened to stumble across it. What's particularly frustrating about this misconfiguration is how easy it is for attackers to find these exposed buckets. Tools like GrayhatWarfare and Bucket Stream are specifically designed to scan the internet for publicly accessible cloud storage. They use dictionary attacks with common naming conventions, subdomain enumeration, and sometimes just brute force. Once they're in, they can exfiltrate data or even upload malicious files. The fix is straightforward, but it requires discipline. First, enable account-level public access blocking. In AWS, there's literally a setting called "Block all public access" at the account level — use it. Don't rely solely on bucket-level policies because they're too easy to accidentally override when you're in a hurry. Second, use explicit deny policies for public access. Make your bucket policies crystal clear about what's not allowed. Third, enable server-side encryption with customer-managed keys. Fourth, set up CloudTrail logging to monitor all bucket access — you want to know who's accessing what and when. And please, for the love of all that's holy, audit your existing buckets regularly. Tools like Prowler, ScoutSuite, or AWS Config can automatically scan for public buckets. There's no excuse for not knowing the security posture of your own infrastructure. 2. Overly Permissive IAM Roles: The "Everyone's an Admin" Problem Identity and access management (IAM) is where good security intentions go to die a slow, painful death. I can't tell you how many times I've seen IAM policies that look like they were designed by someone throwing darts at a permissions board. Here's what typically happens: a developer needs to deploy an application so they get some basic permissions. Then, they need to access a database so they can get database permissions. Then, they need to read from S3, so they get S3 permissions. Then, they need to write logs so they get logging permissions. Pretty soon, they have permission to half the AWS console, and nobody remembers why. The principle of least privilege sounds great in theory. In practice, it's a nightmare. Figuring out the exact minimum permissions needed for a specific role often involves hours of trial and error. When something breaks in production, the quickest fix is usually to add more permissions, not spend time figuring out which specific permission is missing. Microsoft did an analysis in 2021 and found that 99% of cloud identities have more permissions than they actually need. Ninety-nine percent! This isn't a few bad apples — this is systemic across the entire industry. From an attacker's perspective, this is fantastic news. They only need to compromise one account with excessive permissions to get access to a huge amount of resources. Once they're in, they can move laterally across your infrastructure, escalate their privileges, access sensitive data across multiple services, and create persistent backdoors by spinning up new IAM users or roles. The fix starts with understanding what permissions you actually have. Use AWS IAM Access Analyzer, Azure AD Access Reviews, or Google Cloud IAM Recommender to get visibility into unused permissions. These tools will show you which permissions are actually being used and suggest tighter policies. Implement just-in-time access for administrative functions. Instead of giving someone permanent admin access, give them temporary elevated permissions when they need to perform specific tasks. Use service-specific roles instead of broad administrative permissions — if someone only needs to read from S3, don't give them EC2 permissions too. Enable multi-factor authentication for everyone, but especially for privileged accounts. And audit your IAM policies regularly. I know it's tedious, but tools like Rhino Security Labs' Pacu or Prowler can help automate the process and show you exactly how exposed you really are. 3. Exposed Services Without Authentication: The Internet's All-You-Can-Eat Buffet This one makes me want to bang my head against a wall. Database servers, Elasticsearch clusters, Redis instances, and MongoDB databases — all running without authentication on ports that are accessible from the internet. It's like setting up a buffet table on the sidewalk and then acting surprised when people help themselves. The 2017 Equifax breach got a lot of attention for the unpatched Apache Struts vulnerability, but the real damage came from exposed databases that didn't require authentication. The attackers didn't need to use sophisticated techniques or zero-day exploits. They just connected directly to the databases and started downloading data. What's particularly maddening is how easy it is to find these exposed services. Shodan, which is basically Google for internet-connected devices, makes it trivial. Search for "mongodb" or "elasticsearch" and you'll find thousands of unprotected databases. Some of them contain millions of records. Personal information, financial data, health records — all sitting there for anyone to grab. I've personally seen databases containing customer payment information, employee records, and even government data that were accessible without any authentication whatsoever. In some cases, the databases were not only readable but also writable, meaning attackers could modify or delete data. The attack methodology is brutally simple. Attackers use Shodan to find exposed services, connect directly to them without any credentials, and then either exfiltrate the data or deploy ransomware. Sometimes, they'll use these exposed services as pivoting points to attack other parts of your infrastructure. The fix should be obvious: never expose databases directly to the internet. Use VPCs, private subnets, and security groups to restrict access to only the systems that actually need it. Implement authentication for all services, even internal ones — you never know when an attacker might get inside your network. Use multiple layers of security. Network-level firewalls, application-level security groups, and proper VPC design. Enable encryption in transit and at rest. Monitor network traffic for unusual patterns — if someone's downloading gigabytes of data from your database at 3 AM, you probably want to know about it. AWS Systems Manager Session Manager, Azure Bastion, and Google Cloud Identity-Aware Proxy can all provide secure access to your systems without exposing them directly to the internet. 4. Disabled or Missing Logging: Flying Blind Let me be blunt: logging is boring. It's expensive. It generates massive amounts of data that nobody wants to look at. It slows down systems. It fills up storage. So it gets disabled, misconfigured, or just ignored. This is exactly what attackers are counting on. If you're not logging security events, you have no idea when someone's attacking your systems. The average time to detect a breach is still over 200 days, according to IBM's research. That's more than six months of an attacker having free reign in your environment. I've seen organizations get breached where the attackers had been inside their systems for over a year. They had access to customer data, financial records, intellectual property — everything. And nobody knew because the logging was either disabled or nobody was monitoring it. From an attacker's perspective, disabled logging is a gift. They can operate for extended periods without detection. They can delete or modify logs to cover their tracks. They can escalate their attacks knowing that there's no monitoring in place. They can establish persistent access without triggering any alerts. The fix starts with enabling proper logging. In AWS, that means CloudTrail. In Azure, it's Activity Log. In Google Cloud, it's Audit Logs. But don't just enable them — configure them properly. Log to immutable storage so attackers can't delete the evidence. Set up real-time alerting for suspicious activities. Use SIEM tools like Splunk, the ELK Stack, or cloud-native solutions to actually analyze the logs. And here's the key: actually monitor them. I can't tell you how many organizations have comprehensive logging set up, but nobody's paying attention to the alerts. Dead logs are worse than no logs because they give you false confidence. Tools like Falco, Osquery, or cloud-native monitoring services can provide real-time threat detection. But they're only useful if someone's actually responding to the alerts. 5. Hardcoded Secrets: The Digital Equivalent of Writing Your Password on a Post-It Note API keys, database passwords, encryption keys, OAuth tokens — hardcoded directly in source code, configuration files, or environment variables. It's like writing your house key's location on your front door, except worse because the information is permanently stored and searchable. GitHub's 2023 security report found that developers accidentally commit secrets to public repositories over 10 million times per year. That's roughly one secret every three seconds. And here's the kicker: attackers have automated tools that scan new commits in real time, looking for exactly these kinds of exposed credentials. I've seen AWS access keys hardcoded in JavaScript files, database passwords in Docker containers, and API keys in configuration files that get deployed to production. Once these secrets are exposed, attackers can use them to access your cloud resources, databases, and third-party services. The attack methodology is increasingly automated. Attackers scan public repositories on GitHub, GitLab, and Bitbucket, looking for patterns that match common secret formats. They analyze container images for embedded secrets. They compromise CI/CD pipelines to extract environment variables. They scan running containers for configuration files. The worst part is that these secrets often have broad permissions. A single exposed AWS access key might have permission to read S3 buckets, launch EC2 instances, and access databases. An exposed database password might provide access to customer data, financial records, and personal information. The fix is to use dedicated secrets management services. AWS Secrets Manager, Azure Key Vault, HashiCorp Vault, and Google Secret Manager — these tools are specifically designed to handle sensitive credentials securely. Never hardcode secrets in source code. Use environment variables, but make sure they're properly secured and have appropriate access controls. Implement pre-commit hooks that scan for secrets before code gets committed to version control. Tools like GitLeaks, TruffleHog, or cloud-native secret scanning can catch exposed credentials before they become a problem. And rotate your secrets regularly — assume that anything that's been in your codebase for more than a few months has been compromised. 6. Unpatched Services: The Classics Never Go Out of Style Here's a misconception I hear all the time: "We're in the cloud, so security is someone else's problem." Wrong. Your virtual machines, containers, and services still need patches. They still need updates. They still need maintenance. The 2017 WannaCry ransomware attack exploited a Windows vulnerability that Microsoft had patched months earlier. The organizations that got hit weren't running some exotic, unsupported software — they were running standard Windows systems that simply hadn't been updated. In the cloud, this problem is actually amplified because it's so easy to spin up infrastructure and then forget about it. I've seen cloud environments with hundreds of virtual machines running operating systems that haven't been updated in years. Critical vulnerabilities with public exploits are available, and nobody's bothering to patch them. Sometimes, the virtual machines are running applications that were deployed once and then forgotten about. From an attacker's perspective, this is low-hanging fruit. They run vulnerability scanners to identify unpatched systems, then use existing exploit tools to compromise them. Once they have access to one system, they can use it as a launching point for attacks against other resources. They might deploy ransomware, install crypto-mining software, or use the compromised systems to attack other organizations. The fix is to treat patching as a first-class operational concern. Enable automatic patching wherever possible. AWS Systems Manager Patch Manager, Azure Update Management, and Google Cloud OS Patch Management can all handle routine patching automatically. Build patching into your CI/CD pipelines. Use immutable infrastructure where you replace entire systems rather than patching them in place. Implement vulnerability scanning as part of your deployment process — don't deploy systems that have known vulnerabilities. Tools like Nessus, OpenVAS, or cloud-native vulnerability scanners can help identify unpatched systems. But scanning is only useful if you actually remediate the vulnerabilities they find. 7. Lack of Network Segmentation: The Digital Open Floor Plan Flat networks where every resource can communicate with every other resource. It's like having an office building where every door is unlocked, and anyone can walk into any room. Once an attacker gets access to one system, they can potentially access everything. Network segmentation is one of the most effective security controls available, but it's also one of the most neglected. Cloud platforms make it incredibly easy to deploy resources quickly, but proper network design requires planning, documentation, and ongoing maintenance. It's not sexy, and it doesn't directly contribute to shipping features, so it often gets overlooked. I've seen cloud environments where the production database server can communicate directly with the public-facing web servers, the development environment, and the internal administrative systems. There's no logical separation between different tiers of the application, different environments, or different security zones. From an attacker's perspective, this is fantastic. Once they compromise one system, they can move laterally throughout your entire infrastructure. They can access sensitive resources from compromised systems. They can escalate privileges using network-based attacks. They can exfiltrate data across network boundaries without triggering alerts. The fix is to implement proper network design from the beginning. Use VPCs with private and public subnets. Implement security groups and Network Access Control Lists (NACLs) to restrict traffic between different tiers of your application. Consider micro-segmentation for critical resources. Use network monitoring tools to detect unusual traffic patterns. If your web server is suddenly communicating with your database server on non-standard ports, you want to know about it. Consider implementing a zero-trust network architecture where every connection is authenticated and authorized. Tools like AWS VPC Flow Logs, Azure Network Watcher, and Google Cloud VPC Flow Logs can provide visibility into network traffic patterns and help you identify potential security issues. Moving Forward: It's Not Just About Technology Here's the thing that really frustrates me about cloud security: these misconfigurations aren't just technical problems. They're organizational problems. They happen when security is treated as something you bolt on at the end rather than something you build in from the beginning. They happen when development teams are incentivized to ship features quickly but not to ship them securely. The solution isn't just better tools, although tools certainly help. The solution is better processes, better training, and better organizational culture. Infrastructure as Code with security scanning built in. Automated compliance checks that run as part of your CI/CD pipeline. Regular security audits that actually get acted upon. Security training for developers that goes beyond "don't click on suspicious links." Cloud Security Posture Management (CSPM) tools like Prisma Cloud, Wiz, and Orca Security can help automate the detection and remediation of these misconfigurations. But tools alone aren't enough. You need people who understand the risks, processes that incorporate security from the beginning, and leadership that values security as much as they value new features. The reality is that attackers aren't getting more sophisticated — they're getting more efficient at exploiting the same basic mistakes that organizations have been making for years. The question isn't whether your cloud infrastructure has misconfigurations. The question is whether you'll find them before the attackers do. My advice? Review your configurations today. Not next week when you have more time. Not next month when the current project is finished. Today. Because in the cloud, security isn't someone else's responsibility — it's yours. And trust me, fixing these issues proactively is a lot cheaper than dealing with a data breach.

In my previous blog post, I introduced the concept of vibe coding. It is one of the new ways that is attracting even non-programmers. Users can describe their thoughts using natural language, and AI tools can convert that into a working application. Spotting this opportunity, I thought I should experiment and understand what that actually looks like in action. I took this opportunity to test out a few tools and see how they really impact my workflow. Vibe coding is a declarative approach It is not just about automating tasks; it is about changing our behavior on how to approach a problem. To me, it feels like a declarative approach, especially when you are navigating a new framework or language for the first time. Smarter Coding in the IDE: GitHub Copilot I first started with the most common tool that is gaining popularity in the corporate world. It is GitHub Copilot. I have been using it regularly in VS Code, and while it is not exactly magic, it is undoubtedly helpful. When I am deep into the code, I use it for quick assistance with things like scaffolding code, writing tests, or debugging tricky edge cases. It saves me time context switching to a browser. You can interact with Copilot right in your code, or you can open a chat window to explain your issue. Copilot now has an “agent mode,” which allows you to give broader instructions that can span across multiple files at the feature level. It is not flawless and can sometimes come up with generic solutions. But most of the time, it helps to cut down time spent on the boilerplate code. Github Copilot with agent mode The best part of Copilot is that it is embedded in the editor, which means it is just a click away whenever I need it. Also, it is context-aware; it often makes suggestions that fit with existing application style and architecture. Github Copilot inline editing Terminal-First Development: Claude Code There are developers who love working in the terminal. I would suggest tools like Claude Code or OpenCodeX CLI for them. These are worth checking out. Although it works from the terminal, but it understands your entire codebase and can implement changes based on natural language commands. I recently tried out Claude Code for a side project. From the CLI, I was able to set up a basic project, refactor functions, and even tidy up some old scripts. It is not a traditional, simple text-based interface, and I recommend it as a must-have in your toolkit. Anthropic has done a good job with Claude Code documentation and working examples. It is worth checking out if you are curious. AI in the Browser: Replit The other tool I tried was Replit. It has the latest AI features in the browser and targets developers above the beginner level. I provided the below prompt to generate a Trello-like dashboard to manage agile teams. "Generate me an app, something like Trello, where I can track my tasks from to-do, in-progress, and done. It should have a board to move tickets and ticket analysis capabilities." Replit prompt example What really impressed me was how collaborative the experience is — you provide a prompt, and it guides you through the development process step by step. It feels almost like working alongside a junior developer who has done their homework. You are not just getting code; you are getting a plan, followed by clean, organized output. And if something goes wrong, Replit adapts and tries again. It really helps you navigate the development journey, generating and even debugging as it goes. This tool is especially convenient for generating a quick prototype. While it might not be the best fit for highly complex systems, it shines when you need to get something up and running in a browser environment. Below is a sample application that it generated for my above prompt. It has done a decent job and has added features like tags for tickets, included a date on every ticket, and even presented an in-progress bar per ticket. Replit preview example A Growing Ecosystem These are not the only available options in the market. The app store for AI-driven development is rapidly filling up, and each tool has its own feature set and niche. Lovable is a great option for working with UIs and interfaces that you can tweak visually. It is another non-IDE alternative that lets you interact with simple natural language text prompts. You can describe what you want, and it updates the interface accordingly. It also supports backend connections, integrations, and multi-user collaboration.Bolt.new is another available option for full stack apps and frameworks like `Next.js` or `Svelte`, and mobile via Expo. I think Bolt is outstanding if you are a beginner. The design outputs are sometimes a bit better than what I was expecting.Another similar tool is V0 by Vercel. It is a more developer-focused online tool. It is built by the team behind `Next.js`, and it supports modern web technologies out of the box. The tool to consider and adopt really depends on your problem statement. Final Thoughts I think AI tools are enhancing our programming capabilities. If I have to pick one of those, I will pick one that seamlessly blends into the background, providing just the right amount of assistance without being intrusive. Tools are only valuable if they help you build faster and improve the overall experience. I will continue the discussion and explore System Prompts in part 3 of this series.

By Gaurav Gaur CORE

Mobile development presents unique challenges in delivering new features and UI changes to users. We often find ourselves waiting on App Store or Play Store review cycles for even minor UI updates. Even after an update is approved, not all users install the latest version right away. This lag means some portion of our audience might be stuck on older UIs, leading to inconsistent user experiences across app versions. In traditional native development, any change to the interface — from a simple text tweak to a full layout overhaul — requires releasing a new app version. Combined with lengthy QA and release processes, this slows down our ability to respond to feedback or run timely experiments. Teams have explored workarounds to make apps more flexible. Some have tried loading portions of the UI in a web view, essentially embedding web pages in the app to avoid full releases. Cross-platform frameworks like React Native and Flutter reduce duplicated effort across iOS and Android, but they still package a fixed UI that requires redeployment for changes. In short, mobile UIs have historically been locked in code at build time. This rigidity clashes with the fast pace of modern product iterations. We need a way to change app interfaces on the fly — one that doesn’t sacrifice native performance or user experience. This is where server-driven UI (SDUI) enters the picture. The Concept of Server-Driven UI Server-driven UI (SDUI) is an architectural pattern that shifts much of the UI definition from the app to the server. Instead of baking every screen layout and widget into the mobile app binary, with SDUI, we allow the server to determine what UI components to display and how to arrange them. The client application (the app) becomes a rendering engine that interprets UI descriptions sent from the backend. In practice, this often means the server delivers a JSON payload that describes screens or components. This JSON defines what elements should appear (text, images, buttons, lists, etc.), their content, and possibly style attributes or layout hints. The mobile app is built with a library of native UI components (views) that correspond to possible element types in the JSON. When the app fetches the JSON, it maps each element to a native view and renders the interface accordingly. The server effectively controls both the data and presentation. This approach is reminiscent of how web browsers work — HTML from the server tells the browser what to render — except here the “HTML” is a custom JSON (or similar format) and the “browser” is our native app. How does this help? It decouples releasing new features from app releases. We can introduce a new promotion banner, change a screen layout, or run A/B tests by adjusting the server response. The next time users open the app (with internet connectivity), they’ll see the updated UI instantly, without updating the app. The client code remains simpler, focusing mainly on rendering logic and native integrations (like navigation or accessing device features), while the server takes on the responsibility of deciding what the UI should contain for each context or user. Let's illustrate with a simple example. Imagine a retail app’s home screen needs to show a “Featured Item Card” as part of a promotional campaign. Traditionally, we would implement this card natively in the app code and release a new version. With SDUI, we can define the card on the server and send it to the app: JSON { "type": "card", "id": "featured-item-123", "elements": [ { "type": "image", "url": "https://example.com/images/promo123.png", "aspectRatio": "16:9" }, { "type": "text", "style": "headline", "content": "Limited Edition Sneakers" }, { "type": "text", "style": "body", "content": "Exclusive launch! Available this week only." }, { "type": "button", "label": "Shop Now", "action": { "type": "deep_link", "target": "myapp://shop/featured/123" } } ], "style": { "padding": 16, "backgroundColor": "#FFF8E1", "cornerRadius": 8 } } In this JSON, the server describes a card component with an image, two text blocks, and a button. The client app knows how to render each element type (image, text, button) using native UI widgets. The layout and content (including text and image URL) come from the server. We could change the title or swap out the image URL in the JSON tomorrow, and every user’s app would reflect the new content immediately. All without a new binary release. Comparing SDUI With Native Development How does server-driven UI compare to the traditional native development approach? In native development, each screen and component is implemented in platform-specific code (SwiftUI/UIKit on iOS, Jetpack Compose/View system on Android, etc.). The client is in full control of the UI, and the server typically only supplies raw data (e.g., a list of products in JSON, but not how they should be displayed). Any change in presentation requires modifying the app’s code. This gives us maximum flexibility to use platform features and fine-tune performance, but it also means any significant UI update goes through the full development and deployment cycle. With SDUI, we trade some of that granular control for agility. The server defines what the UI should look like, but the rendering still happens with native components. That means users get a native look and feel — our JSON example above would result in actual native image views, text views, and buttons, not a web page. Performance can remain close to native since we’re not introducing a heavy abstraction layer; the app is essentially assembling native UI at runtime. However, the client must be generic enough to handle various combinations of components. We often implement a UI engine or framework within the app that knows how to take a JSON like the one above and inflate it into native views. This adds complexity to the app architecture — essentially, part of our app becomes an interpreter of a UI description language. Another consideration is version compatibility. In purely native development, the client and server have a fixed contract (e.g., the server sends data X, the client renders it in pre-defined UI Y). In SDUI, that contract is more fluid, and we must ensure the app can gracefully handle JSON meant for newer versions. For example, if we introduce a new element type "carousel" in the JSON but some users have an older app that doesn’t know about carousels, we need a strategy — perhaps the server avoids sending unsupported components to older app versions, or we build backward compatibility into the client’s SDUI engine. In summary, compared to native development, SDUI offers: Fast UI iterations: We can deploy UI changes like we deploy backend changes, without waiting on app store releases.Consistency: The server can ensure all users (on compatible app versions) see the updated design, reducing the divergence caused by slow adopters of app updates.Reduced client complexity in business logic: The app focuses on rendering and basic interactions, while business logic (what to show when) lives on the server. This can simplify client code for complex, dynamic content. On the flip side, native development still has the edge in certain areas. If an app’s UI rarely changes or demands pixel-perfect, platform-specific design tailored for each OS, the overhead of SDUI might not be worth it. Native apps also work offline out-of-the-box with their built-in UI since the interface is packaged with the app, whereas SDUI apps need to plan for offline behavior (more on that soon). There’s also the matter of tooling and debugging: a native developer can use interface builders and preview tools to craft screens, but debugging an SDUI layout might involve inspecting JSON and logs to understand what went wrong. We have to invest in developer experience for building and testing UI configurations on the server side. Comparing SDUI With Cross-Platform Frameworks Cross-platform frameworks (like React Native, Flutter, and Kotlin Multiplatform Mobile) address a different problem: the duplication of effort when building separate apps for iOS and Android. These frameworks allow us to write common UI code that runs on multiple platforms. React Native uses JavaScript and React to describe UI, which is then bridged to native widgets (or, in some cases, uses its own rendering). Flutter uses its own rendering engine and Dart language to draw UI consistently across platforms. The benefit is a unified codebase and faster development for multiple targets. However, cross-platform apps still typically follow a client-driven UI approach — the app’s packaged code dictates the interface. Comparing cross-platform with SDUI, we find that they can solve complementary issues. Cross-platform is about “build once, run anywhere”, whereas SDUI is about “update anytime from the server”. You could even combine them: for instance, a React Native app could implement a server-driven UI system in JavaScript that takes JSON from the server and renders React Native components. In fact, many cross-platform apps also want the ability to update content dynamically. The key comparisons are: Deployment vs. code sharing: SDUI focuses on decoupling deployment from UI changes. Cross-platform focuses on sharing code between platforms. With SDUI, we still need to implement the rendering engine on each platform (unless using a cross-platform framework underneath), but each platform’s app stays in sync by receiving the same server-driven definitions. Cross-platform frameworks remove the need to implement features twice, but when you need to change the UI, you still have to ship new code (unless the framework supports code push updates).Performance: Modern cross-platform frameworks are quite performant, but they introduce their own layers. For example, Flutter’s engine draws every pixel, and React Native uses a bridge between JavaScript and native. SDUI, by contrast, typically leans on truly native UI components for each platform (the JSON is interpreted by native code), so performance can be as good as native for the rendering, though there is some overhead in parsing JSON and the network round-trip.Flexibility: Cross-platform UIs may sometimes not feel perfectly “at home” on each platform, especially if the framework abstracts away platform-specific UI patterns. SDUI lets each platform render components in a way that matches its native guidelines (since the app code for rendering is platform-specific), while still coordinating the overall look via server. However, SDUI limits how much a local platform developer can tweak a screen’s behavior — the structure comes from server. In cross-platform, a developer could drop down to native code for tricky parts if needed; in SDUI, if a new interaction or component is needed, it likely requires server-side support and possibly updating the app to handle it. In practice, organizations choose one or the other (or both) based on their needs. For example, a team might stick with fully native iOS/Android development but add SDUI capabilities to handle highly dynamic parts of the app. Another team might build the bulk of the app in Flutter but reserve certain sections to be server-driven for marketing content. Importantly, SDUI is not a silver bullet replacement for cross-platform frameworks — it operates at a different layer. In fact, we can think of SDUI as introducing a cross-release contract (the JSON/DSL for UI) rather than a cross-platform codebase. We, as developers, still maintain code for each platform, but that code is largely generic. Cross-platform, conversely, minimizes platform-specific code but doesn’t inherently solve the deployment agility problem. A Concrete Example: "Featured Item Card" To make SDUI more tangible, let's walk through the Featured Item Card example in detail. Suppose our app has a home screen where we want to show a special promotional item to users. In a traditional app, we’d implement a new view class or fragment for this card, define its layout in XML (Android) or SwiftUI, and fetch the data to populate it. With SDUI, we instead add a description of this card to the server’s response for the home screen. When the app requests the home screen layout, the server responds with something like: JSON { "screen": "Home", "sections": [ { "type": "FeaturedItemCard", "data": { "itemId": 123, "title": "Limited Edition Sneakers", "description": "Exclusive launch! This week only.", "imageUrl": "https://example.com/images/promo123.png", "ctaText": "Shop Now", "ctaTarget": "myapp://shop/featured/123" } }, { "type": "ProductGrid", "data": { ... } } ] } In this hypothetical JSON, the home screen consists of multiple sections. The first section is of type "FeaturedItemCard" with associated data for the item to feature, and the second might be a product grid (the rest of the screen). The client app has a registry of component renderers keyed by type. When it sees "FeaturedItemCard", it knows to create a card UI: perhaps an image on top, text below it, and a button. The values for the image URL, text, and button label come from the data field. The ctaTarget might be a deep link or navigation instruction that the app will use to wire the button’s action. The beauty of this setup is that if next month Marketing wants to feature a different item or change the text, we just change the server’s JSON output. If we want to experiment with two different styles of featured cards (say, a larger image vs. a smaller image variant), we could have logic on the server that sends different JSON to different user segments. The app simply renders whatever it’s told, using its existing capabilities. Now, building this FeaturedItemCard in the app still requires some work upfront. We need to ensure the app knows how to render a section of type "FeaturedItemCard". That likely means in our SDUI framework on the client, we have a class or function that takes the data for a FeaturedItemCard and constructs the native UI elements, e.g., an ImageView with the image URL, a TextView or SwiftUI Text for the title, another for description, and a styled Button for the CTA. We might have done this already if FeaturedItemCard was a planned component; if not, adding a completely new type of component would require an app update to teach the client about it. This is a crucial point: SDUI excels when using a known set of components that can be reconfigured server-side, but introducing truly new UI paradigms can still require coordinated client-server releases. In our example, as long as “FeaturedItemCard” was built into version 1.0 of the app, we can send any number of different FeaturedItemCard instances with various data. But if we suddenly want a “FancyCarousel” and the app has no carousel component built-in, we have to update the app to add that capability. Our example also hints at nesting and composition. The JSON could describe entire screens (like the Home screen) composed of sections, or it could be more granular (maybe the server sends just the FeaturedItemCard JSON, and the app knows where to insert it). The design of the JSON schema for UI is a big part of SDUI implementation. Some teams use a layout tree structure (hierarchical JSON of containers and components), while others might use a flat list of components per screen with implicit layout logic. Regardless of the approach, the client needs to interpret it and transform it into a user interface. Handling Offline Support: Client-Side Caching Strategy One of the biggest challenges with a server-driven approach is offline support. If the app’s UI is coming from the server at runtime, what happens when the user has no internet connection or when the server is slow to respond? We certainly don’t want to show a blank screen or a loading spinner indefinitely. To mitigate this, we need a caching strategy on the client side. The idea is to cache the JSON (or whatever UI description) from the last successful fetch so that we can fall back to it when needed. A common approach is to use a small SQLite database on the device to store cached UI data. SQLite offers a lightweight, file-based database that’s perfect for mobile apps. By caching the server responses, the app can quickly load the last known UI state for a given screen without network access. For example, we might set up a SQLite table to store UI layouts by a key, such as the screen name or endpoint URL: SQLite CREATE TABLE IF NOT EXISTS ui_layout_cache ( screen_id TEXT PRIMARY KEY, layout_json TEXT NOT NULL, last_updated TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); In this schema, whenever the app fetches a UI description from the server (say for the “Home” screen or for a “FeaturedItemCard” module), it will store the JSON text in the ui_layout_cache table, keyed by an identifier (like "Home"). The last_updated timestamp helps us know how fresh the cache is. If the user opens the app offline, we can query this table for the layout of the Home screen. If found, we parse that JSON and render the UI as usual. This way, the user at least sees the last known interface (and possibly data content) instead of an error. The Future Potential: AI-Generated and AI-Optimized UIs Looking ahead, one exciting avenue for server-driven UI is the incorporation of AI in the UI generation process. Since SDUI already centralizes UI decisions on the server, it opens the door for using machine learning models to decide what UI to present, effectively AI-generated UIs. What could this look like in practice? Imagine a scenario where we want to personalize the app’s interface for each user to maximize engagement or accessibility. With SDUI, we have the flexibility to serve different layouts to different users. AI could be employed to analyze user behavior and preferences, and then dynamically select which components or layout variations to show. For example, if a user tends to ignore a certain type of banner but engages more with lists, an AI model might decide to send a more list-focused home screen JSON to that user. This goes beyond A/B testing into the realm of continuous optimization per user or per segment, driven by algorithms. Another possibility is using generative AI to actually create new UI component configurations. There are already early experiments in using AI to generate code or design assets. One could envision an AI system that, given high-level goals (like “promote item X to user Y in a non-intrusive way”), generates a JSON layout snippet for a new card or modal, which the app then renders. The AI could even iterate and test different generated layouts, measure user interactions (with proper caution to user experience), and gradually converge on optimal designs. In essence, the app’s UI becomes data-driven in real-time, not just server-driven but AI-driven on the server. AI-optimized UIs might also assist in responsive design across device types. If our app runs on phones and tablets, an AI model could adjust the layout JSON for the larger screen, maybe adding additional elements if space allows, or reordering content based on usage patterns. While traditional responsive design can handle simple resizing, an AI could learn from user engagement to decide, for instance, that tablet users prefer a two-column layout for a particular screen, and then automatically deliver that via SDUI. We should note that these ideas are still on the frontier. Implementing AI-driven UI decisions requires careful consideration of user trust and consistency. We wouldn’t want the app UI to become erratic or unpredictable. Likely, AI suggestions would be filtered through design constraints to ensure they align with the brand and usability standards. Server-driven UI provides the delivery mechanism for such dynamic changes, and AI provides the decision mechanism. Together, they could enable a level of UI personalization and optimization that was previously hard to achieve in native apps. In summary, the future may see SDUI and AI converge to produce apps that tailor themselves to each user and context, all without constant human micromanagement. We would still keep humans in the loop for oversight, but much of the heavy lifting in UI optimization could be offloaded to intelligent systems. This concept aligns well with the SDUI philosophy of the client being a flexible renderer, as that flexibility can now be exploited by advanced decision-making algorithms on the server.

A grand gala was being held at the Jade Palace. The Furious Five were preparing, and Po was helping his father, Mr. Ping, in the kitchen. But as always, Po had questions. Po (curious): "Dad, how do you always make the perfect noodle soup no matter what the ingredients are?" Mr. Ping (smiling wisely): "Ah, my boy, that’s because I follow the secret recipe—a fixed template!" Mr. Ping Reveals the Template Method Pattern Mr. Ping: "Po, the Template Method Pattern is like my noodle recipe. The skeleton of the cooking steps stays the same, but the ingredients and spices can vary!" Po: "Wait, you mean like... every dish has a beginning, middle, and end—but I can change what goes inside?" Mr. Ping: "Exactly! The fixed steps are defined in a base class, but subclasses—or in our case, specific dishes—override the variable parts." Traditional Template Method in Java (Classic OOP) Java public abstract class DishRecipe { // Template method public final void cookDish() { boilWater(); addIngredients(); addSpices(); serve(); } private void boilWater() { System.out.println("Boiling water..."); } protected abstract void addIngredients(); protected abstract void addSpices(); private void serve() { System.out.println("Serving hot!"); } } class NoodleSoup extends DishRecipe { protected void addIngredients() { System.out.println("Adding noodles, veggies, and tofu."); } protected void addSpices() { System.out.println("Adding soy sauce and pepper."); } } class DumplingSoup extends DishRecipe { protected void addIngredients() { System.out.println("Adding dumplings and bok choy."); } protected void addSpices() { System.out.println("Adding garlic and sesame oil."); } } public class TraditionalCookingMain { public static void main(String[] args) { DishRecipe noodle = new NoodleSoup(); noodle.cookDish(); System.out.println("\n---\n"); DishRecipe dumpling = new DumplingSoup(); dumpling.cookDish(); } } //Output Boiling water... Adding noodles, veggies, and tofu. Adding soy sauce and pepper. Serving hot! --- Boiling water... Adding dumplings and bok choy. Adding garlic and sesame oil. Serving hot! Po: "Whoa! So each dish keeps the boiling and serving, but mixes up the center part. Just like kung fu forms!" Functional Template Method Style Po: "Dad, can I make it more... functional?" Mr. Ping: "Yes, my son. We now wield the power of higher-order functions." Java import java.util.function.Consumer; public class FunctionalTemplate { public static <T> void prepareDish(T dishName, Runnable boil, Consumer<T> addIngredients, Consumer<T> addSpices, Runnable serve) { boil.run(); addIngredients.accept(dishName); addSpices.accept(dishName); serve.run(); } public static void main(String[] args) { prepareDish("Noodle Soup", () -> System.out.println("Boiling water..."), dish -> System.out.println("Adding noodles, veggies, and tofu to " + dish), dish -> System.out.println("Adding soy sauce and pepper to " + dish), () -> System.out.println("Serving hot!") ); prepareDish("Dumpling Soup", () -> System.out.println("Boiling water..."), dish -> System.out.println("Adding dumplings and bok choy to " + dish), dish -> System.out.println("Adding garlic and sesame oil to " + dish), () -> System.out.println("Serving hot!") ); } } Po: "Look, dad! Now we can cook anything, as long as we plug in the steps! It's like building recipes with Lego blocks!" Mr. Ping (beaming): "Ah, my son. You are now a chef who understands both structure and flavor." Real-World Use Case – Coffee Brewing Machines Po: “Dad, Now I want to build the perfect coffee-making machine, just like our noodle soup recipe!” Mr. Ping: “Ah, coffee, the elixir of monks and night-coders! Use the same template method wisdom, my son.” Step-by-Step Template – Java OOP Coffee Brewer Java abstract class CoffeeMachine { // Template Method public final void brewCoffee() { boilWater(); addCoffeeBeans(); brew(); pourInCup(); } private void boilWater() { System.out.println("Boiling water..."); } protected abstract void addCoffeeBeans(); protected abstract void brew(); private void pourInCup() { System.out.println("Pouring into cup."); } } class EspressoMachine extends CoffeeMachine { protected void addCoffeeBeans() { System.out.println("Adding finely ground espresso beans."); } protected void brew() { System.out.println("Brewing espresso under high pressure."); } } class DripCoffeeMachine extends CoffeeMachine { protected void addCoffeeBeans() { System.out.println("Adding medium ground coffee."); } protected void brew() { System.out.println("Dripping hot water through the grounds."); } } public class CoffeeMain { public static void main(String[] args) { CoffeeMachine espresso = new EspressoMachine(); espresso.brewCoffee(); System.out.println("\n---\n"); CoffeeMachine drip = new DripCoffeeMachine(); drip.brewCoffee(); } } //Ouput Boiling water... Adding finely ground espresso beans. Brewing espresso under high pressure. Pouring into cup. --- Boiling water... Adding medium ground coffee. Dripping hot water through the grounds. Pouring into cup. Functional and Generic Coffee Brewing (Higher-Order Zen) Po, feeling enlightened, says: Po: “Dad! What if I want to make Green Tea or Hot Chocolate, too?” Mr. Ping (smirking): “Ahhh... Time to use the Generic Template of Harmony™!” Functional Java Template for Any Beverage Java import java.util.function.Consumer; public class BeverageBrewer { public static <T> void brew(T name, Runnable boil, Consumer<T> addIngredients, Consumer<T> brewMethod, Runnable pour) { boil.run(); addIngredients.accept(name); brewMethod.accept(name); pour.run(); } public static void main(String[] args) { brew("Espresso", () -> System.out.println("Boiling water..."), drink -> System.out.println("Adding espresso grounds to " + drink), drink -> System.out.println("Brewing under pressure for " + drink), () -> System.out.println("Pouring into espresso cup.") ); System.out.println("\n---\n"); brew("Green Tea", () -> System.out.println("Boiling water..."), drink -> System.out.println("Adding green tea leaves to " + drink), drink -> System.out.println("Steeping " + drink + " gently."), () -> System.out.println("Pouring into tea cup.") ); } } //Output Boiling water... Adding espresso grounds to Espresso Brewing under pressure for Espresso Pouring into espresso cup. --- Boiling water... Adding green tea leaves to Green Tea Steeping Green Tea gently. Pouring into tea cup. Mr. Ping’s Brewing Wisdom “In code as in cooking, keep your recipe fixed… but let your ingredients dance.” Template Pattern gives you structure.Higher-order functions give you flexibility.Use both, and your code becomes as tasty as dumplings dipped in wisdom! Mr. Ping: "Po, a great chef doesn't just follow steps. He defines the structure—but lets each ingredient bring its own soul." Po: "And I shall pass down the Template protocol to my children’s children’s children!" Also read... Part 1 – Kung Fu Code: Master Shifu Teaches Strategy Pattern to Po – The Functional WayPart 2 – Code of Shadows: Master Shifu and Po Use Functional Java to Solve the Decorator Pattern MysteryPart 3 – Kung Fu Commands: Shifu Teaches Po the Command Pattern with Java Functional Interfaces