Data Resources

DZone's Featured Data Resources

A Low-Latency Routing Pattern for Multiple Small Language Models

By Akhil Madineni

A multi-SLM platform creates value only when specialization does not introduce a new latency tier. Small language models are inexpensive enough to dedicate to focused work such as extraction, code handling, safety filtering, or short-form reasoning, but that advantage disappears if model selection itself becomes expensive. Research on LLM routing shows that query difficulty varies enough for model choice to materially affect efficiency and quality, and modern serving stacks expose enough control over routing, batching, and cache locality to turn that insight into an operational design rather than an academic one. In practice, the routing layer has to behave like a tiny data-plane decision engine, not like another inference hop. Why Multiple SLMs Need Routing A single small model rarely gives the best latency-quality trade-off for every prompt type. Short structured requests, such as JSON extraction and classification, differ sharply from code repair, and both differ again from prompts that need broader reasoning. RouteLLM describes routing as assigning simpler queries to weaker models and reserving stronger models for harder cases, while FrugalGPT reports that a learned cascade can preserve strong-model quality with very large cost reductions. Although those papers evaluate broader LLM portfolios, the underlying lesson transfers cleanly to a fleet of small specialized models: heterogeneity in request shape makes heterogeneity in model choice economically and operationally rational. That conclusion rules out a router that behaves like another generative model call. RouteLLM explicitly treats effective routing as a pre-decision that minimizes cost and latency relative to broader multi-model execution, which means the dominant path should remain inside in-memory feature extraction and lookup. Prompt length, requested output shape, language, code markers, safety category, session identity, and prior cache affinity are all signals that can be computed before any model is invoked. A practical design target is to keep that first decision under a millisecond, so its cost remains far below prefill and decode work. The moment the main path depends on an additional model inference, the latency budget starts competing with the very SLM call it is supposed to optimize. Keeping the Decision Path Short The cleanest design is a two-stage router. The first stage is deterministic and resolves obvious cases immediately. A short request demanding strict JSON can go to an extraction model. A prompt containing fenced code, compiler errors, or repository paths can go to a code model. A safety-sensitive request can be pinned to a policy model. Only when simple predicates fail to produce a confident mapping should the second stage run, and that second stage should be a lightweight complexity scorer rather than another generator. Ray Serve’s request-routing API is built around this kind of custom replica selection, and its FIFO mixin is specifically intended for algorithms that can route requests as soon as they arrive without waiting for content-heavy processing. That is the right shape for an ultra-low-latency router: deterministic fast path first, optional scorer second. A routing metadata object makes that design practical because it compresses request interpretation into cheap primitives: Java record RoutingContext( int tokenCount, boolean codeRequest, boolean structuredOutput, String language, boolean repeatedPrefix, double complexityScore ) {} This record is deliberately plain. Primitive fields are cheap to serialize, cheap to log, and easy to replay during debugging. That choice aligns with PyTorch and vLLM production notes on disaggregated serving, where complex metadata objects in scheduler paths increased serialization cost and hurt inter-token behavior, and it fits the general shape of request routers that repeatedly rank candidate replicas under load. The complexityScore field should therefore come from a compact classifier or calibrated heuristic trained offline on task outcomes, escalation rates, or preference labels, not from a runtime SLM call. The router’s intelligence belongs in the thresholds and features, not in an extra generation step. The routing function should then read like admission control rather than orchestration: Java ModelTarget route(RoutingContext ctx) { if (ctx.structuredOutput() && ctx.tokenCount() < 800) return ModelTarget.EXTRACTION_SLM; if (ctx.codeRequest()) return ModelTarget.CODE_SLM; if (ctx.complexityScore() > 0.72) return ModelTarget.REASONING_SLM; if (ctx.repeatedPrefix()) return ModelTarget.GENERAL_SLM_CACHE_HOT; return ModelTarget.GENERAL_SLM; } The important detail is ordering. The cheapest predicates run first, the optional scorer appears only after clear task signals have been checked, and cache affinity refines the generic path instead of overriding obvious specialization. That mirrors how high-performance request routers rank candidates and then filter out replicas that are already saturated. Thresholds should be calibrated from observed latency and task-success data, but the architectural rule is stable: most traffic should leave the router with a decision produced entirely from fields already in memory. Making Selection Cache-Aware Cache-aware selection is where routing often starts to produce visible latency gains. vLLM’s automatic prefix caching reuses KV cache from earlier queries when a new request shares the same prefix, allowing shared prompt computation to be skipped, and its design notes describe prefix caching as close to a free lunch because it avoids redundant work without changing outputs. SGLang reaches a similar result with RadixAttention, which keeps reusable KV state in a radix tree, adds LRU eviction, and applies cache-aware scheduling to improve hit rate while introducing only negligible overhead when no cache hit occurs. That combination matters because a fast model on a warm prefix can easily outperform a nominally better model on a cold path. Routing without cache awareness, therefore, leaves substantial latency savings on the table. That is why a field such as repeatedPrefix, promptFamilyId, or session hash belongs in the routing context. Ray Serve exposes locality-aware and multiplex-aware helpers so that requests can prefer nearby replicas or replicas that already hold the relevant model, and Meta’s PyTorch and vLLM production write-up reports that sticky routing of the same session to the same prefill host significantly boosts prefix-cache hit rate, reaching 40% to 50% hit rate in the described deployment. The practical lesson is broader than that specific architecture. Similar prompt families should be steered toward the same warm replicas whenever possible, even if a purely load-balanced policy would have spread them evenly. Equal distribution is not the same thing as minimal latency once KV reuse becomes available. Keeping the System Fast in Production Once the routing logic is correct, the queueing policy and replica shape become the next sources of latency. Triton documents that dynamic batching combines requests to maximize throughput and allows bounded queue delay, while concurrent model execution and instance groups allow multiple copies of the same model to run in parallel on selected devices. That argues for selective rather than universal batching. Short extraction or moderation SLMs often benefit from aggressive batching because their service time is small and predictable, while interactive reasoning models need tighter queue-delay bounds to prevent batching from inflating p95 latency. Replica placement matters as well. Heavy or frequently chosen models deserve more parallel instances, and cold-start penalties should be reduced through explicit warmup, since Triton notes that model warmup can prevent the slow initial inferences seen before a model is fully initialized. Backpressure and observability complete the design. Ray Serve supports bounded queues and load shedding through max_queued_requests, and its autoscaling guidance ties lower ongoing-request targets to tighter latency objectives. Ray Serve LLM also exposes request latency, throughput, TTFT, and TPOT, while Triton exposes Prometheus metrics for GPU and request behavior. Those signals should be segmented by routed model, decision path, cache-hit class, and warm versus cold replica so that routing regressions become visible before they surface as user-facing tail latency. Without route-level telemetry, an apparently accurate router can quietly push traffic onto cold replicas, oversized queues, or cache-miss-heavy paths. In a low-latency SLM system, observability is not just for debugging. It is the only reliable way to keep routing policy aligned with actual serving behavior. Conclusion An ultra-low-latency routing layer for multiple SLMs is best treated as a serving primitive rather than as a separate intelligence feature. The strongest design keeps most requests on a deterministic first stage, invokes a lightweight complexity scorer only for ambiguous prompts, represents route state with compact metadata, and treats prefix locality as a first-class selection signal. Around that core, warm replicas, selective batching, bounded queues, and route-level observability determine whether specialization actually improves latency or merely rearranges it. When routing is cheaper than a single token step and cache locality is preserved instead of ignored, a multi-SLM system stops looking like a collection of models and starts behaving like a disciplined low-latency inference fabric. More

A Fully Self‑Contained Text Embedding Service in C#

By Mangesh Walimbe

Modern semantic search, retrieval-augmented generation (RAG) pipelines, and large-scale recommendation models heavily rely on embeddings — transformations of natural language text into dense numeric representations called vectors. These embeddings position semantically related text in nearby regions of vector space. It enables similarity computation through distant metrices such as Cosine similarity or Euclidean distance. Cloud-hosted services like OpenAI has text-embedding-ada-002 provide high-quality vector encodings. But it comes with API keys, network latency, and per-token usage costs. In contrast, LocalEmbeddingService does all the computation within hosted process, no GPUs, no outbound requests, no model files to manage. The method it uses is called the hashing trick (or feature hashing). The same algorithm is implemented in scikit-learn’s HashingVectorizer. 1. Contract: IEmbeddingService C# public class LocalEmbeddingService : IEmbeddingService { public int Dimensions => 512; The service creates 512-dimensional float vectors. This is intentional. It is large enough to capture document semantics yet small enough for in-memory dot-product similarity searches across millions of vectors. These dimensions can be increased to 1024 or 2048, but will require additional GPU and memory usage. 2. Stop Words C# private static readonly HashSet<string> StopWords = new(StopAnalyzer.ENGLISH_STOP_WORDS_SET, StringComparer.OrdinalIgnoreCase); Stop words are common high-frequency words like “and”, “the”, “is”, and “while”. It does contain minimal/no semantic information, but can heavily influence vectorized output if these are not filtered. In the above code, Lucene.NET’s nuget package is used, instead of hardcoding, which has a predefined set StopAnalyzer.ENGLISH_STOP_WORDS_SET. It is well curated and validated. The set is wrapped in HashSet<string> with OrdinalIgnoreCase which provides fast case-insensitive lookup without any extra allocation at query time. 3. Text Cleaning — Tokenization C# private static Dictionary<string, int> Tokenize(string text) { var freq = new Dictionary<string, int>(StringComparer.OrdinalIgnoreCase); var tokens = text .ToLowerInvariant() .Split(new[] { ' ', '\t', '\n', '\r', ',', '.', '!', '?', ';', ':', '"', '\'', '(', ')', '[', ']', '{', '}', '-', '_', '/', '\\' }, StringSplitOptions.RemoveEmptyEntries) .Where(t => t.Length > 2 && !StopWords.Contains(t)); foreach (var token in tokens) freq[token] = freq.GetValueOrDefault(token) + 1; return freq; } Tokenization is the very first step of text cleaning. Each word has to go through it. It has 3 main things. Lowercasing: It keeps all the words in lower case. “System” and “system” have the same meaning.Split based on delimiter/punctuation: Each delimiter/punctuation is considered as a word boundary. “top-of-the-line” will become [“top”, “line”] after splitting and removing stop words.Filtering: If the tokens are less than 3 characters, then they will be skipped with stop words. After tokenization, it gives a term-frequency map like { "compute": 2, "learn": 3, "embedding": 1, … }. 4. Hashing Trick/Feature Hashing The core challenge here is the size of real-world vocabularies. There are millions of distinct terms. It makes it almost impossible to allocate a separate vector dimension per term/token. Hashing tricks solve this problem by hashing tokens directly into a bounded index range via a hash function. It will eliminate the need to store a vocabulary. C# private static int StableBucket(string token, int size) { unchecked { uint hash = 2166136261u; // FNV offset basis foreach (char c in token) { hash ^= (byte)c; hash *= 16777619u; // FNV prime } return (int)(hash % (uint)size); } } Here FNV-1a (Fowler–Noll–Vo) hash function is used. It is a lightweight, non-cryptographic hash ideal for short strings with excellent bit distribution. It uses two canonical constants. FNV offset basis: Decimal: 2166136261, Hex: 0x811C9DC5FNV prime: Decimal: 16777619, Hex: 0x01000193 Each character is processed by XOR-ing the current hash with the character’s byte value. Then it is multiplied by FNV prime. The XOR-then-Multiply order ensures every byte influences 32 bits, improving avalanche behavior for short tokens like English words. Here .NET’s string.GetHashCode() is not useful because it randomizes per process run against hash flooding attacks. The StableBucket is required to return same bucket indices across every run for deterministic 32-bit results. The use of unchecked in C# ensures overflow checking for 32-bit integer semantics. 5. Log-Based TF Normalization C# float weight = MathF.Log(1f + count); Term frequency does not scale linearly with semantic importance. For example, a word/term that appears 10 times in a document is not actually 10 times more important that the term appears once. When the log log(1 + count) is applied, it compresses the raw frequency. The table below shows how this log-based frequency works. This ensures that no single repeated term disproportionately shapes the embedding, the same reasoning behind TF-IDF in traditional information retrieval systems. 6. Trigram Features for Morphology Capture C# if (token.Length >= 4) { for (int i = 0; i <= token.Length - 3; i++) { string trigram = token[i..(i + 3)]; int trigramBucket = StableBucket(trigram, Dimensions); vector[trigramBucket] += weight * 0.5f; } } Whole world hashing can produce hard edge cases for terms like “play”, “player”, “playing”. These terms are treated as separate features and land in different buckets. Trigrams help to reconnect them and smooth out these gaps. Here are trigrams for “playing” and “player”. C# playing - pla, lay, ayi, yin, ing player - pla, lay, aye, yer Here, common trigrams like pla and lay cause both terms to accumulate weight in some of the same hashed buckets, which pulls their vectors closer in embedding space. The half weight (o.5f ) ensures that trigram features do not dominate the whole-word signal. 7. L2 Vector Normalization — Cosine Similarity via Direct Dot Products C# private static void NormalizeL2(float[] vector) { float magnitude = 0f; foreach (float v in vector) magnitude += v * v; magnitude = MathF.Sqrt(magnitude); if (magnitude > 0f) for (int i = 0; i < vector.Length; i++) vector[i] /= magnitude; } Once all token and trigram weights have been applied, the resulting vector is normalized so that its Euclidean length equals 1. This normalization enables a key mathematical identity: C# cosine_similarity(a, b) = a · b when ‖a‖ = ‖b‖ = 1 When vectors are already L2-normalized, the cosine similarity is evaluated using the raw dot product operation, eliminating the need for any division. 8. Utility: GetTopTokenWeights C# public Dictionary<string, float> GetTopTokenWeights(string text, int topN = 10) { var tokenFreq = Tokenize(text); return tokenFreq .Select(kv => new { Token = kv.Key, Weight = MathF.Log(1f + kv.Value) }) .OrderByDescending(x => x.Weight) .Take(topN) .ToDictionary(x => x.Token, x => x.Weight); } This diagnosis method highlights the tokens that contributed most to the final embeddings. It provides critical insight into why two documents achieve high similarity scores and confirms that the stop word removal and tokenization are working as expected. Limitations and Production Enhancements This service is fully deterministic, fast, and requires zero supporting infrastructure. It performs well for vocabulary-driven similarity — cases where documents share the same vocabulary. It does not encode semantic relationships. For example, “car” and “sedan” will end up in separate buckets and will not have the same similarity score. For production-grade semantic search, LocalEmbeddingService can be replaced with either OpenAI or a local ONNX sentence transformer. The shared IEmbeddingService interface by both implementations ensures that no code change is required for any components like API Controllers, vector index, or retrieval logic. Project repository: TextEmbeddingService More

Fine-Tuning LLMs at Scale With Databricks MLflow and Spark

By Jubin Abhishek Soni

CORE

Wayland Compositor Debugging in C++: Hunting Null Pointer Crashes in the Display Stack

By Rajasekhar sunkara

Data Pipeline Observability: Why Your AI Model Fails in Production

By Abhilash Rao Mesala

Building High‑Precision Vector Search for Document Retrieval on Databricks

For years, search technology meant one thing: type in a keyword, and the system goes hunting for an exact match. That works fine for product SKUs or error codes, but it falls apart the moment someone asks a real question. If your knowledge base is full of manuals, support tickets, transcripts, and reports, a person searching for "why does the machine shut down during startup" shouldn't have to guess the exact phrase the original author used. This is the gap that vector search closes. Instead of matching words, it matches meaning. And on Databricks, building this kind of system is more accessible than most teams expect, once you understand the moving pieces. Why Vector Databases Work Differently A vector database doesn't store text the way a traditional database does. It stores text as numbers, specifically, as long lists of numerical values that represent the meaning of a piece of content. Two sentences that say the same thing in different words end up with similar number patterns, even if they don't share a single word in common. This unlocks three distinct ways of searching: Similarity search finds content that's conceptually related, even when the wording is completely different. Hybrid search blends that conceptual matching with traditional keyword scoring, so you get the best of both worlds. Full-text search sticks to exact matches, which still matters when precision is non-negotiable. Together, these give developers the tools to build something that feels less like a search box and more like a colleague who actually understands what you're asking. Getting Your Data Ready Before any of this works, your data needs to be in the right shape. On Databricks, that means your source table needs Change Data Feed turned on. Think of this as a way for the vector index to "listen" for changes, so when documents get updated, added, or removed, the index stays in sync automatically rather than going stale. You'll also need a unique identifier for every row. This becomes the primary key that ties each chunk of text back to its source, which matters later when you're filtering or tracing results back to the original document. Turning Text Into Embeddings Embeddings are the numerical fingerprints mentioned earlier, the representations that let the system compare meaning instead of matching strings. Databricks gives you two paths here. With managed embeddings, Databricks handles the entire process: it generates the embeddings and keeps them updated as your data changes. With manual embeddings, you generate them yourself using an external tool and store the results in a column. For the vast majority of projects, managed embeddings are the easier and more reliable choice. There's less to maintain, and compatibility with the platform is guaranteed out of the box. One question that comes up constantly: what does it mean when someone says an embedding has 1,024 dimensions? It simply means each chunk of text is represented by 1,024 numbers. That number isn't arbitrary; it's baked into whichever embedding model you choose, such as GTE-large. If you want a different dimensionality, you'd need to switch models entirely; it's not a setting you can tweak independently. Building the Index Once your embeddings are in place, you create the actual vector search index. Databricks gives you two routes: the SDK, using the databricks-vectorsearch library for programmatic, repeatable setups, or the UI, which walks you through configuration visually. A few decisions matter most here. The index type determines whether you're doing pure semantic search or hybrid search; for most real-world use cases, hybrid is the safer default since it catches both conceptual matches and exact terminology. The embedding model, like databricks-gte-large-en, determines how your text gets converted into vectors. And the sync mode controls how fresh your index stays: continuous sync keeps things updated automatically, while triggered sync gives you manual control over when refreshes happen. Choosing the Right Search Method With the index built, you have three retrieval modes to choose from, and picking the right one depends entirely on what your users are asking. Similarity search shines when people ask natural-language questions or when the same concept might be described using different terminology across documents. Hybrid search becomes valuable when domain-specific terms carry real weight, think compliance codes or technical standards like ISO 13849-1, where an exact match matters just as much as conceptual relevance. Full-text search is your fallback when precision trumps everything else, and you need exact keyword hits, no exceptions. Don't Skip Metadata Filtering Here's a piece of advice that's easy to overlook: don't make your search work harder than it needs to. If a user only cares about PDFs from the last quarter, let the system know that upfront. Filtering by document path, page number ranges, or document type narrows the search space before the heavy lifting even starts. The result is faster queries and more relevant results, because the system isn't wasting effort sifting through content that was never going to be useful anyway. When Re-Ranking Earns Its Keep Sometimes the top results from a semantic search are technically "close" in meaning but miss the point of the question. That's where re-ranking comes in, a second pass that re-scores your top candidates using something more sophisticated, like a cross-encoder or an LLM. This extra step is worth the computational cost when queries are nuanced, when domain context really matters, or when the stakes for getting the right answer are high. It's not something you need everywhere, but used selectively, it can be the difference between a good answer and the right one. A Few Practical Tips A handful of best practices can save you headaches down the road. Don't over-invest in embedding dimensionality. If a smaller model performs nearly as well as a larger one, take the smaller one and enjoy the lower latency. Keep your num_results parameter reasonable; pulling back 10 to 100 results is usually plenty, and larger sets just slow things down. Match your endpoint SKU to your scale; standard tiers work fine under roughly 2 million vectors, while storage-optimized tiers make sense beyond that. And lean on metadata filters wherever possible; they're one of the simplest ways to boost both speed and relevance. The Bigger Picture Vector search isn't just a buzzword bolted onto a database. It's the connective tissue between how humans naturally ask questions and how systems find answers. Get the fundamentals right- solid embeddings, a well-configured index, smart filtering, and selective re-ranking- and you're not just building a search feature. You're building something that genuinely understands what people are looking for.

By Ramesh Bellamkonda

Who Owns the Data Stack?: How AI Is Reshaping Ownership, Architecture, and Accountability Across Teams

Editor’s Note: The following is an article written for and published in DZone’s 2026 Trend Report, Cognitive Databases, Intelligent Data: Unified Infrastructure for Vector Search, AI-Optimized Queries, and Hybrid Workloads. For years, some of us have argued that the data stack is part of the product and should be engineered like the application layer: as code and as a service. The market matured toward it, and the data mesh has been the clearest recent expression. AI has eclipsed those debates and settled the matter. The data stack is now product-facing, shaping what users see, what AI answers, and which automated decisions and workflows fire. That makes one question unavoidable: When an answer depends on data across many systems and teams, who is accountable for accuracy? An AI answer is assembled at request time from corporate data. The data stack is inside the response. AI Turns Data Infrastructure Into Product Behavior AI makes the data stack part of product behavior, but raw infrastructure should not leak into the product. The goal is to abstract the stack behind durable, governed interfaces. An AI feature should consume meaning, relationships, permissions, and context. Following data mesh and data contracts, the API layer has to evolve from returning data to exposing capabilities. A consumer, including an AI model, should depend on a contract that carries: Metadata – origin, lineage, meaningQuality – freshness, completeness, confidenceRelationships – how entities compose and traverseSecurity – authorization applied consistently across operational, analytical, and vector stores When meaning lives in the contract, infrastructure becomes interchangeable, and a misbehaving AI feature is no longer an opaque failure — it’s a question with an owner. Where Ownership Breaks First Ownership does not break at the edges of systems but much earlier, in how the organization is designed. Most technology organizations still distribute teams around components and technical specialization: applications, databases, pipelines, governance, indexing, and analytics. Each team owns its layer, though no one owns the end-to-end meaning of the data. That worked when data only fed analytics. It fails in AI-native products, where data is product behavior, and the two lifecycles are inseparable. AI composes its behavior across every layer at once, inheriting each inconsistency in semantics, freshness, permissions, and relationships. So this is not a handoff problem; it is a Conway’s Law problem. Architecture mirrors the organization, and AI makes the organizational seams visible to the user. Platform teams remain essential for shared abstractions, governance primitives, and standards. But product teams need to own both their features and APIs and their data end to end: its lifecycle, meaning, quality, and governance. Splitting teams by technical layer scatters one business entity across many disconnected owners, and AI inherits that fragmentation. AI-native organizations give product teams end-to-end ownership of the data, with platform teams providing shared standards. Accountability Follows Product Behavior When data only fed dashboards, accountability could stay narrow: Did the pipeline run, and did the report match? AI moves that boundary. Once retrieval, copilots, and agents start making decisions and generating answers from data, a correct pipeline, a healthy index, and a valid access policy still don’t guarantee a correct user-facing result. Accountability can’t be pinned to technical layers. It has to follow the behavior the user experiences. The product team that owns an AI capability is responsible for the end-to-end correctness, freshness, explainability, and safety of the data behind it. Its job is to own the contract that defines what the AI may know and retrieve. Platform teams provide the standardized primitives that make this accountability structure possible: semantic contracts, lineage, quality signals, access enforcement, observability, and governance-aware retrieval. The question shifts from “which team owns this layer?” to “which product team owns this behavior, and which platform capabilities guarantee it?” In AI-native systems, accountability rests with the team that owns the behavior, not the system that happened to fail. Table: Accountability Differences Between the Layer-by-Layer and AI-Native Models arealayer-by-layerai-nativeSource of truthEach system decides locallyThe product team owns the authoritative semantic contractQualityThe data team checks pipelinesThe product team owns user-facing correctness; the platform provides quality signalsRetrievalThe platform team owns indexes as infrastructureA governed product capability with explicit SLOsAccessThe security team owns policies separatelyEnforced consistently across product, data, and AI layersIncidentsRouted to whichever layer failedThe product team leads; the platform, data, and security teams support as capability owners Architecture Choices Are Also Operating Model Choices Architecture decisions also decide how an organization governs and evolves meaning. AI-native systems raise the stakes here because copilots and agents consume meaning — entities, relationships, metrics, and permissions — rather than tables. Semantic consistency becomes part of how the product behaves. No central team can own the meaning of every domain, so meaning has to live close to the domain that owns the capability. But decentralization alone backfires: Without platform-enforced standards, the old central bottleneck just turns into semantic fragmentation, with every domain exposing its own definitions and contracts. The fix is to split ownership cleanly: Domains own the meaningPlatform teams own the contracts that keep it consistent Underneath, storage and processing keep churning. What actually lasts is whether stable abstractions (e.g., “employee,” “payroll,” “entitlement”) survive above them. The principle is simple: Infrastructure should be replaceable, and meaning should not. So the real operating-model choice comes down to who owns meaning, and who keeps it consistent. Shared Data Contracts Make Accountability Concrete If organizational fragmentation is the root problem, contracts make ownership explicit. A classic data contract is necessary but insufficient. Schema validation catches a renamed column, but it misses semantic drift, stale meaning, or a changed business definition. Those failures don’t break a build. They break behavior. The contract has to grow from schema into semantics, carrying meaning, lineage, quality, and authorization. Crucially, it abstracts the capability and meaning a domain exposes, not the storage format underneath, so it behaves the same whether the source is a table, a document, an event, or an embedding. That makes the data contract both a producer-to-consumer check and a runtime semantic interface that retrieval, copilots, and agents all consume. Its real value is relocating accountability to the source so drift surfaces in the producing domain while context stays local, which accelerates interoperability rather than centralizing control. Governance Has to Travel With the Data Traditional governance sat beside the data in the form of periodic reviews, approvals, and access checks. AI breaks that model. Data now moves continuously through pipelines, caches, embeddings, indexes, and agents, recomposed at runtime faster than any review can observe. Governance must be part of the execution model itself. Governance travels with meaning, not storage. An embedding holds no raw rows yet reveals sensitive meaning, so policy must follow the semantic classification. The gap is sharpest in authorization. Identity systems stop at the API boundary, and AI doesn’t preserve security boundaries on its own, which turns every embedding, cache, and retrieval step into a new one to defend. Governance therefore becomes a runtime capability that decides what AI may retrieve, infer, expose, and act on. Solving that calls for composable, declarative governance primitives embedded in the platform so auditability becomes a property of the system rather than the outcome of a project. Accountability Gaps That Slow AI Data Work The real cost of fragmented accountability is the constant drag on every data-powered capability. Friction is never neutral, so when teams can’t trust the platform’s freshness, semantics, or governance, they route around it and build their own, resulting in shadow pipelines, local indexes, and duplicated transformations. Each workaround makes sense locally even as it corrodes the whole, fragmenting governance and eroding trust in the very platform it was meant to replace. And piling on more central control only hides the problem — the fragmentation just migrates into those shadow systems. So the deeper gap was missing platform contracts. What Clear Ownership Looks Instead of adding more teams on more layers, clear ownership means aligning accountability with the single product experience the user meets. What you’re really investing in is the stable semantic abstractions that outlast whatever infrastructure comes and goes. And the hardest problem is how to make the organization understandable to its own AI systems. Additional resources: DAMA-DMBOK: Data Management Body of KnowledgeDAMA International – foundational guidance on data ownership, stewardship, and governance rolesOpen Data Contract Standard (ODCS) – an open spec for declaring schema, semantics, quality, and service levels between data producers and consumersOpenLineage – an open standard for collecting data lineage across pipelines and services, useful for tracing what AI features consumeNIST AI Risk Management Framework (AI RMF) – a vendor-neutral framework for accountability and governance of AI systemsCoral – exposes diverse data sources to agents through one declared SQL and semantic layer; an example of meaning being owned per source rather than centrallyGetting Started With Data Quality, DZone Refcard by Miguel García LorenzoData Pipeline Essentials, DZone Refcard by Sudip SenguptaOpen-Source Data Management Practices and Patterns, DZone Refcard by Abhishek GuptaReal-Time Data Architecture Patterns, DZone Refcard by Miguel García Lorenzo“Building Trusted, Performant, and Scalable Databases: A Practitioner’s Checklist” by Saurabh Dashora This is an excerpt from DZone’s 2026 Trend Report, Cognitive Databases, Intelligent Data: Unified Infrastructure for Vector Search, AI-Optimized Queries, and Hybrid Workloads.Read the Free Report

By Miguel Garcia

CORE

Delta Sharing in Action: Securely Share Data Across Organizations With Databricks

For most companies, sharing data with a partner still looks roughly like this: someone writes a CSV to an S3 bucket on Friday, emails a pre-signed URL, and crosses their fingers until Monday. The data is stale before it is even consumed. Schemas drift silently. There is no audit trail of who actually read what. Delta Sharing was Databricks' attempt to fix this, and unlike a lot of "open" things in our industry, the protocol really is open — there is a spec, a reference server, and SDKs in Python, Java, and Go. We have used Delta Sharing in two distinct flavors: Databricks-to-Databricks (D2D), where both sides happen to be on Databricks, and the open protocol, where the recipient is on literally anything that can speak HTTPS. This article walks through both, with the code we actually run. Figure 1. Delta Sharing — provider exposes governed shares; recipients consume without copying data. The Mental Model Three objects do most of the work in Delta Sharing: SHARE – a logical bundle of tables, views, notebooks, or volumes you want to expose.RECIPIENT – the entity that consumes a share. Can be another Databricks account or any external party.PROVIDER – on the recipient side, this is the inbound counterpart of the share, used to mount catalogs. When a recipient queries a shared table, the provider's Delta Sharing server hands back short-lived signed URLs for the underlying parquet files plus a Delta protocol manifest. There is no copy. There is no nightly job. The reader is reading the same parquet files that the provider's own pipelines write. Provider Side: Creating a Share Everything below assumes Unity Catalog is enabled — Delta Sharing is a UC feature. The principal creating the share needs the USE CATALOG and SELECT privileges on what they intend to share, plus CREATE SHARE on the metastore. SQL -- 1. Create the share CREATE SHARE acme_partner_share COMMENT 'Daily order facts and product dim for ACME partner'; -- 2. Add tables (and optionally views) ALTER SHARE acme_partner_share ADD TABLE prod_retail.gold.orders_daily; ALTER SHARE acme_partner_share ADD TABLE prod_retail.gold.product_dim WITH HISTORY; -- enables time travel on the recipient side -- 3. (Optional) partition pruning by recipient property ALTER SHARE acme_partner_share ADD TABLE prod_retail.gold.transactions PARTITION (region = CURRENT_RECIPIENT().region); That last block is one of the more useful tricks: a single share can serve multiple recipients, with the data each one sees scoped automatically by a recipient property. We use this for multi-tenant partner programs — one share, dozens of recipients, no per-tenant view duplication. Provider Side: Defining the Recipient If the recipient is also on Databricks, you exchange a metastore identifier, and you are done — credentials and trust are handled inside Databricks. If they are not, you generate an activation link that the recipient turns into a credential file. SQL -- Open protocol recipient (off-Databricks) CREATE RECIPIENT acme_open COMMENT 'ACME partner, consuming via OSS delta-sharing client'; -- Set a tag the share's partition rule will match ALTER RECIPIENT acme_open SET PROPERTIES (region = 'EU-WEST'); -- Bind the share to the recipient GRANT SELECT ON SHARE acme_partner_share TO RECIPIENT acme_open; -- Show the activation URL once — the recipient downloads a config.share file DESCRIBE RECIPIENT acme_open; Treat the activation link like a password. It can be redeemed exactly once, but if it leaks before that single use, you have to rotate the recipient. Recipient Side: Reading From Pandas The simplest possible consumer is a Python script with the open-source `delta-sharing` package. No Spark required, no Databricks required. Python # pip install delta-sharing import delta_sharing # Path to the .share profile file the provider sent profile = "/secrets/acme.share" # Tables are addressed as <share>.<schema>.<table> table_url = f"{profile}#acme_partner_share.gold.orders_daily" df = delta_sharing.load_as_pandas(table_url) print(df.head()) print(f"rows: {len(df):,}") If the table is too big for a single machine, swap one line: Python # Spark on any cluster — Databricks, EMR, OSS Spark on Kubernetes, etc. df = delta_sharing.load_as_spark(table_url) df.createOrReplaceTempView("orders_daily_shared") spark.sql(""" SELECT region, SUM(net_revenue) AS rev FROM orders_daily_shared WHERE order_date >= current_date() - 7 GROUP BY region """).show() Recipient Side: Mounting in Databricks (D2D) When both sides are Databricks, the recipient creates a PROVIDER object from the share identifier and then mounts the share as a catalog. From that point on, the data behaves exactly like a local UC catalog — same SELECT syntax, same lineage, same governance. SQL -- On the recipient's metastore CREATE PROVIDER acme_provider USING JSON '<provider-json>'; -- Mount the share as a read-only catalog CREATE CATALOG acme_shared USING SHARE acme_provider.acme_partner_share; -- Give analysts access GRANT USE CATALOG, USE SCHEMA, SELECT ON CATALOG acme_shared TO `[email protected]`; -- Query like any other UC table SELECT region, SUM(net_revenue) rev FROM acme_shared.gold.orders_daily WHERE order_date >= current_date() - 7 GROUP BY region; What About Audit? Both providers and recipients get audit events in the system tables. A query we run on the provider side every Monday morning: SQL SELECT event_time, user_identity.email AS recipient, request_params.share_name, request_params.schema_name, request_params.table_name, action_name FROM system.access.audit WHERE service_name = 'unityCatalog' AND action_name LIKE 'deltaSharing%' AND event_time >= current_date() - 7 ORDER BY event_time DESC; If a recipient suddenly stops reading, that is usually a sign of a broken pipeline on their side — and it is easier to ping them proactively than to wait for a support ticket. Pitfalls We Hit Sharing tables that have not been OPTIMIZED produces a brutal number of file listings on the recipient. Compact aggressively before exposing.Time travel on the share requires WITH HISTORY when adding the table. You cannot retroactively flip it on without re-adding.Column masks and row filters defined on the provider apply to the share — make sure your masks behave correctly with NULL recipients.Activation links expire. Bake renewal into your offboarding/onboarding runbook for partners.If the data underneath gets vacuumed too aggressively (under 7 days), recipients with stale snapshots see hard failures. Keep retention at 7+ days. Closing Thoughts Delta Sharing is one of those features that does not look like a big deal in a slide deck and ends up reshaping your data partnerships. The first time a partner reads from your share with a five-line Python script and gets the data live — no copy, no lag, no broken schema — the conversation about "how do we share data" is over. You stop talking about transport and start talking about contracts, SLAs, and what the dataset actually means. Which, honestly, is where the conversation should have been the whole time.

By Seshendranath Balla Venkata

Solving Data Traffic Jams in Your Network

Stop, start. Stop, start. Nothing brings data flows to a grinding halt (or raises an admin’s blood pressure) quite like network congestion. The unwanted, unexpected extra step in an information request or response operation chain is a telltale sign that something’s changed or isn’t working in your infrastructure. And heavier traffic is more than just an inconvenience – it’s a multifaceted problem with knock-on business effects that falls upon admins to identify and fix. Let’s dig deeper into network traffic jams, their primary causes, and how to resolve and prevent them. Understanding What Causes a Digital Traffic Jam Network congestion occurs when the demand for sending or receiving data exceeds the network’s capacity. In other words, a computer network link can’t handle the volume of data trying to use it. It’s like what happens when a person tries to pour more water through a straw than it can handle at once. At a certain point, there’s simply not enough space, causing a backup in the straw. In computer networks, when data packets exceed the network’s capacity, they’re similarly queued in network devices, leading to increased latency and, in turn, traffic jams. 7 Most Common Causes of Network Congestion Bandwidth bottlenecks: When the capacity of network links (such as cables or wireless connections) is insufficient to handle the amount of data being sent.Network device limitations: Routers, switches, and other devices have limited processing power and memory and can become overwhelmed when handling large volumes of traffic.Broadcast storms: A situation where a network becomes flooded with broadcast or multicast packets, often caused by misconfigured devices or faulty hardware.High-bandwidth applications: Applications that consume a lot of network resources, such as video streaming, large file transfers, and backup operations.DDoS attacks: A distributed denial-of-service (DDoS) attack occurs when a network is intentionally flooded with excessive traffic from multiple sources.Poor network architecture: Inefficient routing or inadequate network capacity planning can lead to congestion hotspots.Insufficient internet speeds: Slow service-provider connections can cause bottlenecks at the edge of the network. Performance and Business Consequences of Network Congestion The consequences of network congestion extend far beyond the digital realm, wreaking havoc on your entire IT infrastructure. As data packets get caught in the congestion chaos, you’ll see increased latency and sluggish application performance. Network devices, overwhelmed by traffic, might then start dropping packets, causing retransmissions that add more load and exacerbate congestion. Worse, applications can start to time out because they can’t handle the lengthy delays in data transmission, further compounding the problem. You're also likely to notice jitter, or uneven packet delays, that affect real-time applications like VoIP and video conferencing. Network throughput suffers too, with the overall amount of data that can be transmitted over the network taking a nosedive. Ultimately, users soon begin to notice this digital snarl-up, with slow network performance leading to a decline in productivity and potentially a negative impact on your bottom line. Quality of Service (QoS) for critical applications can degrade as they struggle to receive the priority they need amid congestion. The overarching message is that network congestion can have serious repercussions on performance, end-user experience, and business operations as a whole. Maintaining healthy network traffic is about speed, sure, but it’s also about supporting day-to-day operations. 10 Proven Solutions for Fixing Bad Network Traffic This doesn’t need to be the network status quo. Here’s how admins can and should take back control: Bandwidth management and QoS: Implement QoS policies to prioritize important traffic, effectively creating an express lane for your VIP data packets. Use traffic shaping to control data flow and prevent one application from hogging all the bandwidth.Network segmentation: Divide your network into smaller subnets to contain congestion and prevent a problem in one area from spreading like wildfire.Upgrade network infrastructure: Sometimes you just need more oomph. Upgrade your network devices, increase link capacities, and consider SDN for greater flexibility in traffic management. Optimize application performance: Collaborate with your development teams to improve network efficiency via data compression and caching.Implement caching and Content Delivery Networks (CDNs): For frequently accessed data or web content, use caching or CDNs to lighten the load on your primary network and improve data transfer speeds.Regular network performance monitoring and analysis: Keep a watchful eye on your network performance to identify congestion points and proactively address network issues before they spiral out of control.Load balancing: Distribute network traffic across multiple paths or servers to prevent any single point from becoming a bottleneck.Traffic prioritization: Prioritize critical unicast and multicast traffic over less important data flows.Optimize routing: Regularly review and optimize routing protocols and configurations to ensure efficient traffic flow.Firewall optimization: Ensure your firewalls are properly configured and can handle the traffic load; poorly configured or underpowered firewalls can become network bottlenecks. Keeping Data Speeds Up and Bottom Line Impacts at Bay Again, this is more than about speed (or lack thereof), but the impact of bad network traffic and how it can become a serious business problem. The good news is that it doesn’t have to be a digital death sentence for your IT infrastructure. With a combination of smart network management strategies and the right monitoring tools, you can effectively tackle network congestion and keep your network in the fast lane.

By Sascha Neumeier

From Open SQL to CDS Views: Rewriting SAP Data Access for Performance at Scale

Modern SAP landscapes running on SAP HANA demand a rethink of how ABAP programs access data. Traditional Open SQL queries embedded in ABAP code have served developers for decades, but at large data volumes, they can become performance bottlenecks. SAP’s introduction of Core Data Services (CDS) views offers a new paradigm: push more work to the in-memory database and retrieve only what’s needed. Traditional ABAP Data Access With Open SQL Open SQL is the standard SQL interface in ABAP that allows developers to query the underlying database in a database-agnostic way. For example, an ABAP report might join two tables and fetch results like this: Plain Text SELECT bkpf~bukrs, bkpf~belnr, bkpf~gjahr, bseg~koart, bseg~wrbtr, bseg~shkzg FROM bkpf INNER JOIN bseg ON bkpf~bukrs = bseg~bukrs AND bkpf~belnr = bseg~belnr AND bkpf~gjahr = bseg~gjahr INTO TABLE @DATA(it_fi_docs) WHERE bkpf~bukrs = '1000' AND bkpf~gjahr = '2023' AND bseg~koart = 'K'. This Open SQL example joins the BKPF and BSEG tables to retrieve financial documents. Open SQL sends such queries to the database, and on SAP HANA, the heavy lifting of the join and filtering is done in-memory on the DB server. The result is then brought back to the ABAP application server. However, the challenge with Open SQL at scale comes when ABAP code handles large data sets or complex logic in the application layer. Common performance issues in legacy ABAP include: Too much data transferred: Selecting wide tables or not filtering enough leads to heavy network and memory usage. Best practice is to filter and aggregate in the query to keep the result set small and transfer only the required columns (avoid SELECT *). Multiple round-trips: Performing calculations with many small queries or loops causes repeated DB calls. It’s more efficient to push joins and subqueries into one SQL if possible. Each context switch adds overhead. Application-side processing: If business logic runs on millions of records in ABAP, the application server CPU becomes the bottleneck. The database could perform these operations faster, set-wise. In summary, while Open SQL can express complex data retrieval, ABAP developers traditionally had to be very disciplined in query design to avoid performance issues at scale. This paved the way for a new approach leveraging SAP HANA’s strengths. The Case for Change: Code-to-Data Paradigm SAP HANA’s in-memory, columnar architecture enables it to execute aggregations, filters, and joins extremely fast at the database level. To exploit this, SAP advocated the code-to-data paradigm. push computations down to the database rather than pulling data up to the code. Rewriting data access using CDS views is a key technique in this paradigm, alongside others like AMDP. By offloading heavy operations to the DB, we minimize data transfer and let HANA’s optimized engines handle crunching the data. For example, instead of reading a full table and then filtering in ABAP, you pass WHERE conditions so the DB does it. Instead of multiple selects and merges in ABAP, you perform a JOIN or a subquery in one shot. Another driver for change is SAP’s new data models in S/4HANA. Many classic transparent tables were replaced by HANA-optimized structures or compatibility views. Custom ABAP code written for ECC often breaks or needs adaptation for S/4HANA’s simplified data model. In these cases, SAP often provides CDS views as the new interface to data. As one DZone article notes, engineers moving to S/4 must switch to the S/4 equivalents to replace old data access logic. In short, adopting CDS views is not only about performance but also about aligning with SAP’s modern architecture. Introducing ABAP Core Data Services (CDS) Views ABAP CDS is a framework to define rich data models directly on the database, using a declarative syntax in ABAP Development Tools (ADT). A CDS view is essentially a view in the HANA database, defined via an ABAP DDL statement. For example, here’s a simple CDS view definition joining two tables: Plain Text @AbapCatalog.sqlViewName: 'ZDEMO_FLIGHTS' define view ZFlightInfo as select from spfli inner join scarr on spfli.carrid = scarr.carrid { scarr.carrname as carrier, spfli.connid as flight, spfli.cityfrom as departure, spfli.cityto as arrival } This CDS view ZFlightInfo performs the same join between SPFLI and SCARR as an equivalent Open SQL join would. In fact, you could copy-paste the join logic from ABAP into the CDS definition with minor syntax changes. After activating this view in ADT, the system creates a database view in HANA. ABAP programs can then consume the CDS view just like a table: SQL SELECT * FROM ZFlightInfo INTO TABLE @DATA(it_flights) ORDER BY carrier, flight. The result set it_flights from the CDS view will be identical to what an Open SQL join would produce for the same input tables. Under the hood, both approaches result in the database executing a similar SQL SELECT. So, why use CDS? The benefits become evident as complexity grows: Reusability and model centralization: CDS definitions are stored in the ABAP Dictionary and can be reused by any number of programs or even other CDS views. Instead of writing the same joins or calculations in multiple ABAP reports, you define them once in a CDS view. SAP recommends using a CDS view when you need to retrieve data from multiple related tables, because it involves the least amount of coding and can be reused in multiple objects. In large-scale systems, this consistency is key to a single source of truth for that piece of data logic. Rich expression and metadata: CDS supports advanced SQL features and built-in functions. You can define calculated fields, aggregations, and even leverage specialized HANA capabilities within the view. CDS also allows adding annotations, making the data model self-descriptive. Performance through pushdown: By moving logic into the CDS (and thus into SQL on the database), you reduce the workload on the ABAP layer. The database can apply filters, joins, and computations in parallel, using its optimized engines. Only the final result is sent back to ABAP. Secure and controlled access: CDS views integrate with the SAP authorization concept, ensuring consistent enforcement of business security rules at the data model level, rather than scattering checks in ABAP code. This means performance benefits without sacrificing governance. Tutorial: Converting an Open SQL to a CDS View (with Code) To solidify the concept, let’s walk through a simple conversion. Imagine we have an ABAP report that needs to list flight routes with the airline name. In classic ABAP, you might do this with an inner join in Open SQL as shown below: Open SQL Approach (Legacy ABAP code): Plain Text DATA: lt_flights TYPE TABLE OF zflight_info. "Structure for results SELECT scarr~carrname AS carrier, spfli~connid AS flight, spfli~cityfrom AS departure, spfli~cityto AS arrival FROM spfli INNER JOIN scarr ON spfli~carrid = scarr~carrid INTO TABLE @lt_flights ORDER BY carrname, connid. This code joins SPFLI with SCARR and populates an internal table lt_flights. It works, but the logic is embedded in the program. Now, suppose we want to reuse this same join in multiple places. We can refactor it into a CDS view: CDS View Approach: Define the view in ABAP DDL (e.g., in Eclipse ADT): Plain Text @AbapCatalog.sqlViewName: 'ZFLIGHTINF' @AccessControl.authorizationCheck: #NOT_REQUIRED define view ZFlightInfo as select from spfli inner join scarr on spfli.carrid = scarr.carrid { scarr.carrname as carrier, spfli.connid as flight, spfli.cityfrom as departure, spfli.cityto as arrival } We give the view a name ZFlightInfo. Note that this is almost identical to the Open SQL, just expressed as a view definition. Once activated, the CDS is available system-wide. Now our ABAP report can simply do: Plain Text SELECT * FROM ZFlightInfo INTO TABLE @lt_flights ORDER BY carrier, flight. The result in lt_flights will be the same. We have effectively decoupled the data retrieval logic from the program and centralized it in the DB layer. This not only improves reuse; in a HANA system, it can also improve performance. The database can better optimize a single persistent view than ad-hoc SQL scattered in code. And if we needed to adjust the join or add a new field. Performance Considerations and Best Practices When rewriting Open SQL to CDS, ABAP developers should keep a few important considerations in mind: Measure, don’t guess: Simply converting an Open SQL to a CDS view doesn’t magically speed up the query if it was already efficient. As noted earlier, for straightforward SELECTs or joins, the performance will be equivalent in many cases. The real gains come when you use CDS to do more complex processing in one go. Always use tools like ST05 SQL trace or HANA’s PlanViz to ensure the new design is actually optimal. The execution plan is what matters, not whether you wrote it in Open SQL or CDS. Avoid over-complex views: It’s possible to go overboard with stacking CDS views on top of each other. While layering is good for separation of concerns, too many nested views or excessive use of associations can lead to very complex SQL at runtime. This can confuse the optimizer or prevent predicate pushdown. Be wary of heavy calculations in a single CDS. If performance suffers, consider alternatives like ABAP Managed DB Procedures (AMDP) for really complex logic or break the problem down differently. Select only what you need: Just as with Open SQL, a CDS view should be designed to return only necessary fields and records. Don’t define a CDS with SELECT * from a wide table list the needed fields. This ensures consumer queries aren’t unknowingly pulling extra data. One common pitfall is using CDS to expose an entire table with all columns, which defeats the purpose. Instead, tailor views to use cases or use parameters in CDS to filter data. Use CDS features wisely: Leverage CDS capabilities like aggregations, calculated fields, and unions to eliminate extra work in ABAP. Reuse and consistency: Replace multiple Open SQL implementations of the same logic with a single CDS. Not only does this reuse improve maintainability, but it also means the database might handle the unified load more efficiently. SAP itself follows this approach in S/4HANA with the Virtual Data Model, hundreds of CDS views that serve as the source for Fiori apps and reports, rather than raw table access. By moving to CDS, you align your custom code to the same philosophy. Conclusion Rewriting data access from Open SQL to CDS views is a strategic move for ABAP developers aiming to maximize performance at scale. By pushing more logic to the SAP HANA database, we take full advantage of its in-memory speed and parallel processing. CDS views enable complex data gathering in one shot, reduce the load on the application server, and provide a modular, reusable data model for your SAP applications. That said, an engineer must also approach CDS with a critical eye, understanding the execution plan and ensuring that moving to CDS truly improves the situation, rather than blindly adding abstraction. Advanced ABAP development is about choosing the right tool for the job. In the case of data-intensive operations, CDS views have proven to be a powerful tool, aligning with SAP’s modern direction and delivering robust performance at scale. By rewriting your data access with CDS and following best practices, you can future-proof your ABAP code for the HANA era, achieving faster results and a cleaner, more sustainable codebase for the long run.

By Deepika Paturu

Jakarta NoSQL: Why JPA Is Not Enough for the AI Era

The most effective way to present this idea is to begin with the challenge architects face: AI has transformed the persistence landscape. Enterprise applications were once built almost exclusively on relational databases, making JPA a keystone of Jakarta EE. Today, modern systems use a mix of relational databases, document stores, caches, graph engines, and increasingly, vector databases that support semantic search, retrieval-augmented generation (RAG), and AI-powered applications. Polyglot persistence is now the industry standard. While Jakarta EE standardized relational persistence through JPA, it still lacks a vendor-neutral standard for non-relational persistence. This gap forces developers to rely on fragmented, proprietary solutions, creating barriers to portability, productivity, and innovation. The rise of AI makes this gap critical. Vector databases are now essential to intelligent systems, supporting semantic search, embeddings, and contextual retrieval. For Jakarta EE to remain the leading enterprise Java platform in the AI era, it must offer a standardized approach to NoSQL persistence, as it did for relational databases. Jakarta NoSQL is not just another specification; it constitutes a strategic investment in the ecosystem's future. By offering a familiar programming model, reducing vendor lock-in, and integrating with AI workloads, Jakarta NoSQL ensures that Jakarta EE remains relevant and competitive for the next generation of enterprise applications. NoSQL in the AI Era: Understanding the Modern Data Landscape For years, enterprise data persistence focused on relational databases. Systems relied on tables, rows, foreign keys, and SQL, making relational technology the standard for business applications. While still essential, modern architectures now use polyglot persistence, where multiple database types coexist, each satisfying specific requirements. Today, NoSQL refers to a family of database paradigms, each engineered for specific workloads and architectural needs, rather than just document databases. Key-value databases store data as key-value pairs, enabling fast lookups and low latency. Typical uses include caching, user sessions, feature flags, and temporary application state.Document databases store data as structured documents, such as JSON or BSON. They are effective for applications having hierarchical or evolving schemas, including web applications, e-commerce platforms, and content management systems.Column-family databases organize data by columns instead of rows, supporting high write throughput and horizontal scalability. They are used for IoT telemetry, event logging, analytics, and large-scale distributed systems.Graph databases model entities and relationships as nodes and edges. This structure is ideal for social networks, fraud detection, recommendation engines, dependency analysis, and knowledge graphs in which relationships are critical.Vector databases store high-dimensional embeddings from machine learning models and large language models (LLMs). They enable semantic search, similarity matching, retrieval-augmented generation (RAG), recommendation platforms, and other AI-driven features via understanding meaning instead of exact text matches.Time-series databases specialize in timestamped data that changes over time. They are used for observability, monitoring, financial markets, industrial sensors, and operational metrics where high-performance temporal data storage and analysis are essential. These database types often coexist within the same architecture. Modern applications may use PostgreSQL for transactions, Redis for caching, MongoDB for documents, Neo4j for relationships, InfluxDB for telemetry, and a vector database like Milvus, Pinecone, or Weaviate for AI-powered search and retrieval. This approach, known as polyglot persistence, is now standard in enterprise systems. The industry has embraced this shift. The Stack Overflow Developer Survey shows that while relational databases still dominate enterprise workloads, NoSQL technologies are now standard tools for developers. Technologies like Redis, MongoDB, and Elasticsearch are used alongside PostgreSQL and MySQL. Organizations no longer choose between SQL and NoSQL; instead, they combine multiple persistence technologies to leverage their strengths. Polyglot persistence is now the baseline for modern software systems. Vector databases are especially important among NoSQL categories, as they are basic to modern Artificial Intelligence systems. In contrast to traditional databases that store explicit business data, vector databases store numerical representations called embeddings. Generated by machine learning models, these embeddings encode the semantic meaning of words, documents, images, or other content as mathematical vectors. This enables software to search and retrieve information based on meaning rather than exact text matches. The distinction between lexical and semantic search illustrates the significance of vector databases. For example, a traditional SQL search for “Pet” returns records with that exact term, such as “Pet Shop,” but ignores related expressions like “Dog” or “Puppy.” Semantic search, by comparing embeddings, retrieves documents about dogs, puppies, or animal companions because it recognizes their semantic relationship. The search engine matches meaning, not just syntax. This function is vital for modern AI architectures. Large language models do not process relational tables directly; they use embeddings and contextual connections between concepts. Systems such as retrieval-augmented generation (RAG), enterprise knowledge search, recommendation engines, and intelligent assistants depend on similarity searches across millions of vectors. While relational databases can support some vector operations through extensions, vector databases are purpose-built for these workloads, offering optimized indexing and similarity algorithms for large-scale semantic retrieval. As AI adoption grows, vector databases are becoming a strategic component of enterprise architecture. Appreciating the importance of NoSQL, several Java ecosystems have developed their own solutions. Spring offers independent projects like Spring Data MongoDB, Spring Data Redis, and Spring Data Cassandra. These integrations provide a productive programming model but are tightly coupled to the Spring ecosystem. Quarkus supports NoSQL persistence through Panache and database-specific integrations, emphasizing developer productivity and cloud-native deployment. Micronaut Data supports several NoSQL engines, using compile-time code generation and ahead-of-time processing to improve performance and reduce execution overhead. While these solutions are effective, they remain framework-specific rather than platform standards. Developers switching frameworks encounter different APIs, abstractions, annotations, and operational models, even when solving similar persistence challenges. Jakarta EE addressed this for relational persistence with Jakarta Persistence (JPA), delivering a standardized, vendor-independent programming model. As NoSQL technologies expand and AI workloads more and more depend on vector databases, the lack of a vendor-neutral NoSQL standard is a significant gap in the Jakarta ecosystem. The Java Standardization Journey The need for a standardized NoSQL solution in the Java ecosystem has been discussed for years. During the Java EE era, several proposals tried to integrate non-relational databases into the enterprise platform. As NoSQL technologies grew in popularity throughout the 2010s, developers anticipated a dedicated specification to accompany traditional enterprise APIs at JavaOne conferences. Despite clear demand, no such initiative emerged within Java EE. The platform remained focused on relational persistence via JPA, leaving NoSQL adoption to rely on vendor-specific libraries and framework integrations. The transition of Java EE to the Eclipse Foundation provided an opportunity to address this challenge. Instead of waiting for a platform-level solution, the community launched Eclipse JNoSQL, an open-source project supplying a unified programming model for NoSQL databases. Drawing on JPA's success, Eclipse JNoSQL introduced mapping annotations, repositories, templates, and communication APIs that support document, key-value, column-family, and graph databases. The project showed that a consistent developer experience could be attained without compromising each database model's unique features. As Jakarta EE matured, Eclipse JNoSQL became the foundation for a new standardization effort: Jakarta NoSQL. Jakarta NoSQL was the first persistence specification created entirely within the Jakarta EE process. Unlike earlier specifications that migrated from Java EE, Jakarta NoSQL was conceived, developed, and released under the Eclipse Foundation governance model. It was among the first to complete the full Jakarta Specification Process from inception to release. Jakarta NoSQL's impact extended beyond its initial scope. During development, the expert group identified a common challenge for both relational and non-relational databases: developers needed a consistent repository abstraction independent of the underlying persistence engine. This led to the creation of a separate specification, Jakarta Data. The need to standardize NoSQL access patterns directly influenced the development of Jakarta Data's repository-oriented programming model, which applies across multiple persistence technologies. The relationship between these specifications highlights Jakarta NoSQL's broader influence on the Jakarta EE ecosystem. Jakarta NoSQL focuses on mapping and interacting with non-relational databases, while Jakarta Data delivers a unified repository abstraction for both relational and NoSQL implementations. Together, they significantly reduce fragmentation in enterprise persistence. This evolution continued beyond Jakarta Data. The drive to standardize modern persistence requirements has inspired new specifications, such as Jakarta Query, which aims to deliver a portable, type-safe, and expressive query language for various persistence technologies. As the Jakarta ecosystem grows, Jakarta NoSQL acts as a key milestone. It addressed the long-standing absence of a NoSQL standard and helped lay the foundation for the next generation of persistence specifications within Jakarta EE. Jakarta NoSQL: Built for NoSQL, Not Adapted to It When architects consider standardizing NoSQL development in Jakarta EE, a common question arises: why not extend Jakarta Persistence (JPA) to support NoSQL databases? JPA has long provided a unified programming model for relational databases in the Java ecosystem. The answer is based on a core architectural principle: tools should be optimized for their intended purpose. The first challenge is that JPA was designed specifically for relational databases, relying on concepts like tables, columns, joins, foreign keys, and transactional consistency. These are not simply implementation details but core elements of the specification. Forcing document, graph, key-value, or vector databases into this model creates friction and limits the use of each database’s native features. The second challenge is that NoSQL systems behave fundamentally differently. Graph databases perform path traversals, document databases store nested structures without normalization, key-value databases focus on fast lookups, and vector databases handle similarity calculations. These systems also differ in consistency, transactions, query languages, indexing, and scalability capabilities. Representing all these paradigms through a single relational abstraction leads to compromises. The third challenge is the importance of specialization. As Abraham Maslow noted, “if the only tool you have is a hammer, it is tempting to treat everything as if it were a nail.” Relational databases are effective, but not ideal for every persistence need. Semantic search, graph traversal, and high-volume telemetry storage are not relational problems. Applying a relational abstraction to all database types runs the risk of losing the unique optimizations each technology provides. Examine the analogy of transportation: cars, boats, submarines, and airplanes all address transportation but are specialized for different environments. Forcing them to use the same controls would result in mediocrity across all. Similarly, a single persistence abstraction may remove the features that make each database effective. Therefore, Jakarta NoSQL does not extend JPA beyond its intended scope. Instead, it offers a dedicated persistence model for non-relational databases, while continuing to maintain the familiar developer experience that contributed to JPA’s success. A key design goal of Jakarta NoSQL is to reduce mental effort for enterprise Java developers. Teams experienced with JPA should find the specification immediately approachable, as Jakarta NoSQL intentionally uses familiar terminology and concepts from the Jakarta EE community. Developers will encounter annotations like @Entity, @Id, and @Column, enabling a smooth transition from relational to non-relational persistence. Java @Entity public class Car { @Id private Long id; @Column private String name; @Column private CarType type; } At first glance, this entity closely resembles a JPA entity, which is intentional. However, the underlying implementation is fundamentally different. Jakarta NoSQL is built to support schema flexibility, embedded structures, nested documents, and database-specific storage models. This approach is reflected throughout the API. Instead of requiring developers to oversee low-level driver details, Jakarta NoSQL offers a high-level programming model via the Template API. Java @Inject Template template; Car ferrari = Car.builder() .id(1L) .name("Ferrari") .build(); template.insert(ferrari); List<Car> sports = template.select(Car.class) .where("type").eq(CarType.SPORT) .orderBy("name") .result(); The objective mirrors JPA’s original mission: permitting developers to focus on domain models and business logic, rather than serialization, connection management, or vendor-specific APIs. This foundation shaped Jakarta NoSQL 1.0. The initial release introduced the mapping layer, CDI integration, repository support, template operations, and standardized endpoints for four major NoSQL categories: Document databasesKey-value databasesColumn-family databasesGraph databases Jakarta NoSQL 1.0 showed that a unified Java programming model can respect the particular characteristics of each database family. Jakarta NoSQL 1.1 continued this evolution. While version 1.0 focused on mapping and persistence, version 1.1 expanded querying capabilities through integration with Jakarta Query. A key addition is support for parameterized queries, letting developers to safely bind parameters instead of manually constructing query strings. Java List<Car> cars = template.query( "FROM Car WHERE type = :type") .bind("type", CarType.SPORT) .result(); Version 1.1 also introduces projection support, allowing applications to retrieve lightweight views instead of entire entities. Java @Projection public record TechCarView( String name, CarType type) { } List<TechCarView> views = template .typedQuery( "FROM Car WHERE type = 'SPORT'", TechCarView.class) .result(); These features improve performance, reduce data transfer, and comply with modern Java features such as records. An important aspect of Jakarta NoSQL is its long-term architectural vision. While most developers use the mapping layer, the specification also defines a lower-level communication API for advanced scenarios. Java DocumentManagerFactory factory = ...; DocumentManager manager = factory.get("users"); DocumentRecord record = ...; manager.put(record); Optional<DocumentRecord> result = manager.findByKey("user:10"); manager.deleteByKey("user:10"); This communication layer is optional. Application developers can build complete systems without it, but it is valuable for database vendors, framework authors, and advanced integrations needing direct access to database capabilities. This design is fundamentally different from JDBC, which assumes communication through SQL statements and tabular result sets. That model works well because relational databases share a common language and interaction pattern. NoSQL databases do not. Document databases may use BSON, graph databases may offer traversal languages, and vector databases may provide similarity-search APIs. Others use REST endpoints, binary protocols, gRPC streams, or vendor-specific mechanisms. Forcing these models into a JDBC-style abstraction would limit their capabilities or demand ongoing vendor-specific extensions. For this reason, Jakarta NoSQL uses a layered architecture. The mapping layer offers a portable, productive programming model for developers, while the communication layer remains flexible to support diverse NoSQL systems. This architecture positions the specification for future growth. As new technologies like vector databases, time-series engines, and AI-native storage emerge, Jakarta NoSQL can evolve without imposing a relational mindset. Rather than treating every database as a nail for the JPA hammer, Jakarta NoSQL recognizes that different problems require different tools, while still presenting a consistent and familiar experience for enterprise Java developers.

By Otavio Santana

CORE

From printTriangularNumber to Duff’s Device: Mastering Java Switch Statements Old and New

In this blog post, we will see how the humble Java switch statement evolved from a fall-through curiosity into a powerful expression, and how understanding its mechanics unlocks classic techniques like Duff's Device. Java's switch statement has evolved from a fall-through-prone construct into a modern expression syntax introduced in Java 14. The post traces this evolution using a concrete example, a method that computes triangular numbers by intentionally allowing execution to cascade through cases without break statements. The post also connects this behavior to Duff's Device, a 1983 loop-unrolling technique that uses deliberate fall-through to handle remainder elements before processing full blocks. A comparison of old and new switch syntax outlines trade-offs, and practical guidance is offered on when each form is appropriate. The Accidental Discovery I was prepping for the OCP Java 21 exam and stumbled across a tricky question. A method named question2 used a switch statement without any break statements. The output surprised me at first. Once I traced through it, I renamed the method to printTriangularNumber. That one rename told the whole story. This post dives into why. The Old Switch Statement The traditional switch statement has been part of Java since day one. The syntax looks like this: Java int day = 3; switch (day) { case 1: System.out.println("Monday"); break; case 2: System.out.println("Tuesday"); break; case 3: System.out.println("Wednesday"); break; default: System.out.println("Unknown"); break; } As shown above, every case ends with a break. Without it, execution does not stop. It keeps going into the next case. The old switch works on int, char, String, and enum types. Fall-Through: Feature or Bug? The most misunderstood behavior in switch is fall-through. When you omit break, execution literally falls into the next case. Java int x = 2; switch (x) { case 3: System.out.println("three"); case 2: System.out.println("two"); // jumps here case 1: System.out.println("one"); // falls through default: System.out.println("done"); // falls through } Output: Plain Text two one done Most developers treat this as a bug waiting to happen. They are not wrong. Forgetting a break is one of the most common Java mistakes. But intentional fall-through is a different story. It is a deliberate tool. And printTriangularNumber is the perfect example. printTriangularNumber: Fall-Through in Action Here is the method I renamed from question2 during my OCP prep: Java private static void printTriangularNumber(int n) { int res = 0; switch (n) { case 5: res += 5; case 4: res += 4; case 3: res += 3; case 2: res += 2; case 1: res += 1; default: break; } System.out.println(res == 0 ? "Ok, bye." : res); Let us trace through n = 4: Jumps to case 4, adds 4. res = 4 Falls to case 3, adds 3. res = 7 Falls to case 2, adds 2. res = 9 Falls to case 1, adds 1. res = 10 Hits default, breaks Output: 10 The pattern for each input: nResultFormula111232+1363+2+14104+3+2+15155+4+3+2+1 This is n * (n + 1) / 2, the triangular number formula. The fall-through is doing the summation for you. Each case accumulates the remaining values by simply not stopping. For n = 0 or any value above 5, no case matches, default fires immediately, and res stays 0. The ternary prints "Ok, bye.". I personally find it a beautiful example of using language semantics intentionally. This is also the kind of question the OCP exam loves to throw at you. The New Switch Expression (Java 14+) Java 14 introduced switch expressions as a standard feature. The arrow syntax -> eliminates fall-through entirely. Each arm is independent. Java int day = 3; String name = switch (day) { case 1 -> "Monday"; case 2 -> "Tuesday"; case 3 -> "Wednesday"; default -> "Unknown"; }; System.out.println(name); // Wednesday A few things to notice here: Switch is now an expression. It returns a value. The arrow -> replaces : and break together. No fall-through. Each arm executes independently. Multiple labels on a single arm: case 1, 7 -> "Weekend"; You can also use it inline: Java System.out.println(switch (day) { case 1, 7 -> "Weekend"; default -> "Weekday"; }); Much cleaner. Much safer. Switch Expressions With Yield Sometimes you need more than a single expression in an arm. That is where yield comes in. Java int n = 4; int result = switch (n) { case 1, 2 -> n * 10; case 3, 4 -> { int temp = n * n; System.out.println("Computing for: " + n); yield temp; // return value from block } default -> 0; }; System.out.println(result); // 16 Think of yield as the return statement for a switch block arm. You need it whenever the arm has multiple statements inside {}. A common mistake is using return instead of yield inside a switch expression block. That compiles only inside a method and it returns from the entire method, not just the switch. Always use yield inside switch expression blocks. Duff's Device: Fall-Through Taken to the Extreme Now that we understand fall-through well, let us look at the most famous intentional use of it: Duff's Device. Tom Duff invented this in 1983 to speed up memory copy operations by reducing loop branch overhead. The trick is to unroll the copy loop and use a switch to jump into the middle of it based on the remainder. In Java, we replicate it in two clean phases since Java does not allow interleaved switch+loop syntax: Java public static void duffCopy(int[] src, int[] dst, int n) { int i = 0; int rem = n % 4; // Phase 1: handle remainder via fall-through switch (rem) { case 3: dst[i] = src[i]; i++; case 2: dst[i] = src[i]; i++; case 1: dst[i] = src[i]; i++; case 0: break; } // Phase 2: full blocks of 4 int fullBlocks = (n - rem) / 4; while (fullBlocks-- > 0) { dst[i] = src[i]; i++; dst[i] = src[i]; i++; dst[i] = src[i]; i++; dst[i] = src[i]; i++; } } Let us trace through n = 13: rem = 13 % 4 = 1 Switch jumps to case 1, copies 1 element. i = 1 fullBlocks = (13 - 1) / 4 = 3 Loop runs 3 times, copying 4 elements each time Total: 1 + 12 = 13 elements The Python equivalent makes the two phases explicit: Python def duff_copy(src, n): dst = [None] * n rem = n % 4 for i in range(rem): # Phase 1: remainder dst[i] = src[i] i = rem while i < n: # Phase 2: full blocks dst[i] = src[i] dst[i+1] = src[i+1] dst[i+2] = src[i+2] dst[i+3] = src[i+3] i += 4 return dst The connection to printTriangularNumber is direct. Both use fall-through intentionally. In printTriangularNumber, the switch jumps to the right case and accumulates downward. In Duff's Device, the switch jumps to the right case and copies the remainder before the main loop takes over. Old vs. New Switch at a Glance FeatureOld Switch (:)New Switch (->)Fall-throughYes (default)NoReturns valueNoYesbreak neededYesNoMultiple labelsNoYes (case 1, 2 ->)Block with yieldNoYesNull safeNoYes (Java 21 preview)OCP exam topicYesYes Which One Should You Use? For new code, always prefer the switch expression with ->. It is safer, cleaner, and expressive. Your reviewers will thank you. Reserve the old switch with fall-through only when you genuinely need the cascading behavior, like in printTriangularNumber or a hand-tuned loop like Duff's Device. In those cases, add a comment explaining the intent. Otherwise, the next developer (including future you) will assume the break is missing by accident. My personal observation: the OCP Java 21 exam tests both heavily. Knowing when fall-through is intentional versus accidental is the key distinction examiners probe. Make sure you can trace through any switch block without running it. Happy testing! What is your take: is intentional fall-through clever engineering or a maintenance nightmare waiting to happen? Drop your thoughts below!

By NaveenKumar Namachivayam

CORE

A Practical Guide to Temporal Workflow Design Patterns

Long-running, distributed business processes often require careful coordination, state management, and fault handling. Temporal offers a code-first approach to durable workflows: developers write ordinary code for orchestration, and the Temporal service persists state, retries failed tasks, and resumes execution after failures. This shifts focus from plumbing (queues, retries, timeouts) to domain logic, but it also encourages reuse of proven patterns. The Temporal community and documentation highlight several orchestration patterns — for example, sagas, state machines/actors, polling strategies, fan-out/fan-in, and versioning patterns — that solve recurring problems in workflow design. This article surveys these patterns, explaining when and how to use them, with concise code snippets to illustrate their implementation in Temporal. A classic pattern in distributed transactions is the Saga (compensating transaction). In a saga, a business process is broken into a sequence of steps, each with its own “undo” compensation. If any step fails, the saga executes compensations in reverse order to restore consistency. In Temporal, this maps naturally to a try/catch around activities or to the built-in Saga helper. For example, a vacation booking workflow might book a hotel, then a flight, then an excursion. Each step registers a compensation action before invoking the activity. If a failure occurs, the catch block calls saga.compensate() to run all registered compensations in reverse. The following Java-like snippet shows this approach: Java public void bookVacation(BookingInfo info) { Saga saga = new Saga(new Saga.Options.Builder().build()); try { saga.addCompensation(activities::cancelHotel, info.getClientId()); activities.bookHotel(info); saga.addCompensation(activities::cancelFlight, info.getClientId()); activities.bookFlight(info); saga.addCompensation(activities::cancelExcursion, info.getClientId()); activities.bookExcursion(info); // If all succeed, method returns normally. } catch (TemporalFailure e) { saga.compensate(); // undo previous steps throw e; // propagate failure } } If any book* activity throws an exception, the catch invokes saga.compensate(), which calls cancelExcursion, cancelFlight, and cancelHotel in reverse order. This pattern ensures that even if the workflow crashes after partial work, Temporal’s durable execution will eventually resume the compensation sequence. Because Temporal workflows are persistent, the saga logic itself is recoverable – the service records each step and its compensation in the history. In effect, workflows become distributed state machines where try/catch embodies the saga pattern. Polling and External Events Workflows often need to wait for external processes or inputs. In Temporal, there are two main polling strategies. Frequent polling (short interval) is implemented inside an activity loop: the activity repeatedly attempts a call, sleeps briefly, and heartbeats after each iteration. Because long-running activities must heartbeat to show liveness, the loop invokes Activity.getExecutionContext().heartbeat(null) each cycle. For example, a polling activity might look like this: Java @Override public String doPoll() { ActivityExecutionContext context = Activity.getExecutionContext(); while (true) { try { return service.getServiceResult(); } catch (TestServiceException e) { // Service not ready; will retry } // Heartbeat to prevent timeout, then sleep briefly context.heartbeat(null); sleep(POLL_DURATION_SECONDS); } } In this snippet, service.getServiceResult() is retried until it succeeds. Each loop iteration heartbeats and sleeps for a fixed interval. If the worker or service crashes, Temporal will resume the loop exactly where it left off. This pattern is ideal for rapid retries or waiting on resources that become available shortly. For infrequent polling, Temporal relies on activity retry options instead of a custom loop. A workflow can call an activity once, but configure its retry backoff so that failures trigger re-execution after longer delays. In practice, one sets a high initial retry interval and backoff coefficient in the ActivityOptions at workflow time. The workflow code itself is just a single activity call (no loop needed). If the activity throws an error, Temporal automatically retries it later, waiting longer each time. This approach leverages the built-in retry policy (e.g., exponential backoff) for occasional checks. To handle arbitrary external signals or time delays, Temporal workflows can also use Workflow.await(timeout, condition) or Workflow.newTimer(). For instance, a workflow might await a boolean flag that is set by a signal handler, or await a fixed timeout for human input. This avoids busy-wait loops at the workflow level. Signals themselves can come at any time; Temporal’s messaging system lets running workflows be interrupted by signals without polling. In short, Temporal workflows mix timers (Workflow.await) and external signals to wait efficiently. Frequent polling lives in an activity with heartbeats, whereas infrequent or one-off waits can use activity retry or workflow timers. Parallel and Batch Processing When processing large data sets or issuing many operations in parallel, Temporal’s fan-out/fan-in pattern is useful. A parent workflow can spawn multiple child workflows or activities concurrently and then wait for all to complete. This is commonly used for batch jobs, bulk queries, or any parallel computations. The following example shows a “page-by-page” batch processing workflow. For each batch of records, it spawns a child workflow per record and then uses Promise.allOf() to wait for all children. When a batch is done, it can optionally continue-as-new to process the next page without growing history indefinitely: Java @Override public int processBatch(int pageSize, int offset) { List<SingleRecord> records = recordLoader.getRecords(pageSize, offset); List<Promise<Void>> results = new ArrayList<>(); for (SingleRecord record : records) { String childId = Workflow.getInfo().getWorkflowId() + "/" + record.getId(); RecordProcessorWorkflow processor = Workflow.newChildWorkflowStub(RecordProcessorWorkflow.class, ChildWorkflowOptions.newBuilder().setWorkflowId(childId).build()); results.add(Async.procedure(processor::processRecord, record)); } // Wait for all child workflows to finish Promise.allOf(results).get(); // If no more records, return result and finish if (records.isEmpty()) { return offset; } // Otherwise continue as new for the next batch (to reset history) return nextRun.processBatch(pageSize, offset + records.size()); } In this code, each child workflow processes one record. The parent collects a list of Promise<Void> and calls Promise.allOf(...).get(), which blocks the parent until all child workflows complete. Using children allows highly parallel processing without overloading a single worker. After finishing a batch, the code checks if (records.isEmpty()) and returns; otherwise it calls a continueAsNew stub (nextRun) with an updated offset. This continueAsNew effectively starts a fresh workflow execution with a new history, avoiding unbounded history growth for long-running loops. As shown, Temporal’s Async and Promise primitives make parallel fan-out/fan-in straightforward. Beyond paging, fan-out can apply to any use case needing parallel work (bulk updates, scatter-gather queries, etc.). Conversely, gathering results into a list or aggregation is just collecting activity/child results into a shared variable, which Temporal safely persists in the history. Actor-Like Workflows and Event-Driven Patterns Temporal workflows are naturally stateful and can run indefinitely, making them suitable for actor or process-manager patterns. A workflow can “sleep” or wait for signals, maintain in-memory state, and react to external events. Clients can use signals (@SignalMethod) to send events into a running workflow and queries (@QueryMethod) to read its state without affecting it. This allows workflows to act like autonomous entities. For example, imagine a subscription service workflow. It starts with a customer on trial, waits for either trial expiration or a cancellation signal, then proceeds to billing periods. Signals like cancelSubscription() can interrupt the main flow. Meanwhile, queries like queryCustomerId() can retrieve the workflow’s state from outside. Temporal’s event system handles all this without polling: “a running workflow can receive external messages without polling, and clients can inspect workflow state at any time”. Internally, the workflow code can use Workflow.await(...) to pause until a signal sets a flag. Here’s a conceptual sketch (TypeScript/JavaScript style) of using signal and query definitions: TypeScript const abortSignal = defineSignal<[string]>('abort'); const updateSignal = defineSignal<[number]>('update'); const getStateQuery = defineQuery<State>('getState'); export async function statefulWorkflow(config: Config): Promise<Result> { let state: State = {...initial...}; let aborted = false; setHandler(abortSignal, (reason: string) => { aborted = true; }); setHandler(getStateQuery, () => state); // Main workflow logic: await condition(() => aborted, '1 minute'); if (aborted) { // cleanup or compensation return { status: 'aborted' }; } // ... continue normal processing return { status: 'completed' }; } In this pattern, external callers would workflow.signal(abortSignal, reason) or workflow.query(getStateQuery). Temporal’s signal-and-query features implement a process manager-style pattern: a workflow can behave like an event-driven state machine, reacting to signals in real time and allowing external inspection. This is more robust than polling, and since all state changes happen in the workflow code, consistency is guaranteed. (If a query is issued while the workflow is mid-activity, it will reflect the last completed state.) Note that newer Temporal releases also support Workflow Updates, which are like synchronous signals that can return values. In environments where Update is available, a workflow can reply to a message directly. Otherwise, a client can query state as a two-step “signal then query” process. Either way, this pattern empowers long-lived processes and human-in-the-loop steps. Versioning and Evolving Workflows Temporal requires workflow code to be deterministic, so changing logic in running workflows must be done carefully. The community and docs describe versioning strategies. For short-lived or rare workflows, one can deploy a new workflow definition (e.g. MyWorkflowV2) or use a new task queue for new versions. For long-lived workflows, Temporal’s Workflow.getVersion API lets the code branch on a version number recorded in the history. This is often called the “patch” strategy. For example: Java int version = Workflow.getVersion("checksumAdded", Workflow.DEFAULT_VERSION, 1); if (version == Workflow.DEFAULT_VERSION) { activities.upload(targetBucket, targetFilename, data); } else { long checksum = activities.calculateChecksum(data); activities.uploadWithChecksum(targetBucket, targetFilename, data, checksum); } Here, on first execution getVersion("checksumAdded", DEFAULT, 1) returns DEFAULT_VERSION and runs the original upload() call. When a new worker with updated code runs getVersion("checksumAdded", DEFAULT, 1) again, Temporal records version = 1 in the history. Future runs hit the else branch and use the new uploadWithChecksum() code. This ensures deterministic replay: workflows that started before the code change continue on the original branch, and newer executions use the new logic. After all old executions finish, the branching logic can often be removed. Overall, versioning patterns let developers evolve workflows without breaking running executions. Temporal offers multiple options — definition names, task queues, and the getVersion API — each with trade-offs. (Using separate definitions or queues isolates versions at the cost of more infrastructure, while getVersion keeps a single codebase but requires planned version markers.) Regardless, versioning is a key pattern to safely deploy workflow updates in production. Conclusion Temporal’s durable workflow engine incorporates many built-in aids for complex process patterns. By applying established designs — such as sagas for compensating transactions, retry and heartbeat loops for polling, fan-out/fan-in via child workflows, and event-driven actors with signals/queries — engineers can build robust systems without manual boilerplate. Each pattern leverages Temporal features: workflows and activities, promises, signals, queries, and continuations. The examples above show how little code is needed: a few method calls and standard control structures achieve what would otherwise be elaborate orchestration logic. In practice, adopting these patterns means that failures are handled gracefully and state is managed cleanly. For example, the saga code snippet illustrates reversing partial work on error, while the parallel batch example shows how to process unbounded data safely with continueAsNew. In summary, understanding Temporal’s idioms — as documented by the Temporal team and community — empowers developers to focus on business logic while the platform ensures reliability. Mastery of these workflow patterns leads to systems that are easier to reason about, easier to maintain, and resilient in production.

By Akhil Madineni

AI Is Finding Bugs Faster Than Enterprises Can Patch — Here's What Data Security Teams Should Do

I have spent the better part of a decade building data protection products for global enterprises. Cloud DLP, CASB, SSPM, Behavior Threats, AI Access Security, ISPM, etc. The kinds of things that sit between a user, an agent, or an application and the sensitive data nobody wants to see in the wrong place. Every conversation I have had with a customer security architect this year eventually arrives at the same question. The threat landscape has clearly changed. What does that mean for the controls we already own? This article is the analysis I have been sharing with security architects across industries who are evaluating how their data protection programs need to evolve. It is grounded in what is publicly documented, what it actually changes for enterprise data security, and where I would direct the next dollar of investment based on a decade of building these products at scale. What Actually Shifted, With Sources There are three publicly verifiable data points worth understanding before any control conversation makes sense. Discovery Is Becoming Inexpensive Mozilla shipped Firefox 150 in April 2026 with two hundred and seventy-one fixes that came out of a single sweep using an early version of Anthropic’s Mythos preview model. That is roughly four times the project’s typical annual baseline, in one pass. Mozilla also added the most honest sentence I have read on this topic all year. They said they have not seen any bug in the set that an elite human researcher could not have found, given enough time. SecurityWeek covered the details: securityweek.com/claude-mythos-finds-271-firefox-vulnerabilities. Read that caveat carefully. The thing that became automated is not novelty. It is the cost of finding a class of bugs that humans were always capable of finding. When the price of an action drops by an order of magnitude, the action gets done at scale. That is the shift, and it is the shift that matters. Patching Is Not Getting Cheaper at the Same Rate HackerOne paused new submissions to its Internet Bug Bounty program on March 27, 2026. The IBB is the oldest crowdsourced vulnerability reward program for open source, dating back to 2013. The pause was not a budget decision. It was an admission that the gap between AI-assisted discovery volume and the ability of volunteer maintainers to ship patches had become unbridgeable on the existing incentive model. Dark Reading’s coverage is here: darkreading.com on the IBB pause. Earlier in the year, the curl project removed bounties from its program for the same reason, after a wave of low-quality AI-generated submissions overwhelmed triage. If the upstream open source ecosystem is struggling to keep pace with discovery, every enterprise that ships software with open source dependencies is downstream of that struggle. That is most enterprises. Autonomous Agents Are Already Creating Real Incidents In April 2026, the Cloud Security Alliance published two surveys that I think every data security team should read. The first study found that fifty-three percent of organizations have had AI agents exceed their intended permissions, and forty-seven percent have already experienced a security incident involving an agent in the past year. The second, published a week later, reported that eighty-two percent of enterprises have discovered previously unknown agents running in their environments, and sixty-five percent have had an agent-related incident. The most common consequence was data exposure. CSA’s findings: Enterprise AI Security Starts with AI Agents and Autonomous but Not Controlled. Take those three threads together. Bug discovery is industrializing. The patch side is bottlenecked. And inside the enterprise, autonomous agents are already operating in places nobody fully maps. That is the operating reality, not a forecast. Why This Matters More for Data Security Than for Any Other Function Most of the AI security conversation is framed around vulnerabilities and exploits. I think that framing misses what is actually changing for enterprises. When a class of vulnerabilities becomes cheaper to discover, the average time between exposure and exploitation shortens. When average exposure time shortens, the probability that any given control fails inside that window goes up. When more controls fail more often, the consequence shows up at the data layer. Data is the asset. Everything else is a path to it. The CSA finding I keep coming back to is the one that says agent incidents most often produce data exposure, not service outages. That tracks with what I see at customer sites. The blast radius of an agent compromise is determined by the data the agent had access to, the policies that were being watched, and the speed at which someone noticed. None of those three is improving on the timeline that adversaries are improving. If an agent has access to your sensitive data, the agent is part of your data security perimeter, whether your DLP product knows it or not. That sentence is the part of the conversation that I find most data security teams are not yet having internally. It needs to happen this quarter. Three Things Data Security Programs Should Rethink Now 1. Stop Treating Non-Human Identities as a Hygiene Problem CyberArk’s 2025 Identity Security Landscape, surveying 2,600 cybersecurity decision-makers globally, found that machine identities now outnumber human identities by more than 80 to 1 in the typical enterprise, up from roughly 45 to 1 in their 2024 study. GitGuardian’s State of Secrets Sprawl 2025 report found 23.8 million new secrets exposed on public GitHub in 2024 alone, a 25 percent year-over-year increase, with non-human identities flagged as the dominant credential population behind that growth. The exact ratio in any given environment is a question for the IAM team, but the order of magnitude is consistent across every serious study I have read, and it is rising fast. Most enterprise IAM programs were designed around human users. Periodic access reviews. Quarterly attestation cycles. Manager signoff. None of that infrastructure was built for a population that is now eighty times larger, that provisions itself, and that often outlives its original use case. CSA’s research found that only 21 percent of organizations have a formal decommissioning process for AI agents. Everyone else is accumulating what the report calls retirement debt: agents who completed their task months ago and still hold credentials, tokens, and data access. From a data security standpoint, the practical consequence is that an enterprise’s most overprivileged identity is rarely a person. It is a service account from 2022 that nobody remembers, an OAuth grant that an integration test attached to a real production scope, or a workflow agent that picked up admin-level permissions during deployment because the person setting it up did not want to debug a permission-denied error at 11 p.m. These identities are reachable by adversaries through a single credential compromise, and they often have direct access to the kinds of data that DLP policies were written to protect at the human user layer. The remediation requires a structured non-human identity program with a named owner, a defined lifecycle covering provisioning, rotation, and decommissioning, and quarterly access reviews that apply to bots the way they apply to humans. Workload identity federation rather than long-lived secrets. Scoped service accounts. Logging that captures what each non-human identity touched, not just whether it authenticated successfully. From a tooling perspective, this work sits at the intersection of CASB, IAM, and DLP, and in most enterprises, it has no clear owner across those three functions. Establishing that ownership is the precondition for everything else. 2. Refresh Classification and Tagging for an Agentic Environment In my own work on DLP product strategy, I have come to think of classification and tagging as the foundation that every other data control sits on. If sensitive content is correctly identified at the moment it is created or ingested, downstream policies have a fighting chance. If it is not, no amount of policy authoring downstream will catch up. Most enterprise tagging programs were designed for documents flowing through email, endpoints, and a manageable list of SaaS applications. The current generation of AI agents and copilots flows through none of those choke points cleanly. An agent reads a corpus, generates a derivative artifact, and writes that artifact somewhere else. The original tag, if there was one, often does not survive the round trip. The derivative may contain sensitive content that was reassembled from sources that were each individually below the policy threshold. Three practical refreshes are worth funding now. Treat AI-generated outputs as a first-class data class. Anything produced by an agent or copilot needs provenance metadata that travels with it: which model produced it, against which prompt, derived from which sources, with which level of human review. Most enterprise classification taxonomies do not have a slot for this yet. Add one.Lower the threshold for tagging at ingestion. The cost of misclassifying a sensitive document used to be that a human eventually emailed it to the wrong person. The cost now includes an agent reading it as part of a larger context and producing a derivative that lands in a SaaS workspace your DLP product does not inspect. Err on the side of more aggressive classification at the source.Audit your DLP coverage of LLM endpoints and agentic SaaS surfaces. Most DLP deployments I see in the field have rich coverage of email and endpoints, partial coverage of cloud applications, and almost no coverage of the LLM and agent traffic that has become a meaningful share of how sensitive data now leaves the environment. That is the coverage gap most likely to show up in a 2026 incident report. 3. Put a Model in the Pull Request Path This is one of the few areas where the offensive shift in AI capability cuts directly in defenders’ favor, and most enterprise application security programs are not yet using it. The traditional SAST and DAST queue is where AppSec hours go to die. Thousands of unverified findings, most of them noise, validated entirely by humans on a backlog that never empties. The newer pattern is to put a model-based reviewer in the pull request path itself. Every PR is reviewed by an automated agent for security defects before a human sees it. Findings show up as inline comments. High-confidence findings can block the merge. OpenAI publicly stated in April 2026 that its Codex Security agent has contributed to over 3,000 critical and high-severity vulnerability fixes across the ecosystem since launch, and that its Codex for Open Source program now provides free security scanning to more than 1,000 open-source projects. Anthropic, Semgrep, and several other vendors have shipped comparable capabilities. Whether you build on a commercial offering or assemble an internal pipeline, the workflow is what matters. One nuance worth knowing about. Standard commercial models often refuse legitimate dual-use security queries by policy. Binary reverse engineering, exploit reasoning, malware analysis. If your AppSec team has been telling you that AI tools “do not work for security,” this refusal threshold is usually the reason. Both Anthropic’s Glasswing program and OpenAI’s Trusted Access for Cyber, expanded on April 14, 2026, to thousands of verified individual defenders, exist precisely to provide a lower refusal threshold for verified defensive use cases. Enterprise procurement and legal teams should start the verification paperwork now, not after a need arises. The Supply Chain Is the Other Half of the Data Exposure Problem Two recent incidents are worth holding in mind whenever this conversation comes up. On September 8, 2025, eighteen widely used npm packages, including chalk, debug, and ansi-styles, were trojanized after a phishing campaign targeting the maintainer known as qix. Those eighteen packages collectively account for over 2.6 billion weekly downloads. The malicious versions were live for roughly two hours and were written to drain cryptocurrency wallets, but the same access could have been used to exfiltrate environment secrets, build credentials, or sensitive data from any CI pipeline that pulled the bad version during that window. Palo Alto Networks Unit 42 and others published detailed breakdowns: paloaltonetworks.com on the qix incident. A week later, on September 15, 2025, the Shai-Hulud worm became the first self-propagating malware in the npm ecosystem, compromising hundreds of packages in its initial wave and continuing to evolve through follow-on campaigns into late 2025 and early 2026. The malware integrated TruffleHog to scan for secrets in compromised environments, harvested credentials from cloud instance metadata services where available, and weaponized GitHub Actions workflows for persistence. Palo Alto Networks Unit 42, ReversingLabs, Wiz, and others have continued to track variants of the same family. The reason these matter for a data security conversation is that the attacker's objective in both cases was credentials and secrets in build environments. From there, the path to data is short. A compromised CI runner with cloud credentials can read whatever those credentials can read. A compromised GitHub token can read whatever the org allows. A compromised npm publish token can introduce a future payload that does both. Treat the build pipeline as a data security boundary, not just an engineering productivity surface. A dependency firewall that validates package provenance before installation (Sonatype Nexus Firewall, JFrog Xray, Socket.dev, or equivalents) is the highest-leverage single control I know of for closing this attack surface. The Shadow Agent Problem Is a DLP Problem in Disguise The single most striking statistic in the April 2026 CSA research, to me, was that eighty-two percent of organizations had discovered previously unknown AI agents in their environment over the past year, and forty-one percent had discovered them more than once. The agents most commonly emerged in internal automation and scripting environments, in custom assistants and plugins built on LLM platforms, in SaaS tools with built-in automation, and in developer-created workflows. This is, structurally, the same problem that shadow IT was a decade ago, and the same problem that shadow SaaS became five years ago. The difference is that the average shadow agent has read access to more sensitive data than the average shadow application ever did, because agents are useful precisely in proportion to how much context they can reach. A finance team’s reconciliation agent, helpfully built in an afternoon, often ends up with broader visibility into financial data than the human who built it. A customer support copilot frequently has a service account with access to the entire ticket database, including PII. None of this is malicious. It is the path of least resistance for getting an agent to do something useful. Three controls help close the gap, and they are mostly extensions of capabilities a mature data security team already owns. CASB and SSPM coverage of LLM and agent platforms. The platforms hosting these agents (custom GPTs, Copilot Studio, internal MCP servers, vendor copilots) are SaaS applications. They need posture management, sanctioned application policies, and inline data protection just as much as Salesforce or Workday do. Most CASB and SSPM deployments are still catching up here. Push your vendor.Inline DLP on prompt and completion traffic. The point at which sensitive data leaves the environment is increasingly the prompt itself. Inline data inspection at the LLM gateway, using the same content matchers (EDM, IDM, OCR, vector ML) you trust for email and endpoints, is the right architectural pattern. The vendors are building this, but few enterprises have it deployed.An agent registry, even a basic one. Until the agent population is enumerable, no policy applied to it is provable. A spreadsheet is fine to start. The goal is to be able to answer, on any given Monday, three questions: which agents exist in production, what data each one can read, and who is the human owner of each. CSA’s data shows that most enterprises cannot answer those questions today. What I Would Actually Start on This Week Comprehensive ninety-day plans tend to lose momentum after the first two weeks of execution. The more effective approach, which I have refined over years of operationalizing data security programs at enterprise scale, is a focused set of starting moves that can ship in two weeks and that compound into a larger program over the quarter. Run an inventory pass for AI agents and copilots in your environment. Spreadsheet is fine. Capture name, owner, data scope, and approval status. The goal is to convert the CSA shadow agent statistic from an industry survey number into a number you actually have for your own organization.Review the data scope of every service account and OAuth grant tied to an LLM, agent, or copilot platform. Most of them were sized for development convenience, not production. Tighten the ones that need tightening. Decommission the ones that are no longer in active use.Pilot a model-based reviewer in the pull request path of one codebase. Measure the false positive rate and developer satisfaction at week four. If the numbers are reasonable, expand. If they are not, tune and try again.Add provenance metadata to your data classification taxonomy. Even if the only label you can ship this quarter is “generated by an AI system,” shipping it now is more valuable than waiting for a perfect schema. Tagging at ingestion is the part of the program that compounds, and it has been undersized for the agent era at most enterprises I have seen.Open the verified access conversation with your AI vendors. Anthropic Glasswing, OpenAI Trusted Access for Cyber, and equivalent programs from other providers offer pathways to models with reduced refusal thresholds for legitimate defensive work. The application process involves coordination with General Counsel and procurement, which is why initiating it before an urgent need is critical. Programs of this kind will become foundational infrastructure for enterprise security teams over the next two years. These moves represent the structural transition that enterprise data security programs need to make over the next eighteen months. Programs that begin this work now will spend that window refining the controls and integrating them across their existing security stack. Programs that delay will spend the same window writing postmortems that explain why the controls were not in place. Conclusion The cybersecurity industry has navigated several genuine inflection points over the past decade, and the current moment qualifies as one of them on a specific structural ground: the cost curve for finding software flaws has bent, while the cost curve for shipping patches has not. The gap between those two curves is where every enterprise security program now operates, and the consequences land first at the data layer, which is where my work has been concentrated for the past decade. Data security teams that internalize this framing now will spend 2026 building defensible programs around a fundamentally changed threat economy. Teams that wait for a more dramatic signal will spend the same period responding to incidents that the structural shift made predictable.

By Priyanka Neelakrishnan

Top Java Security Vulnerabilities and How to Prevent Them in Modern Java

With the increasing number of security threats, organizations have invested heavily in cybersecurity initiatives to protect their applications, infrastructure, and sensitive data. Security vulnerabilities are rarely introduced intentionally. Most of them creep into applications through shortcuts, overlooked edge cases, outdated libraries, or some bad coding habits. Modern Java has significantly improved its security capabilities, but no framework or JVM version can completely protect an application from insecure coding practices. As developers, we still need to understand where vulnerabilities originate and how to prevent them before they reach production. In this article, I am trying to summarize some of the most common Java security vulnerabilities and practical techniques used to prevent them. These are the same security best practices and lessons learned that I frequently share with new team members joining my team. I am sharing them here in the hope that they can serve as a practical handbook for Java developers looking to build more secure applications. 1. SQL Injection SQL injection remains one of the oldest and most dangerous vulnerabilities. It occurs when user input is directly concatenated into SQL statements. Consider the following example: Java String query = "SELECT * FROM users WHERE username = '" + username + "'"; Statement stmt = connection.createStatement(); ResultSet rs = stmt.executeQuery(query); If an attacker enters, the query can be manipulated to return unintended results. SQL admin' OR '1'='1 Prevention Always use parameterized queries. Java String query = "SELECT * FROM users WHERE username = ?"; PreparedStatement stmt = connection.prepareStatement(query); stmt.setString(1, username); ResultSet rs = stmt.executeQuery(); Prepared statements separate data from executable SQL, eliminating injection opportunities. 2. Hardcoded Secrets One of the most common findings during security reviews is hardcoded credentials. Java private static final String API_KEY = "abcd123456789"; This may seem harmless during development, but once committed to source control, secrets often remain exposed indefinitely. Prevention Store secrets externally. SQL String apiKey = System.getenv("PAYMENT_API_KEY"); Better alternatives are to include it in AWS Secrets Manager, Azure Key Vault, HashiCorp Vault, or Kubernetes Secrets. Secrets should never live inside source code repositories. 3. Insecure Deserialization Java serialization has been responsible for numerous security incidents. Example: Java ObjectInputStream input = new ObjectInputStream(request.getInputStream()); Object obj = input.readObject(); The danger is that attackers can craft malicious serialized objects that execute unexpected code during deserialization. Prevention Avoid Java serialization whenever possible. Prefer formats such as JSON, XML (with secure parsing), or Protocol Buffers. Example using Jackson: Java ObjectMapper mapper = new ObjectMapper(); User user = mapper.readValue(json, User.class); Using structured formats reduces attack surfaces significantly. 4. Cross-Site Scripting (XSS) Although often associated with front-end applications, backend services can accidentally enable XSS vulnerabilities when user-generated content is returned without sanitization. Example: Java String comment = request.getParameter("comment"); response.getWriter().write(comment); If the user submits, the browser executes the script. HTML <script>alert('Hacked')</script> Prevention Always encode output. Using Spring: Java String safeComment = HtmlUtils.htmlEscape(comment); Additionally, validate inputs, sanitize rich text, and implement Content Security Policies (CSP). 5. Path Traversal Attacks File download functionality often introduces path traversal vulnerabilities. Example: Java String file = request.getParameter("file"); Path path = Paths.get("/documents/" + file); An attacker could submit and potentially access sensitive files. Shell ../../../etc/passwd Prevention Normalize and validate paths. Java Path base = Paths.get("/documents"); Path resolved = base.resolve(file).normalize(); if (!resolved.startsWith(base)) { throw new SecurityException( "Invalid file path"); } Never trust file names coming directly from user input 6. Weak Password Storage Storing passwords improperly remains surprisingly common. Bad practice: Java String passwordHash = DigestUtils.md5Hex(password); MD5 and SHA-1 are no longer considered secure for password storage. Prevention Use adaptive hashing algorithms. Example with BCrypt: Java BCryptPasswordEncoder encoder = new BCryptPasswordEncoder(); String hash = encoder.encode(password); BCrypt automatically includes salting and work-factor adjustments. Other strong alternatives include Argon2, PBKDF2 or SCrypt 7. Dependency Vulnerabilities Modern Java applications often contain more third-party code than custom code. A secure application can still become vulnerable because of outdated dependencies. Prevention Integrate dependency scanning into CI/CD pipelines. Example Maven plugin: XML <plugin> <groupId>org.owasp</groupId> <artifactId>dependency-check-maven</artifactId> </plugin> Additionally, tools such as Snyk can automatically identify known vulnerabilities. We have been using Snyk for the last couple of years, and it is effective. Regular dependency updates should be part of every release cycle. 8. Improper Logging of Sensitive Data Developers often log information for troubleshooting without considering security implications. Example: Java logger.info( "Login request received for user={} password={}", username, password); This exposes credentials inside log files. Prevention Mask or exclude sensitive information. Java logger.info( "Login request received for user={}", username); Never log passwords, access tokens, credit card information, Personal health information (PHI), or PII information. This is especially important in regulated industries such as healthcare, like ours. 9. Insufficient Authentication and Authorization Authentication verifies identity, and authorization determines access. Many applications perform authentication correctly but fail to enforce authorization consistently. Example: Java @GetMapping("/admin/users") public List<User> getUsers() { return userService.findAll(); } Without authorization checks, any authenticated user might gain access. Prevention Use role-based security. Java @PreAuthorize("hasRole('ADMIN')") @GetMapping("/admin/users") public List<User> getUsers() { return userService.findAll(); } Security should be enforced at every layer, not just the UI. 10. Lack of Input Validation Many vulnerabilities originate from accepting unexpected input. Example: Java String age = request.getParameter("age"); int userAge = Integer.parseInt(age); Invalid input can cause exceptions or unexpected behavior. Prevention Validate all external input. Java @Min(18) @Max(120) private Integer age; Bean Validation provides a simple and consistent approach for validating request payloads. Never assume user input is safe. Final Thoughts Security is not a feature that can be added at the end of a project. It needs to be part of the development process from the very beginning. The vulnerabilities discussed here are not theoretical. They are among the most common findings during security assessments, penetration tests, and production incident investigations. Fortunately, modern Java provides mature frameworks, libraries, and tools that make secure development significantly easier than it was a decade ago. The key is building security awareness into everyday development practices: Use parameterized queriesProtect secrets properlyValidate all inputsKeep dependencies updatedApply strong authentication and authorizationLog responsiblyContinuously scan for vulnerabilities Security is ultimately about reducing risk. Small improvements applied consistently across a codebase can prevent incidents that would otherwise become expensive lessons later.

By Muhammed Harris Kodavath

Data

DZone's Featured Data Resources

Top Data Experts

The Latest Data Topics