Engineering High-Scale Real Estate Listings Systems Using Golang, Part 1

In high-scale real estate platforms, the real challenge isn’t just about fetching listings, it’s about processing and serving millions of records across diverse MLS (Multiple Listing Service) providers with speed, resilience, and cost-efficiency. Whether you’re integrating with the United States’ BrightMLS or Canada’s TRREB, each MLS comes with its own quirks: varying data models, inconsistent metadata, irregular update cycles, and evolving schemas.

At scale, those differences are edge cases no longer; they’re daily roadblocks. Your backend must not only ingest and normalize terabytes’ worth of listing data but also cope with real-time sync, deduplication, tagging, scoring, and advanced filtering, bottleneck-free. For our team at a high-flying Vancouver-based real estate startup, the answer was to design a system in Go (Golang) that would tolerate high concurrency, have hard memory limits, and operate under realistic production pressure.

In a bid to create some sort of order out of this chaos, industry bodies developed RETS (Real Estate Transaction Standard) and, more recently, RESO (Real Estate Standards Organization). RETS was a sincere effort at data exchange standardization, while RESO went a step further by utilizing RESTful APIs and a centralized Data Dictionary. However, standard adoption is still incomplete, and handling raw MLS data still necessitates strong normalization layers, schema flexibility, and domain-specific reasoning.

This entry is part one of an even more detailed series that walks you through our system design decisions, development practices, and Go-based architecture that we use to solve these problems. In this article, we first deconstruct the nucleus standards, RETS, and RESO Web API, not as history but as the necessary precepts for what comes next.

Real Estate Data Standards: RETS, RESO, and the Quest for Consistency

One of the most persistent problems in building scalable real estate listing infrastructure is handling inconsistencies in data across various MLS providers. While the industry has made a stab at introducing standard measures, the on-the-ground reality is far from uniform. Two major attempts at solving this problem have been RETS and RESO, each offering a vision of homogeneity that is achieved only half-heartedly in reality.

The Real Estate Transaction Standard, or RETS, came about around the early 2000s as a means to impose order on accessing MLS data. It employed XML and had its own query language, allowing for more dynamic extraction of data than previous methods like FTP-based flat files. By its day, RETS was a genuine step forward. However, it was difficult to maintain over time. There were very diverse implementations, and inconsistencies between servers became the norm. Also, the XML payloads were heavy and cumbersome, and lacking a standard authentication mechanism caused friction. With the introduction of new web standards, RETS began to look outdated and rigid, especially for engineers who had to deal with JSON and RESTful APIs.

Looking at these constraints, the Real Estate Standards Organization (RESO) introduced the RESO Web API as an alternative to RETS. This new spec takes advantage of modern web conventions, using JSON for structuring data, OAuth 2.0 for auth, and OData for querying. The aim was to have more open real estate data that is developer-friendly and malleable. Ideally, RESO offers a less clunky and more standardized interface for MLS data. In practice, adoption has been unbalanced. The majority of MLS providers lag behind in the implementation of or publishing APIs that only partially fulfill the RESO standard. Even those providers who claim RESO compliance have different field definitions, naming conventions, and metadata available. This discrepancy between the theoretical standard and the practical implementation experience is in presenting these feeds.

In my work, I’ve had to integrate data from several major providers, including CREA and MRED. While standards like RESO provide a useful foundation, we’ve found it essential to build custom parsers, transformation pipelines, and validation layers for each provider. These components act as a buffer between the raw, inconsistent data and the rest of our system. They allow us to maintain a consistent internal schema and dissociate upstream anomalies. The reality is that no standard can ever replace the need for good engineering practices when working with broken and dynamic data environments.

Lastly, RETS and RESO are part of a long-range plan to impose order on a chaotic world. They are not silver bullets but provide the industry with a common language and path forward. Real estate platform creators need to learn these standards, but most importantly, they need to be receptive to adapting to their edges with elasticity and persistence.

RETS vs. RESO

When engineering scalable systems for real estate listings, one of the core challenges is handling integrations with Multiple Listing Services (MLS). These providers operate on different standards, and if you’re building in Go, the contrast between RETS and RESO isn’t just technical, it’s deeply practical.

RETS (Real Estate Transaction Standard) is an older standard pre-dating current RESTful APIs. It was introduced for the sake of standardizing access to MLS but ended up as a universe of diverse implementations. RESO Web API, on the other hand, is a newer JSON-based RESTful standard that plays well with current service design and tooling, especially in a language like Go.

Using RETS as a Go developer is like reverse-engineering an old machine with no manual. It uses HTTP Digest Authentication, XML payloads, and a query language called DMQL. Go lacks a RETS SDK, so you either create raw HTTP integrations or use brittle wrappers. On Roomvu, when we began to integrate REBGV’s RETS feed, our first step was to create a custom microservice in Go that constructed DMQL queries and processed RETS metadata. Column names like SqftTotFin or Status were not documented, and even minor modifications on their side would silently break our pipeline.

Our service had to request data in batches, download media on their own, and retry failed photo downloads. The most difficult part wasn’t the lack of SDK support; it was the inconsistency. Some listings would return valid metadata and zero images, for example, and we would have no idea unless we included post-validation checks. Debugging at runtime became the norm. The system worked, but was brittle.

Conversely, integrating with RESO was effortless. With OAuth2 authentication and field names that were standardized, building an integration was a walk in the park. We defined a Go struct with tagged fields and invoked json.Unmarshal directly. Pagination, filtering, and even change tracking with delta syncs all just worked. This enabled us to build robust synchronization services, add caching layers, and poll ETags to observe updates successfully.

With RESO, the listing normalization layer went down by orders of magnitude. One of our recent integrations with a US provider using RESO reduced naming and structure consistency for field naming by half or more. We no longer had to run an independent normalization job just to normalize the concept of “square footage” or “status.”

Despite these benefits, RESO adoption still falls behind with older boards. RETS is still used, often because the providers haven’t converted or don’t see a short-term gain. For developers using Go, this coexistence means that your system needs to be able to support both paradigms at the same time. We preferred to encapsulate these as separate services under a common interface so we could maintain cleaner boundaries while diverting traffic to RESO-based integrations over time as they were released.

Lastly, you’ll need to support both standards if your business is active on more than one board. RETS will challenge your engineering, while RESO will reimburse you with convenience. On a Go platform that values clarity, performance, and concurrency, RESO is clearly the better choice, but RETS is the legacy you can’t yet relinquish.

RETS Fundamentals

RETS was designed to standardize the transfer of real estate listing information between MLS systems and third-party software. Although many MLS vendors are dropping RETS for more modern solutions, it is still deeply embedded within the design of most older systems. As such, an understanding of how RETS works remains essential for developers building scalable integrations in the real estate marketplace.

At its most basic, RETS is a session-based, XML-over-HTTP protocol. Unlike newer RESTful APIs, though, RETS sessions are initiated with a login that returns a token and establishes permanent access via cookies. authenticated developers can thereafter fetch metadata, which defines the data structure that’s available, anything from resources like Property or Agent, down to their own classes and fields. This kind of metadata must be parsed programmatically to understand how to construct syntactically valid queries or how to demultiplex response forms.

One of RETS’ most defining features is querying listing data through its utilization of the Search endpoint. A simple search query specifies a resource (e.g., Property), a class (e.g., Residential), and a query filter in C-style syntax as for instance, (Status=|A) to return active listings. Responses are in either compact or full XML, and developers should be capable of supporting both, depending on the MLS server setup.

Incremental syncing is yet another basic RETS concept. Clients can use fields like ModificationTimestamp to download only listings that have changed since their last run, thus providing efficient synchronizing. RETS also enables media downloading, in addition to listing information. This is made possible by the GetObject endpoint, which is capable of batch downloading listing photos, virtual tours, and other media objects.

But RETS is not easy to work with. MLSs are extremely inconsistent in using the standard. Field names, lookup values, and even metadata schemas differ dramatically, so your code will frequently need to be extremely flexible and configurable. Rate limits, session timeouts, and inconsistent server behavior are also common impediments.

While RETS itself is being slowly replaced by newer, developer-focused alternatives like the RESO Web API, it’s nevertheless a vital part of the real estate data landscape. For any engineer tasked with building an enterprise-scale, stable listings site, especially one that needs to interact with multiple MLS providers, understanding the inner workings and quirks of RETS is very much still part of the brief.

Embracing the Modern Trends: RESO Web API in Real Estate Development

As the real estate industry further evolves online, the RESO Web API is the digital replacement for RETS, providing a time of more interoperable, scalable, and developer-friendly access to several MLS systems. The Real Estate Standards Organization (RESO) developed and maintains this API standard to offer a more RESTful, secure, and consistent method of accessing real estate data on different MLS systems.

At its heart, the RESO Web API relies on open web standards like REST, OData, and JSON. That immediately lowers the bar for developers, who can now take to heart familiar models like URL-based access to resources, HTTP verbs, and stateless architecture to make integrations. The shift from RETS to RESO Web API is more than a technical upgrade; it’s an embrace of consistency, scalability, and performance.

The Power of OData

Another characteristic of the RESO Web API is its use of OData (Open Data Protocol), a protocol for queries and updates against data. OData allows developers to form richly expressive queries with filters, pagination, sorting, and projections directly in the URL. For example:

GET /Property?$filter=ModificationTimestamp gt 2024-01-01T00:00:00Z&$orderby=ModificationTimestamp desc&$top=100

This means that complex data queries are readable by humans but are highly compatible with contemporary web infrastructure and cloud services.

Compared to RETS, which often employed session-based cookies, RESO Web API takes the route of OAuth 2.0 for authentication and authorization. This current approach is secure token-based access and is more microservices and cloud-native-friendly. Individual MLS can employ its own OAuth provider or utilize centralized identity systems, hence offering flexibility without compromising compatibility with new security standards.

The second major advancement is metadata standardization. RESO provides a data dictionary, a detailed specification that defines field names, types, and enumerations (such as property types, statuses). That is, when you build an integration against one RESO-compliant MLS, your code is far more portable to others a far cry from the fragmented world of RETS.

RESO also allows for more efficient delta syncs. Developers can use ModificationTimestamp or LastUpdatedTimestamp fields to retrieve only changed records, both conserving bandwidth and processing. Though polling remains an option, some MLSs are beginning to support event-based or push patterns with webhooks or streaming APIs, allowing real-time updates at scale.

Conclusion

Understanding the underlying standards, RETS and RESO Web API, is vital for any solid developer who wants to develop scalable and stable real estate systems. While these protocols were developed to bring order and uniformity to MLS data, the reality is that there are still inconsistencies, and scalability really depends on how well your system can accept, normalize, and adapt to them.

By embracing Go for its performance and concurrency benefits, and by designing with resilience and flexibility in mind, we’ve been able to build systems that not only keep up with the volume and velocity of MLS data but are also ready for what’s next.

In the next part, we’ll go beyond definitions and dive into the technical architecture, data pipelines, and key implementation strategies that power our listing engine in production.