Geospatial Data: Apache Spark vs. PostGIS
Geospatial Data: Apache Spark vs. PostGIS
We take a look at how the big data tool Apache Spark stacks up against the geospatial tool PostGIS when it comes to handling big data sets.
Join the DZone community and get the full member experience.
Join For FreeThe Architect’s Guide to Big Data Application Performance. Get the Guide.
In my company, we've had a heated debate about geospatial data processing between old school conservative SQL fans and progressive big data and NoSQL fans.
Data Querying (MongoDB vs. PostGIS)
This section provides an overview of PostGIS and Mongodb, and their geospatial capabilities.
| Mongodb | Postgis | |
|---|---|---|
| Spark drivers | spark:mongo | spark:jdbc |
| Scalability |
|
|
| Response time |
|
|
| Geospatial index and Geoqueries |
|
|
Spark vs. PostGIS (Performance)
The purpose of this section is to compare the performance Spark and PostGIS with respect to different data analyses (max, avg, geospatial:within, etc.).
For PostGIS tests, the data is already preprocessed and indexed geospatially, while Spark will use directly raw data (parquet, csv, shape, etc.).
As the data size grows, Spark's response time remains stable while PostGIS's response time grows exponentially.
Max KPI:
Mean KPI:
KPIs Within a Bounding Box:
Learn how taking a DataOps approach will help you speed up processes and increase data quality by providing streamlined analytics pipelines via automation and testing. Learn More.
Like This Article? Read More From DZone
Opinions expressed by DZone contributors are their own.
{{ parent.title || parent.header.title}}
{{ parent.tldr }}
{{ parent.linkDescription }}
{{ parent.urlSource.name }}