The problem
Spatial analytics needed reliable ETL from Saudi governmental GIS sources, queries over Overture Maps' open map data, and map serving for rendering and visualisation.
What I did
- Orchestrated every data pipeline in Prefect, including ETL for GIS data from Saudi government sources.
- Queried Overture Maps with AWS Athena and DuckDB for spatial analytics.
- Served and visualised maps with MapProxy, MBTileServer, MapLibre and Kepler.gl.
Outcome
- Every Tabaqat data pipeline designed and managed in Prefect, plus a map-serving stack for rendering and visualisation.
-
Users and apps
- Spatial analytics
- MapLibre + Kepler.gl
-
Retrieval
- AWS Athena + DuckDB
-
Data
- Saudi government GIS sources
- Processed GIS data
- Overture Maps
-
Services
- MapProxy + MBTileServer
-
Pipelines
- Prefect ETL flows
Connections
- Saudi government GIS sources to Prefect ETL flows
- Prefect ETL flows to Processed GIS data
- Overture Maps to AWS Athena + DuckDB
- AWS Athena + DuckDB to Spatial analytics
- MapProxy + MBTileServer to MapLibre + Kepler.gl
Figure 1 Component view, three workstreams: Prefect ETL from government GIS sources; Athena and DuckDB queries over Overture Maps; tile servers behind web maps. Diagrams show the components named in public descriptions of the work. They are simplified, not complete system maps.
