SAS Integration With ArcGIS Online in 3-Tier Architecture

Introduction

SAS has powerful tools for statistical computing along with the Consumer and Business Intelligence suite of products, which has been used in various fields like clinical research, finance, and healthcare. SAS also has Fraud Detection products that are based on machine learning and artificial intelligence, which are becoming handy to prevent fraud across multiple industries.

On the other hand, ArcGIS Online is a cloud-based software-as-a-service (SaaS) for web mapping and Geographic Information System. ArcGIS is the market-leading Geographic Information System for arranging geographical data in order to create and use maps in workflow-specific apps. When spatial data analysis capability of ArcGIS is added to SAS textual and numeric data analysis, the efficiency of the users who are using the system will improve as the users will now have immediate recognition and understanding of the results.

This article explains how SAS software, when deployed in a 3-tier architecture, can be integrated with external systems like ArcGIS online or any other web service for that matter.

You may also like:  Descriptive Analysis With SAS

SAS 3-Tier Architecture

SAS deployment primarily comprises of three components — SAS Metadata Server, SAS Compute Grid and SAS Web Application Server. In an environment that is based on a 3-tier architecture, SAS Metadata Server and Compute Server are generally deployed to data tier whereas SAS Web Application Server is deployed to Application tier with a web server acting as a proxy in the presentation tier.

This makes integration of SAS with an external system like ArcGIS Online very complex since components in data tiers need to communicate directly to the URL that might be on the internet. Most of the organizations don’t like to open the communication to the outside world directly from the data tier due to security reasons.

When a user submits a SAS program through SAS Enterprise Guide or SAS Web Applications, the submitted program will be executed on the SAS Grid server that resides in Data Tier. Due to this reason, any SAS Program that is expected to communicate with external web services cannot be executed successfully by SAS Grid Server. For example, below SAS program extracts the dataset of all Earthquakes that are larger than magnitude 5.0 and tag the coordinates with ArcGIS mapping service. Since Data Tier will not have access to services.arcgisonline.com, the program execution will not be successful.

DATA major_quakes;
SET sashelp.quakes (WHERE=(magnitude>5.0));
RUN;
PROC SGMAP ;
ESRIMAP URL= 'https://services.arcgisonline/arcgis/rest/services/World_Topo_Map';
TITLE 'Earthquakes > 5.0 magnitude';
BUBBLE /
FILLATTRS=();
RUN;

Solution

SAS deployment comes with a Web Server that is built on Apache and a Tomcat Web Application server. The solution is to use the web servers in each tier as a proxy and send requests to the external web service from the presentation tier.

Changes to Application Tier SAS Web Server's httpd.conf

When the SAS Compute Grid Server sends any requests to the Application Tier's SAS Web Server for /arcgis context, the requests need to be forwarded to the Presentation tier's load balancer. In order for that to happen, ProxyPass and ProxyPassReverse settings need to be updated in httpd.conf of Application Tier's SAS Web Server.

ProxyPass /arcgis http://<saspresentation.loadbalancer.host>:<saspresentation.outgoing.port>/arcgis

ProxyPassReverse /arcgis http://<saspresentation.loadbalancer.host>:<saspresentation.outgoing.port>/arcgis

Since the web servers are being used for incoming as well as for outgoing traffic, ports for these purposes must be different.

Converting HTTP traffic to HTTPS

The traffic that is originating from Application Tier consists of HTTP requests but the communication with ArcGIS online must be over HTTPS. For this reason, ArcGIS Online website certificate is required to be installed on the Presentation Tier's Http Server for a successful handshake.

In our example, the HTTP server that resides in Presentation Tier is an IBM IHS server. I have used the IBM Key Management tool to import the certificate to the key store and placed the keystore files in the IHS server to enable HTTPS communication.

https://www.ibm.com/support/knowledgecenter/en/SSD29G_2.0.0/com.ibm.swg.ba.cognos.tm1_inst.2.0.0.doc/t_pa_use_ibmkeymgmt.html

• /opt/IHS/IHS85/IBMIHS/certs/key.kdb
• /opt/IHS/IHS85/IBMIHS/certs/key.sth

Changes to Presentation Tier Web Server's httpd.conf

The IHS server cluster will have to handle both incoming HTTP requests and the Outgoing HTTPS request simultaneously. All the existing Proxy and ReverseProxy settings for SAS incoming requests will need to be defined on a VirutalHost as below to isolate the incoming and the outgoing communications.

<VirtualHost *:2073>
ServerName presentationtierwebserver
ErrorLog "logs/sas-error.log"
ProxyPass /SASBIPortlets http://applicationtierloadbalancer:7981/SASBIPortlets
ProxyPassReverse /SASBIPortlets http://applicationtierloadbalancer:7981/SASBIPortlets
ProxyPass /SASLogon http://applicationtierloadbalancer:7981/SASLogon
ProxyPassReverse /SASLogon http://applicationtierloadbalancer:7981/SASLogon
ProxyPass /SASGridManager http://applicationtierloadbalancer:7981/SASGridManager
ProxyPassReverse /SASGridManager http://applicationtierloadbalancer:7981/SASGridManager
ProxyPass /SASContentServer http://applicationtierloadbalancer:7981/SASContentServer
ProxyPassReverse /SASContentServer http://applicationtierloadbalancer:7981/SASContentServer
ProxyPass /SASPrincipalServices http://applicationtierloadbalancer:7981/SASPrincipalServices
ProxyPassReverse /SASPrincipalServices http://applicationtierloadbalancer:7981/SASPrincipalServices
ProxyPass /SASPackageViewer http://applicationtierloadbalancer:7981/SASPackageViewer
ProxyPassReverse /SASPackageViewer http://applicationtierloadbalancer:7981/SASPackageViewer
ProxyPass /SASWIPSoapServices http://applicationtierloadbalancer:7981/SASWIPSoapServices
ProxyPassReverse /SASWIPSoapServices http://applicationtierloadbalancer:7981/SASWIPSoapServices
ProxyPass /SASAdmin http://applicationtierloadbalancer:7981/SASAdmin
ProxyPassReverse /SASAdmin http://applicationtierloadbalancer:7981/SASAdmin
ProxyPass /SASPermissionManager http://applicationtierloadbalancer:7981/SASPermissionManager
ProxyPassReverse /SASPermissionManager http://applicationtierloadbalancer:7981/SASPermissionManager
ProxyPass /SASStudio http://applicationtierloadbalancer:7981/SASStudio
ProxyPassReverse /SASStudio http://applicationtierloadbalancer:7981/SASStudio
ProxyPass /SASFlexThemes http://applicationtierloadbalancer:7981/SASFlexThemes
ProxyPassReverse /SASFlexThemes http://applicationtierloadbalancer:7981/SASFlexThemes
ProxyPass /SASTemplateEditor http://applicationtierloadbalancer:7981/SASTemplateEditor
ProxyPassReverse /SASTemplateEditor http://applicationtierloadbalancer:7981/SASTemplateEditor
ProxyPass /SASBIDashboardEventGen http://applicationtierloadbalancer_Singleton1:7981/SASBIDashboardEventGen
ProxyPassReverse /SASBIDashboardEventGen http://applicationtierloadbalancer_Singleton1:7981/SASBIDashboardEventGen
ProxyPass /FolderModule http://applicationtierloadbalancer:7981/FolderModule
ProxyPassReverse /FolderModule http://applicationtierloadbalancer:7981/FolderModule
ProxyPass /SASPortal http://applicationtierloadbalancer:7981/SASPortal
ProxyPassReverse /SASPortal http://applicationtierloadbalancer:7981/SASPortal
ProxyPass /SASIdentityServices http://applicationtierloadbalancer:7981/SASIdentityServices
ProxyPassReverse /SASIdentityServices http://applicationtierloadbalancer:7981/SASIdentityServices
ProxyPass /SASLASRAuthorization http://applicationtierloadbalancer:7981/SASLASRAuthorization
ProxyPassReverse /SASLASRAuthorization http://applicationtierloadbalancer:7981/SASLASRAuthorization
ProxyPass /SASStoredProcess http://applicationtierloadbalancer:7981/SASStoredProcess
ProxyPassReverse /SASStoredProcess http://applicationtierloadbalancer:7981/SASStoredProcess
ProxyPass /SASEnterpriseMinerServices http://applicationtierloadbalancer:7981/SASEnterpriseMinerServices
ProxyPassReverse /SASEnterpriseMinerServices http://applicationtierloadbalancer:7981/SASEnterpriseMinerServices
ProxyPass /SASBIDashboard http://applicationtierloadbalancer:7981/SASBIDashboard
ProxyPassReverse /SASBIDashboard http://applicationtierloadbalancer:7981/SASBIDashboard
ProxyPass /SASSharedApps http://applicationtierloadbalancer:7981/SASSharedApps
ProxyPassReverse /SASSharedApps http://applicationtierloadbalancer:7981/SASSharedApps
ProxyPass /SASVisualAnalyticsTransport http://applicationtierloadbalancer:7981/SASVisualAnalyticsTransport
ProxyPassReverse /SASVisualAnalyticsTransport http://applicationtierloadbalancer:7981/SASVisualAnalyticsTransport
ProxyPass /SASBackupManager http://applicationtierloadbalancer:7981/SASBackupManager
ProxyPassReverse /SASBackupManager http://applicationtierloadbalancer:7981/SASBackupManager
ProxyPass /SASWorkflowServices http://applicationtierloadbalancer:7981/SASWorkflowServices
ProxyPassReverse /SASWorkflowServices http://applicationtierloadbalancer:7981/SASWorkflowServices
ProxyPass /SASWebReportStudio http://applicationtierloadbalancer:7981/SASWebReportStudio
ProxyPassReverse /SASWebReportStudio http://applicationtierloadbalancer:7981/SASWebReportStudio
ProxyPass /SASBIWS http://applicationtierloadbalancer:7981/SASBIWS
ProxyPassReverse /SASBIWS http://applicationtierloadbalancer:7981/SASBIWS
ProxyPass /SASSecurityAdminModule http://applicationtierloadbalancer:7981/SASSecurityAdminModule
ProxyPassReverse /SASSecurityAdminModule http://applicationtierloadbalancer:7981/SASSecurityAdminModule
ProxyPass /PlatformWebServices http://applicationtierloadbalancer:7981/PlatformWebServices
ProxyPassReverse /PlatformWebServices http://applicationtierloadbalancer:7981/PlatformWebServices
ProxyPass /SASEnvironmentMgrLibraryMod http://applicationtierloadbalancer:7981/SASEnvironmentMgrLibraryMod
ProxyPassReverse /SASEnvironmentMgrLibraryMod http://applicationtierloadbalancer:7981/SASEnvironmentMgrLibraryMod
ProxyPass /SASWebDoc http://applicationtierloadbalancer:7981/SASWebDoc
ProxyPassReverse /SASWebDoc http://applicationtierloadbalancer:7981/SASWebDoc
ProxyPass /SASTheme_default http://applicationtierloadbalancer:7981/SASTheme_default
ProxyPassReverse /SASTheme_default http://applicationtierloadbalancer:7981/SASTheme_default
ProxyPass /SASThemeDesignerForFlex http://applicationtierloadbalancer:7981/SASThemeDesignerForFlex
ProxyPassReverse /SASThemeDesignerForFlex http://applicationtierloadbalancer:7981/SASThemeDesignerForFlex
ProxyPass /SASJSR168RemotePortlet http://applicationtierloadbalancer:7981/SASJSR168RemotePortlet
ProxyPassReverse /SASJSR168RemotePortlet http://applicationtierloadbalancer:7981/SASJSR168RemotePortlet
ProxyPass /SASEnvironmentMgrMidTier http://applicationtierloadbalancer:7981/SASEnvironmentMgrMidTier
ProxyPassReverse /SASEnvironmentMgrMidTier http://applicationtierloadbalancer:7981/SASEnvironmentMgrMidTier
ProxyPass /SASWIPServices http://applicationtierloadbalancer:7981/SASWIPServices
ProxyPassReverse /SASWIPServices http://applicationtierloadbalancer:7981/SASWIPServices
ProxyPass /SASDeploymentBackup http://applicationtierloadbalancer:7981/SASDeploymentBackup
ProxyPassReverse /SASDeploymentBackup http://applicationtierloadbalancer:7981/SASDeploymentBackup
ProxyPass /SASUserModule http://applicationtierloadbalancer:7981/SASUserModule
ProxyPassReverse /SASUserModule http://applicationtierloadbalancer:7981/SASUserModule
ProxyPass /SASEnterpriseMinerJWS http://applicationtierloadbalancer:7981/SASEnterpriseMinerJWS
ProxyPassReverse /SASEnterpriseMinerJWS http://applicationtierloadbalancer:7981/SASEnterpriseMinerJWS
ProxyPass /SASVisualAnalytics http://applicationtierloadbalancer:7981/SASVisualAnalytics
ProxyPassReverse /SASVisualAnalytics http://applicationtierloadbalancer:7981/SASVisualAnalytics
ProxyPass /SASWIPClientAccess http://applicationtierloadbalancer:7981/SASWIPClientAccess
ProxyPassReverse /SASWIPClientAccess http://applicationtierloadbalancer:7981/SASWIPClientAccess
ProxyPass /SASServerModule http://applicationtierloadbalancer:7981/SASServerModule
ProxyPassReverse /SASServerModule http://applicationtierloadbalancer:7981/SASServerModule
ProxyPass /SASPreferences http://applicationtierloadbalancer:7981/SASPreferences
ProxyPassReverse /SASPreferences http://applicationtierloadbalancer:7981/SASPreferences
ProxyPass /SASAuthorizationServices http://applicationtierloadbalancer:7981/SASAuthorizationServices
ProxyPassReverse /SASAuthorizationServices http://applicationtierloadbalancer:7981/SASAuthorizationServices
ProxyPass /SASWorkflowWebServices http://applicationtierloadbalancer:7981/SASWorkflowWebServices
ProxyPassReverse /SASWorkflowWebServices http://applicationtierloadbalancer:7981/SASWorkflowWebServices
ProxyPass /SASVisualAnalyticsAdministrator http://applicationtierloadbalancer:7981/SASVisualAnalyticsAdministrator
ProxyPassReverse /SASVisualAnalyticsAdministrator http://applicationtierloadbalancer:7981/SASVisualAnalyticsAdministrator
ProxyPass /sasweb http://applicationtierloadbalancer:7981/sasweb
ProxyPassReverse /sasweb http://applicationtierloadbalancer:7981/sasweb
ProxyPass /sas http://applicationtierloadbalancer:7981/sas
ProxyPassReverse /sas http://applicationtierloadbalancer:7981/sas
ProxyPass /PlatformWebServices http://applicationtierloadbalancer:7981/PlatformWebServices
ProxyPassReverse /PlatformWebServices http://applicationtierloadbalancer:7981/PlatformWebServices
</VirtualHost>

ProxyPass and ProxyPassReverse settings for the outgoing HTTPS requests will need to be defined in another Virtualhost configuration in httpd.conf.

<VirtualHost *:6443>
ServerName presentationtierwebserver
ErrorLog "logs/error.log"
SSLProxyEngine On
RequestHeader set Front-End-Https "On"
CacheDisable *
ProxyPass /arcgis https://services.arcgisonline.com/arcgis
ProxyPassReverse /arcgis https://services.arcgisonline.com/arcgis
RedirectMatch ^/$ http://presentationtierwebserver:6443/arcgis
</VirtualHost>

SSL library and the settings like keystore details must be updated to the httpd.conf file:

LoadModule ibm_ssl_module modules/mod_ibm_ssl.so

KeyFile /opt/IHS/IHS85/IBMIHS/certs/key.kdb
SSLStashFile /opt/IHS/IHS85/IBMIHS/certs/key.sth

Below are the modules that are enabled in the IHS server in order for the communication to work:

LoadModule authz_host_module modules/mod_authz_host.so
LoadModule cache_module modules/mod_cache.so
LoadModule authz_default_module modules/mod_authz_default.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule include_module modules/mod_include.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule headers_module modules/mod_headers.so
LoadModule unique_id_module modules/mod_unique_id.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule mime_module modules/mod_mime.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule cgid_module modules/mod_cgid.so
LoadModule dir_module modules/mod_dir.so
LoadModule actions_module modules/mod_actions.so
LoadModule alias_module modules/mod_alias.so

All the servers are required to be restarted after these changes are applied. The configuration changes can be tested by simply running a wget or curl command for the respective URLs on each of the servers.

Updates to SAS Program URL

Since the communication is now established to the external URL, the URL in SAS program will need to be updated to the app tier load balancer's host and outgoing traffic port in order to pass the requests to the HTTP server in Application Tier and subsequently to the Presentation Tier.

DATA major_quakes;
SET sashelp.quakes (WHERE=(magnitude>5.0));
RUN;
PROC SGMAP ;
ESRIMAP URL= 'https://<saswebapp.loadbalancer.dns>:<saswebapp.loadbanlancer.port>/arcgis/rest/services/World_Topo_Map';
TITLE 'Earthquakes > 5.0 magnitude';
BUBBLE /
FILLATTRS=();
RUN;

Results

After the successful execution of the above program, SAS results will show the map and the dataset as below.

SAS can be integrated with any number of external web services without adding new infrastructure components to the architecture with this solution.

Further Reading

A Visual Guide to SAS SQL Joins

GIS for Developers