Short walkthrough example of how the Exposures API was implemented using swagger tools in Python.
At the start we had a notion of the data we wanted to serve, a list of sources to get it from, and a good idea of what this data looked like with some example datasets in a PostgreSQL / PostGIS database. Based on this information a specification file was started prior to the existence of any ReSTful service provider, so going down the rabbit hole of the swagger eco-system seemed a sensible thing to do.
We started with a specification in swagger:
swagger: "2.0"
info:
title: "Environmental Exposures API"
version: "1.0.0"
contact:
name: "Michael J. Stealey"
email: "stealey@renci.org"
url: "http://renci.org"
#responsibleDeveloper: "Michael J. Stealey"
#responsibleOrganization: "RENCI"
description: "API for environmental exposure models for NIH Data Translator program"
termsOfService: "None Available"
host: exposures.renci.org
basePath: /v1
schemes:
- https
- http
paths:
/exposures:
get:
summary: "Get list of exposure types"
description: "Returns a list of all available exposure types"
produces:
- application/json
responses:
200:
description: "Exposure types"
schema:
type: array
items:
$ref: '#/definitions/exposure_type'
404:
description: "No exposure types found"
...
A couple early versions of the specification did make it into the smartAPI registry, but when copying an updated specification to the smart-api.info/editor it yielded no errors, however when the “Save” button is pressed it generates a pop-up error, even though the editor states “All changes saved”.
Reference: Error"<html><title>500: Internal Server Error</title><body>500: Internal Server Error</body></html>"
The swagger-editor offers an option to generate server stubs using swagger-codegen in many different languages. We chose to implement our Exposures API in Python and used the python-flask option.
The resultant code gives some suggestion as to how it “should” be implemented and made reference to a Connexion library on top of Flask.
And when you run the code as described you get this.
database
We had a pre-existing PostgreSQL/PostGIS database that we were making use of, so there was the need to generate the appropriate models for use by our API server.
The initial models.py file was generated using sqlacodegen
Need to add import statement to sqlacodegen/codegen.py
in order to account for geoalchemy2
from sqlalchemy.types import Boolean, String
import sqlalchemy
from geoalchemy2 import Geometry, Geography # <-- this
$ sqlacodegen --outfile models.py postgres://datatrans:somepassword@192.168.56.101:5432/bdtgreen
https
Even if we weren’t starting with formal authentication and authorization, we at minimum wanted to enable https for all of our services. The documentation for how to do this is straight forward enough for PostgreSQL, but when it came to doing this for the swagger recommended connexion package, this is what we found…
After some package reconnaissance we turned up a way to enable https using native gevent features which is an option in the connexion package. This involved a couple extra parameters and monkey patching.
app.py
:
#!/usr/bin/env python3
import connexion
from configparser import ConfigParser
from flask_cors import CORS
from gevent import monkey # <-- this
monkey.patch_all() # <-- this
if __name__ == '__main__':
parser = ConfigParser()
parser.read('ini/connexion.ini')
app = connexion.App(__name__, specification_dir='./swagger/', server=parser.get('connexion', 'server'))
app.add_api('swagger.yaml', arguments={'title': 'Environmental Exposures API'})
if not parser.get('connexion', 'certfile') or not parser.get('connexion', 'keyfile'):
app.run(port=int(parser.get('connexion', 'port')),
debug=parser.get('connexion', 'debug'))
else:
CORS(app.app)
app.run(port=int(parser.get('connexion', 'port')),
debug=parser.get('connexion', 'debug'),
keyfile=parser.get('connexion', 'keyfile'), # <-- this
certfile=parser.get('connexion', 'certfile') # <-- and this
)
As with most things our Exposures API is ever evolving and still requires much optimizing with respect to database queries to be more efficient / timely.
Docker
Finally we packaged everything up to be Docker distributable so that deployment could be made rapidly across multiple endpoints as needed to satisfy a reverse proxy load balancer.
The Environmental Exposures API is available in docker as mjstealey/exposures_api. There is an assumption that it will be used to connect to a PostgreSQL/PostGIS database pre-populated with the appropriate environmental exposures data tables.
Usage examples will be given in the context of the sample database included in this repository.
Example:
$ docker pull mjstealey/exposures_api
$ docker run -d \
--name api-server \
-e POSTGRES_HOST=backend.postgres.server \
-e POSTGRES_PORT=5432 \
-e POSTGRES_DATABASE=bdtgreen \
-e POSTGRES_USERNAME=datatrans \
-e POSTGRES_PASSWORD=somepassword \
-p 5000:5000 \
mjstealey/exposures_api
Once completed the exposures_api would be running at http://localhost:5000/v1/ui/#/
The exposure_api uses environment variables to set itself up, and these can be passed in either individually, or by use of an environment file.
Example environment file (exposures-api.env) with default values
CONNEXION_SERVER=
CONNEXION_DEBUG=True
API_SERVER_HOST=localhost
API_SERVER_PORT=5000
API_SERVER_KEYFILE=
API_SERVER_CERTFILE=
POSTGRES_HOST=backend
POSTGRES_PORT=5432
POSTGRES_DATABASE=bdtgreen
POSTGRES_USERNAME=datatrans
POSTGRES_PASSWORD=somepassword
POSTGRES_IP=
The user can define the CONNEXION_SERVER
to be blank, gevent, or tornado. In our case we’ll use gevent.
The user can also deploy using SSL certificates, as would likely be found in a prodution deployment.
Example (exposures-api.env):
CONNEXION_SERVER=gevent
CONNEXION_DEBUG=False
API_SERVER_HOST=my.api-server.org
API_SERVER_PORT=443
API_SERVER_KEYFILE=/certs/server.key
API_SERVER_CERTFILE=/certs/server.crt
POSTGRES_HOST=backend.postgres.server
POSTGRES_PORT=5432
POSTGRES_DATABASE=bdtgreen
POSTGRES_USERNAME=datatrans
POSTGRES_PASSWORD=somepassword
POSTGRES_IP=
$ docker run -d \
--name api-server \
--env-file=exposures-api.env \
-v /home/docker/certs:/certs \
-p 443:443 \
mjstealey/exposures_api
Once completed the exposures_api would be running at https://my.api-server.org/v1/ui/#/