In an earlier post, Erik talked about why isn’t there more specs on web APIs. In this blog post, I will add to this discussion, not from the spec creation point of view but from the maintenance point of view.
What types of change to the web service rendered specs out-dated and, more problematically, may prevent client code from working?
Which APIs committed such offenses?
The data: 260 Open API Specifications
As part of our group’s effort in building tools to help developers use web APIs, we have looked at web API specs (Open API specifications) from the APIs Guru project. We have also mined GitHub for client code calling to these APIs to see how these APIs are used.
Erik has given an excellent overview on the request endpoints, query parameters, and payloads extracted from this data. Some well-known APIs in this data include Google APIs, the Twilio API, the Spotify API, the Slack API, and GitHub’s API. Although APIs Guru is an active effort with a policy to update Open API Specifications from their original source daily whenever possible, these specs are still incomplete and erroneous according to the official API documentation. When the spec is inconsistent with the official API documentation, is the spec right or the official documentation right? This demonstrates the challenges in maintaining a spec or the API documentation.
Inconsistencies between the Specs and API Documentation
For each case when the spec was inconsistent with the URL endpoint in a request in code we mined from GitHub, we examined the reason behind the inconsistency with the help of the official API documentation of the API provider. The inconsistency can be a result of two reasons:
- The spec and the API documentation is consistent. We assume the error is in the code usage.
- The code usage is consistent with API documentation, but the spec is inconsistent with the official API documentation.
We are interested in reporting the second case, when the spec and API documentation are inconsistent:
Across all specifications, we found 18 endpoints that are legitimate according to APIs’ online documentation and client code, but were missing from the API specifications.
Slack’s spec from APIs Guru project took the top spot with 7 missing endpoints, while various Google APIs a single missing endpoint. In summary, we found:
|Slack API - 7 endpoints missing:||
|Reactome API - 3 endpoints missing:||
|Trello API - 2 endpoints missing:||
|YouTube API - 2 endpoints missing:||
|Google Drive API - 2 endpoints missing:||
|Google Maps Gelocation API - 1 endpoint missing:||
|Google PageSpeed Insights API - 1 endpoint missing:||
Missing authorization URLs in the spec
We found two specs from APIs Guru project where the spec was missing endpoints for authentication purposes:
Backward incompatible change involving a base URL
API service providers can make backward incompatible changes to the base path. For example, Mandrill added a version number to their base URL, changing it from http://mandrillapp.com/api to http://mandrillapp.com/api/1.0. We ranked this low because we have found more code on GitHub uses the correct base path https://madnrillapp.com/api/1.0 than not.
Challenges in Maintaining the Specs
The main point is, even when the spec has been created, changes to the web service make the spec out of sync. Even with vigilant projects such as APIs Guru project collecting and updating the specs, it is not easy to keep up these changes. Our group is actively working on techniques to help web API providers create and maintain the specs… stay tuned!