Data is presented in different formats. This guide contains information, tips and suggestions on how to use some of the formats presented in the Open Data Portal as well as how to work with some of the application programming interfaces (APIs).
On this page
Caching Data
Caching is a broad topic and can be seen at the memory, storage or communication layers by either side of a client/server. In this context, the server side is the data source and the client side is the developer or user who is trying to download the data.
Data Source Caching
Some data sources may use caching to help deal with scalability. As previously mentioned, this caching may be found at different levels within their service. This is typically transparent to end users but it is important to understand these practices to efficiently and respectfully interact with the data source.
If your requests continuously yield the same response yet you are expecting an updated/newer dataset then consider the following:
- You have verified that there is no caching occurring within your tools or framework.
- There is a reasonable expectation that the data source should return a revised response.
- You have looked at the data source’s documentation for information regarding timestamps or cached response rules.
- You have tried changing your query to see if the response differs.
- Try waiting some time to see if the data gets refreshed at a later time.
In some cases, the data source might be using a Content Delivery Network (CDN) to cache common responses. Look at the HTTP headers to see hints of a CDN or other caching mechanisms being used.
Client (Application) Side Caching
You may find optimization opportunities within your application to cache data in part or in its entirety. Caching can dramatically increase the performance of your application and deliver significant improvements to user experiences. Of course, caching on the client-side requires a balance between performance and accuracy as client-side caches can become outdated.
Error Handling
As a general rule for datasets and APIs, it is important to recognize and respectfully react to HTTP response codes. The following is a summary of some of the common HTTP response codes you may encounter.
Code | Summary | Explanation |
---|---|---|
200 | OK | Successful request |
301 | Moved Permanently | The requested item now resides at the following new location |
304 | Not Modified | The requested item has not been modified since your last request for it. |
404 | Not Found | Nothing was found at the requested URI. |
40x | Client Errors | Various errors related to your request. |
50x | Server Errors | Various server side errors. Best to review your request and try again at a later time. |