Please Note:
This project does not involve any of the native FileMaker technologies like FileMaker Custom Web Publishing (CWP), FileMaker Instant Web Publishing (IWP) or FileMaker WebDirect. This is about a FileMaker solution designed to prime/load the cache of a dynamic website in order to improve performance.
For most of the past twenty years I used static websites. Originally the pages were built manually but for most of the past decade the pages were created automatically by a CMS (Movable Type). This involved a "rebuild" step where the CMS would create all of the static pages, but the benefit was performance – the sites were very, very fast.
Switching to a dynamic website (built with Concrete5) has felt like a good move. There is no more rebuilding, and the ability to include dynamic elements like related content blocks is fantastic. The performance is very good but not quite as fast as the static sites generated by Movable Type.
Concrete5 (and, I believe, other CMS systems) attempt to improve performance with a caching option. When a page is visited for the first time the CMS has to generate the html (time consuming), but the cache option allows the html to be saved for subsequent visits. Subsequent visitors only need to download the html, they don't need to wait for the server to generate it.
In theory that would be all a busy site would need. All pages would be loaded and cached and the vast majority of users would see the faster cached version of each page.
For a less busy site, especially one that is constantly being worked on, added to, etc. it isn't that simple. Caches are emptied each time the server or Apache is restarted. And they are often purged deliberately when doing work on the site. This means that more people will be the first visitor to more pages, and won't receive the benefit of caching.
One solution is to preemptively cache all of the pages – sometimes called "priming" or "pre-loading" the cache. There are different ways of doing this, one of the most popular being the wget function. If wget is enabled on your server you could use the following command to prime/pre-load your cache:
Parameters explained:
Additionally wget can be easily scheduled and automated using a cron job. Another option is curl/curl-loader, which can also be used to prime a cache and scheduled with a cron.
But wget and curl are not natively available on all servers. I also wasn't sure how they might affect server performance while underway (some of the sites have thousands of pages), skew traffic reporting etc.
It seemed like a FileMaker solution could provide more granular control and allow for some additional features that are not part of wget or curl-loader.
The desired features include...
Having all websites in a single FileMaker file allows a single machine running that file to prime/pre-load all of the pages. Even an older, underpowered machine could be priming the caches on dozens of large sites on different servers. It would be a great use for an older box otherwise gathering dust.
The fast cache feature allows someone working on a website, or updating content, to quickly reload the cache with the updated html. Meanwhile the drip feed would keep all of the caches primed without spiking server traffic, affecting performance or skewing traffic figures. This could also turn into a performance monitoring/charting feature, keeping track of the performance of each page over time.
The cache primer/loader FileMaker project is hosted on GitHub
