How to set the Expires header correctly in Varnish

Finding a way to set the Expires header properly on HTTP responses that have been cached by Varnish has been in my to-do list for a long time. Normally, the expiration date of each response to requests from the web should be set to the access time plus the amount of seconds indicated by the max-age attribute of the Cache-Control header of the HTTP response from the backend. The problem is that, since responses from the backend are cached by Varnish, the Expires header is also cached, which results in sending the same expiration date to all HTTP clients, regardless of the time of their HTTP request.

The solution to this problem is to dynamically set the expiration date of the response on every request. Typically, such dynamic header manipulation should take place in the vcl_deliver() subroutine. However, after taking a look at the VCL reference of Varnish 4, I realized that the problem needed a more complicated approach, which is outlined below:

The current time is stored in the now variable, which is accessible from all built-in subroutines. So, all we need in order to correctly set the date in the Expires header is the initial TTL as determined by Varnish at the time the response has been fetched from the backend. This happens in the vcl_backend_response() subroutine. The final expiration date will be set to current_time + initial_ttl.
So, in vcl_backend_response() we add a temporary HTTP header, which contains the backend response’s TTL as determined by Varnish, to all cacheable responses from the backend.
Later on, in vcl_deliver() we are going to use this value in order to calculate the expiration date on a per request basis and finally remove the temporary header we used to store the TTL.
Moreover, for the calculation of the final expiration date we are going to need a function that can convert the TTL, which is stored as a string in the temporary header, to a duration object. This can be done by the std.duration() function.

The following code is the required VCL you should add in order to dynamically set the value of the Expires HTTP header. Note, that we do not touch the Expires header of backend responses that are not meant to be cached by Varnish.


# Required VMOD imports
import std;

sub vcl_backend_response {

    #
    # Store the TTL of the backend response object in the
    # temporary 'x-obj-ttl' header on all cachable responses.
    # Notice that we quantify the value with s (seconds).
    #
    if (beresp.ttl > 0s) {
        set beresp.http.x-obj-ttl = beresp.ttl + "s";
    }

}

sub vcl_deliver {

    #
    # Dynamically set the Expires header on every request from the web.
    #
    if (resp.http.x-obj-ttl) {
        # 1. Calculate and reset the Expires header.
        # (3600s is just a fallback value)
        set resp.http.Expires = "" + (now + std.duration(resp.http.x-obj-ttl, 3600s));
        # 2. Delete the temporary header from the response.
        unset resp.http.x-obj-ttl;
    }

}

The above VCL code has been tested with Varnish version 4. My guess is that it should also work fine with Varnish version 3.

Note that the modification of the Expires header takes place only for cached objects to which the temporary HTTP header x-obj-ttl has been added. So, my suggestion is to flush the whole cache after you have added the above VCL code to your configuration.

How to set the Expires header correctly in Varnish by George Notaras is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Copyright © 2016 - Some Rights Reserved

Related Articles