Access Control, Interoperability and Cultural Heritage

The challenges facing IIIF

The International Image Interoperability Framework (IIIF) is a set of open standards that enable the Library of Congress, the Smithsonian, the British Library, the Bibliothèque nationale de France and hundreds of other universities, museums and cultural heritage institutions worldwide to provide digitised books, manuscripts, artworks, audio, video and other objects of cultural heritage as part of a shared digital space. If browser-based viewers, annotation tools, research platforms and other applications work with IIIF, then they work with billions of objects of cultural heritage from around the world.

Happily, the vast majority of this content is open to all with no access restrictions. But sometimes, a publisher — a library or museum — needs to protect access. They might need to know who you are — that your account has access to particular licensed images. Sometimes there is a commercial reason for this. Or they might need to know that a site visitor has at least acknowledged an advisory message, or accepted terms and conditions concerning the use of culturally sensitive content.

Different publishers have different ways of authenticating users and identifying subsequent requests from a valid user after they have authenticated (or an anonymous user after they have agreed to terms). Most of these methods involve cookies. The publisher sets a cookie to establish a session, so that the user can browse around freely once they have passed an access control barrier. This is simple functionality that has been around for more than 20 years. The user interaction that leads to the cookie being set varies from publisher to publisher: social sign-in, a custom scheme, a campus-wide University single-sign-on, OAuth2, and others. But once past the access control step the web requests for images (or audio, or video) look the same — simple requests, accompanied by one or more cookies.

So far, so good. Cookies will be part of the web for a long time to come. If you want to control access to some of the content on, then you build a login page, or integrate with a social sign-in protocol. The end result is that’s servers set a cookie for the user, so that requests for the protected images can be authorised and fulfilled seamlessly: can see the cookie that it previously set for that user, coming in on all their subsequent requests from the same browser.

Most IIIF content, whether access-controlled or not, is consumed like this — the user visits and on that site interacts with a viewer, slideshow or even a plain HTML image tag.

A very simple HTML page on, loading an image also on
The image displays just fine

The site only serves protected.jpg if it sees a valid cookie accompanying the request. The web site at knows whether the user has this cookie, and wouldn’t even render a page with an image tag on it like this if it knew that the user did not have this cookie. It might prompt the user to log in, instead.

Third party contexts

IIIF is all about interoperability. If you are comparing two images from different institutions, or using your own preferred viewer instead of the publisher’s website, or loading images into a web-based annotation tool, then the URL in your browser’s address bar is going to have a different host name from one or more of the images it is loading. This is called a third-party context. This makes no difference for the majority of content, which is open. It has two implications for access controlled content.

The first is : how does the application at know whether you are logged in or not, over at If it simply attempts to load the image and hope for the best, it will result in a bad user experience:

A very simple HTML page on, loading an image from a different site:

The viewer application running on, which is likely to be a client-side JavaScript application, needs a way of finding out whether you have access to the resource protected.jpg, before it tries to show it. For security reasons, it can’t see any cookie you might have for And even if it could, it wouldn’t understand it: each publisher will have their own requirements and implementations of cookies for access control. And if the viewer application does find out that you can’t see it, it needs to know where it can send you to do something about it — usually to the login page over at It has no way of knowing what you might be doing while over at, logging in — again for security reasons any interaction you have with other sites, or cookies you might acquire from them, is invisible to the client at It can only wait for you to come back once you have completed the login process over at

The IIIF Authentication Specification defines a set of interaction patterns — a client-side flow — that allows the application at to find out what it needs to know from, without needing to reveal anything about how it implements access control, or the contents of any cookies it might use. These interaction patterns are implemented by compatible viewers and tools.

Web standards and browser implementations of them currently provide cross-domain communication mechanisms sufficient to enable this sharing of knowledge about the user’s ability to see content on a different domain, without leaking any details of how access control is being enforced. The mechanism is complicated, but it works.

The third party cookie problem

The second implication of third party contexts is that we need those cookies for to be sent by your browser, even when the address bar shows If it doesn’t, you’ll get a broken image on the page at, even if you are currently logged in over at

A very simple HTML page on, loading an image from a different site:

Until fairly recently, browsers would send cookies for with any requests, regardless of the address in the browser address bar. If the requesting page is also at, it’s a first-party context; if it isn’t, it’s a third-party context. Cookies for were sent in both cases. the site at can see that you are making the request from; it can see the referrer of the request.

The problem is that this is how trackers work, to target advertising at you. Your browser will send your cookie back to, if they have paid to include an inconspicuous (or invisible) image hosted by on a page. This lets know it’s you browsing, and also you a bit later when you go to This allows advertisers (and other actors) to follow you around, and build up a picture of your browsing behaviour. The actual techniques used grow increasingly more complicated, in an arms race between browsers and those wishing to track us.

It’s also how Cross-Site Request Forgery (CSRF) works. Often with a little added social engineering, a malicious site can generate requests to sites that you might be currently logged in to elsewhere, for example an API request to transfer a bank balance (on a very poorly engineered banking site).

To counter this, browsers introduced support for the cookie SameSite property. A publisher has to state that a cookie it sets may be sent by the browser in a third party context. Your bank would never set its cookies this way, but publishers of images for IIIF use would. This helps with the CSRF problem, but not with the privacy issue — trackers can just set this property for their cookies, too.

Changing browser defaults

Safari now disables third-party cookies by default, no matter what their SameSite settings. And Chrome intends to phase out third party cookies altogether. This is good from a privacy point of view — but it also disables the same interoperability mechanism that IIIF relies on to ensure that protected images from show up on pages from, once the user has logged in at

The approach taken by the Chromium team is The Privacy Sandbox:

After initial dialogue with the web community, we are confident that with continued iteration and feedback, privacy-preserving and open-standard mechanisms like the Privacy Sandbox can sustain a healthy, ad-supported web in a way that will render third-party cookies obsolete. Once these approaches have addressed the needs of users, publishers, and advertisers, and we have developed the tools to mitigate workarounds, we plan to phase out support for third-party cookies in Chrome.

They are explicit about the motivating use cases for retiring third party cookies:

We believe that part of the magic of the web is that content creators can publish without any gatekeepers and that the web’s users can access that information freely because the content creators can fund themselves through online advertising.

IIIF Authentication depends on third party cookies, but not for tracking, advertising or commercial purposes. Publishers of cultural heritage content have much simpler needs, and (usually) very different motivations for publishing content, and protecting access to it. These use cases need cookies to work in a third party context, so that content that depends on the presence of those cookies will continue to display in a third party context.

Ad-tracking is a surreptitious, to-the-side activity that makes used of third party content as a means to a different end. In our use cases, the third-party content is the primary content. The publisher has already implemented a first-party context access control mechanism. They are already using those cookies on IIIF interoperability for access controlled content assumes the image requests work the same way when the address in the browser address bar is something other than, and that the publisher doesn’t have to implement a new custom protocol for IIIF. We also assume that the publisher does not know who the clients are — that there is no need for a pre-established trust relationship, or an approved list of client origin hostnames.

This no longer works cross-domain in Safari, and eventually will not work in Chrome.

What do we do?

We’re not sure at this stage. The principles behind the Privacy Sandbox are laid out in A Potential Privacy Model for the Web, by Google’s Michael Kleber.

This has more promise for IIIF in that it acknowledges:

It may be OK for a site to learn the fact that a user has earned trust on another site

A formal, opt-in way of learning about third party trust relationships would simplify the IIIF Authentication pattern quite dramatically. But we are stuck until we know the implications of the privacy sandbox approach for reuse of access controlled material in a third party context. Knowing whether or not the user has a cookie for the content doesn’t help the interoperability scenario if they can never see the image loaded by a third party client. If there was a transparent, browser supported opt-in process for a user to allow the browser to send cookies to, it would benefit users and simplify the implementation. But if the privacy sandbox does away with third party cookies, publishers can’t benefit from any existing first party, cookie-based access control they likely already have.

If not cookies, then what?

The privacy model describes the concept of a global identity for a user: constraints on this global identity (or at least, shared identity) are the privacy concerns the W3C Web Privacy Group needs to address. Tracking for advertising purposes is about knowing who you are in two places at once.

But in our use case, we don’t need to establish any shared identity at all. We are, essentially, just referencing an image on a different site, and relying on the browser to send any cookie(s) the user might have when asking for that image. In the viewer, or annotation tool, we aren’t interested in who you are at the point of interoperability — we may be interested in who you are for other reasons (e.g., to save the annotations you are making on those images), but they are different use cases. Those other use cases are about the application knowing who you are. In most cases, has no identity management or persistence at all. And anyone can make a IIIF viewer, and anyone can host a IIIF viewer, without any central registry of implementations or approved client origins. The interaction patterns in the IIIF Authentication Specification allow viewers and tools to work with whatever first-party access control the publisher is using.

The IIIF client does not need to establish your identity. In the discussion of first party and third party contexts, neither side needs to be aware of the other’s shared identity, only the sharing of information about the user’s status with respect to the content.

We hope we can engage with the W3C Privacy Group to raise these valid use cases.

Technology Director, Digirati