The challenges facing IIIF
The International Image Interoperability Framework (IIIF) is a set of open standards that enable the Library of Congress, the Smithsonian, the British Library, the Bibliothèque nationale de France and hundreds of other universities, museums and cultural heritage institutions worldwide to provide digitised books, manuscripts, artworks, audio, video and other objects of cultural heritage as part of a shared digital space. If browser-based viewers, annotation tools, research platforms and other applications work with IIIF, then they work with billions of objects of cultural heritage from around the world.
Happily, the vast majority of this content is open to all with no access restrictions. But sometimes, a publisher — a library or museum — needs to protect access. They might need to know who you are — that your account has access to particular licensed images. Sometimes there is a commercial reason for this. Or they might need to know that a site visitor has at least acknowledged an advisory message, or accepted terms and conditions concerning the use of culturally sensitive content.
Different publishers have different ways of authenticating users and identifying subsequent requests from a valid user after they have authenticated (or an anonymous user after they have agreed to terms). Most of these methods involve cookies. The publisher sets a cookie to establish a session, so that the user can browse around freely once they have passed an access control barrier. This is simple functionality that has been around for more than 20 years. The user interaction that leads to the cookie being set varies from publisher to publisher: social sign-in, a custom scheme, a campus-wide University single-sign-on, OAuth2, and others. But once past the access control step the web requests for images (or audio, or video) look the same — simple requests, accompanied by one or more cookies.
So far, so good. Cookies will be part of the web for a long time to come. If you want to control access to some of the content on publisher.org, then you build a login page, or integrate with a social sign-in protocol. The end result is that publisher.org’s servers set a cookie for the user, so that requests for the protected images can be authorised and fulfilled seamlessly: publisher.org can see the cookie that it previously set for that user, coming in on all their subsequent requests from the same browser.
Most IIIF content, whether access-controlled or not, is consumed like this — the user visits publisher.org and on that site interacts with a viewer, slideshow or even a plain HTML image tag.
The site only serves protected.jpg if it sees a valid cookie accompanying the request. The web site at publisher.org knows whether the user has this cookie, and wouldn’t even render a page with an image tag on it like this if it knew that the user did not have this cookie. It might prompt the user to log in, instead.
Third party contexts
IIIF is all about interoperability. If you are comparing two images from different institutions, or using your own preferred viewer instead of the publisher’s website, or loading images into a web-based annotation tool, then the URL in your browser’s address bar is going to have a different host name from one or more of the images it is loading. This is called a third-party context. This makes no difference for the majority of content, which is open. It has two implications for access controlled content.
The first is : how does the application at viewer.org know whether you are logged in or not, over at publisher.org? If it simply attempts to load the image and hope for the best, it will result in a bad user experience:
The viewer application running on viewer.org, which is likely to be a client-side JavaScript application, needs a way of finding out whether you have access to the resource protected.jpg, before it tries to show it. For security reasons, it can’t see any cookie you might have for publisher.org. And even if it could, it wouldn’t understand it: each publisher will have their own requirements and implementations of cookies for access control. And if the viewer application does find out that you can’t see it, it needs to know where it can send you to do something about it — usually to the login page over at publisher.org. It has no way of knowing what you might be doing while over at publisher.org, logging in — again for security reasons any interaction you have with other sites, or cookies you might acquire from them, is invisible to the client at viewer.org. It can only wait for you to come back once you have completed the login process over at publisher.org.
The IIIF Authentication Specification defines a set of interaction patterns — a client-side flow — that allows the application at viewer.org to find out what it needs to know from publisher.org, without publisher.org needing to reveal anything about how it implements access control, or the contents of any cookies it might use. These interaction patterns are implemented by compatible viewers and tools.
Web standards and browser implementations of them currently provide cross-domain communication mechanisms sufficient to enable this sharing of knowledge about the user’s ability to see content on a different domain, without leaking any details of how access control is being enforced. The mechanism is complicated, but it works.
The third party cookie problem
The second implication of third party contexts is that we need those cookies for publisher.org to be sent by your browser, even when the address bar shows viewer.org. If it doesn’t, you’ll get a broken image on the page at viewer.org, even if you are currently logged in over at publisher.org.
Until fairly recently, browsers would send cookies for publisher.org with any requests, regardless of the address in the browser address bar. If the requesting page is also at publisher.org, it’s a first-party context; if it isn’t, it’s a third-party context. Cookies for publisher.org were sent in both cases. the site at publisher.org can see that you are making the request from viewer.org; it can see the referrer of the request.
The problem is that this is how trackers work, to target advertising at you. Your browser will send your tracker.com cookie back to tracker.com, if they have paid newspaper.com to include an inconspicuous (or invisible) image hosted by tracker.com on a newspaper.com page. This lets tracker.com know it’s you browsing newspaper.com, and also you a bit later when you go to magazine.com. This allows advertisers (and other actors) to follow you around, and build up a picture of your browsing behaviour. The actual techniques used grow increasingly more complicated, in an arms race between browsers and those wishing to track us.
It’s also how Cross-Site Request Forgery (CSRF) works. Often with a little added social engineering, a malicious site can generate requests to sites that you might be currently logged in to elsewhere, for example an API request to transfer a bank balance (on a very poorly engineered banking site).
To counter this, browsers introduced support for the cookie SameSite property. A publisher has to state that a cookie it sets may be sent by the browser in a third party context. Your bank would never set its cookies this way, but publishers of images for IIIF use would. This helps with the CSRF problem, but not with the privacy issue — trackers can just set this property for their cookies, too.
Changing browser defaults
Safari now disables third-party cookies by default, no matter what their SameSite settings. And Chrome intends to phase out third party cookies altogether. This is good from a privacy point of view — but it also disables the same interoperability mechanism that IIIF relies on to ensure that protected images from publisher.org show up on pages from viewer.org, once the user has logged in at publisher.org.
The approach taken by the Chromium team is The Privacy Sandbox:
After initial dialogue with the web community, we are confident that with continued iteration and feedback, privacy-preserving and open-standard mechanisms like the Privacy Sandbox can sustain a healthy, ad-supported web in a way that will render third-party cookies obsolete. Once these approaches have addressed the needs of users, publishers, and advertisers, and we have developed the tools to mitigate workarounds, we plan to phase out support for third-party cookies in Chrome.
They are explicit about the motivating use cases for retiring third party cookies:
We believe that part of the magic of the web is that content creators can publish without any gatekeepers and that the web’s users can access that information freely because the content creators can fund themselves through online advertising.
IIIF Authentication depends on third party cookies, but not for tracking, advertising or commercial purposes. Publishers of cultural heritage content have much simpler needs, and (usually) very different motivations for publishing content, and protecting access to it. These use cases need cookies to work in a third party context, so that content that depends on the presence of those cookies will continue to display in a third party context.
Ad-tracking is a surreptitious, to-the-side activity that makes used of third party content as a means to a different end. In our use cases, the third-party content is the primary content. The publisher has already implemented a first-party context access control mechanism. They are already using those cookies on publisher.org. IIIF interoperability for access controlled content assumes the image requests work the same way when the address in the browser address bar is something other than publisher.org, and that the publisher doesn’t have to implement a new custom protocol for IIIF. We also assume that the publisher does not know who the clients are — that there is no need for a pre-established trust relationship, or an approved list of client origin hostnames.
This no longer works cross-domain in Safari, and eventually will not work in Chrome.
What do we do?
We’re not sure at this stage. The principles behind the Privacy Sandbox are laid out in A Potential Privacy Model for the Web, by Google’s Michael Kleber.
This has more promise for IIIF in that it acknowledges:
It may be OK for a site to learn the fact that a user has earned trust on another site
A formal, opt-in way of learning about third party trust relationships would simplify the IIIF Authentication pattern quite dramatically. But we are stuck until we know the implications of the privacy sandbox approach for reuse of access controlled material in a third party context. Knowing whether or not the user has a cookie for the content doesn’t help the interoperability scenario if they can never see the image loaded by a third party client. If there was a transparent, browser supported opt-in process for a user to allow the browser to send cookies to publisher.org, it would benefit users and simplify the implementation. But if the privacy sandbox does away with third party cookies, publishers can’t benefit from any existing first party, cookie-based access control they likely already have.
If not cookies, then what?
The privacy model describes the concept of a global identity for a user: constraints on this global identity (or at least, shared identity) are the privacy concerns the W3C Web Privacy Group needs to address. Tracking for advertising purposes is about knowing who you are in two places at once.
But in our use case, we don’t need to establish any shared identity at all. We are, essentially, just referencing an image on a different site, and relying on the browser to send any cookie(s) the user might have when asking for that image. In the viewer, or annotation tool, we aren’t interested in who you are at the point of interoperability — we may be interested in who you are for other reasons (e.g., to save the annotations you are making on those images), but they are different use cases. Those other use cases are about the viewer.org application knowing who you are. In most cases, viewer.org has no identity management or persistence at all. And anyone can make a IIIF viewer, and anyone can host a IIIF viewer, without any central registry of implementations or approved client origins. The interaction patterns in the IIIF Authentication Specification allow viewers and tools to work with whatever first-party access control the publisher is using.
The IIIF client does not need to establish your identity. In the discussion of first party and third party contexts, neither side needs to be aware of the other’s shared identity, only the sharing of information about the user’s status with respect to the content.
We hope we can engage with the W3C Privacy Group to raise these valid use cases.
