How to stop search engines indexing media and assets in Umbraco

Today my colleague asked me if I could help them work out how to stop search engines from indexing media items and static assets. We already had a disallow rule in the robots.txt, but we wanted to be more specific about it with the individual files.

We came up with this code which sets the robots header to be "noindex, nofollow" on anything that starts with /media or /assets.

I thought you might find it useful too. It works in Umbraco 10 and above. You can add it in the Program.cs file or Startup.cs file above app.UseUmbraco()

app.UseWhen(
    context => context.Request.Path.StartsWithSegments("/media") 
    || context.Request.Path.StartsWithSegments("/assets"),
    appBuilder => appBuilder.UseStaticFiles(new StaticFileOptions
    {
        OnPrepareResponse = ctx =>
        {
            ctx.Context.Response.Headers.Append("x-robots-tag", "noindex, nofollow");
        }
    })
);