8

Background

I recently came across an answer where an archived link (the one that point to the web archive) was used instead of the direct link.

When I asked the user for the intention I got a reply from another stating

because it might not keep working in the future.

Issue

The problem with this approach is that when the original link stops working we can find an archive link using the original and replace it. But if the archive link stops working we will not be able to trace back the original link from it (https://archive.fo/QGtny) and the reference will be gone forever.

Questions

  1. Should we prefer original links to archives? (And create a snapshot in archive maybe).

  2. If the answer to the first question is Yes, then should we replace the archive links with original one when we come across it? (And point the author to this meta post maybe).

2
  • 2
    I've thought about this before. When you refer directly to an external image file, it doesn't end up just pointing to that original location. A copy is made and hosted by imgur.com. It would be nice if Stack Exchange implemented a similar system for archiving URLs. Also, a nice tool for citing references and attributing sources wouldn't go astray.
    – voices
    Nov 17 '18 at 14:18
  • 1
    we will not be able to trace back the original link from it -- for this reason, whenever I'm referring to the oldest capture, I'll format it as such: https://archive.is/o/www.dailydot.com/debug/elon-musk-bitcoin-scam-verified-twitter-account/ (I mostly use this on my own blog, and am not actually advocating for a blanket policy of archived links being used at-first here; I just wanted to share a way that it could trivially be done more documentedly if needed.)
    – JamesTheAwesomeDude
    May 19 at 21:46
16

I was the author of the answer with the archived link. In general I think that no, we should not provide archived links. I only used the archive for this specific link because, in my experience, The Daily Dot often deletes or modifies articles, making them invalid for future reference. Using an archive ensures that people are able to see what was present when the article was first linked, not anything else.

The problem with this approach is that when the original link stops working we can find an archive link using the original and replace it. But if the archive link stops working we will not be able to trace back the original link from it (https://archive.fo/QGtny) and the reference will be gone forever.

That is a good point. I had actually intended to create an HTML comment with the original URL just in case, but completely forgot about it when I got sucked into something completely unrelated.

Should we prefer original links to archives? (And create a snapshot in archive maybe).

Generally yes. As for creating a snapshot in the archive, I actually do that for almost all the links I post unless they are reproductions of redundant data like man pages, source code, or are on a site that is regularly archived already, like Wikipedia or Stack Exchange.

If the answer to the first question is Yes, then should we replace the archive links with original one when we come across it? (And point the author to this meta post maybe).

I don't think that's necessary, but I'm obviously biased.

Anyway, I've went ahead and found a more reliable source and linked to it instead.

Note: The reason I don't use the Internet Archive is because they retroactively exclude sites from the public if a robots.txt policy is in place, which is often the case when a website is turned into a landing page. Proof: https://web.archive.org/web/*/www.truecrypt.org and http://www.truecrypt.org/robots.txt

14
  • I didn't know that the daily dot does that. That is good to know.
    – Kolappan N
    Nov 15 '18 at 2:29
  • @KolappanNathan It's just in my personal experience. I could be misremembering.
    – forest
    Nov 15 '18 at 2:36
  • 1
    Oddly enough, archive.fo can't establish a TLS connection with my browser (Firefox 64.0b9).
    – Nonny Moose
    Nov 17 '18 at 0:58
  • @NonnyMoose That happens to me occasionally (on Tor Browser, which is Firefox ESR). Oddly enough, it seems to depend on factors like which version of the archive you visited first, and sometimes it just spontaneously starts working again. I haven't had an opportunity to actually test it, but it is strange.
    – forest
    Nov 17 '18 at 1:00
  • That is interesting! I guess I'll try it again later. Also, the plain http site is currently down. (strange) Edit: Apparently it's just Cloudflare messing with me; it works on my phone?
    – Nonny Moose
    Nov 17 '18 at 1:01
  • @NonnyMoose For me, both http and https work (http redirects to https). While archive.is fails for https but works for http (redirects to archive.fo), and archive.today works for both http and https (both redirect to archive.fo). It's all very strange. Also... SSLTest refuses to connect to it.
    – forest
    Nov 17 '18 at 1:05
  • Weird! It might even be because on my computer I'm using Cloudflare's DNS.
    – Nonny Moose
    Nov 17 '18 at 1:07
  • @NonnyMoose I doubt it. DNS would affect the domain itself, not only the https version or only the http version of the site.
    – forest
    Nov 17 '18 at 1:07
  • 1
    I believe you're using wrong archive service. Way Back Machine preserves original URLs in archived links, as opposed to shortening of the service you use. In this question this is the only important thing. By the way, this answer is vague, since there must be a policy explaining why it is important that all (or some) links work in future, then owner of website kills them.
    – Croll
    Nov 17 '18 at 19:40
  • 5
    @Croll Unfortunately, the Way Back Machine has a fatal problem. If the domain is ever re-registered and blocks the archiver in robots.txt, the site will be retroactively removed. I've tried it myself, registering a domain from an old archived site and setting robots.txt to disallow all, and the site (from 2007) suddenly stopped showing. This is a huge problem because domain squatters will buy a domain and will, by default, add an exclusionary robots.txt to it... So no, I will never use the Way Back Machine.
    – forest
    Nov 18 '18 at 1:38
  • @Croll As for preserving the URL itself, you could easily append the original URL in the form of a fragment.
    – forest
    Nov 18 '18 at 1:40
  • 1
    If you're using an archive, use web.archive.org (just put it before the actual url). This archive is much more likely to keep available than smaller sites like archive.fo and your link will contain the original URL, which can be used to look for other archives (or see the page with more recent updates). @forest: They stopped doing this some time ago. There even was some outrage when media wrote "archive.org stops respecting robots.txt".
    – allo
    Nov 21 '18 at 15:20
  • What I do is try to find the original article (often there's a 'source' URL at the bottom of the article) or the author's home page (if it's a large company/university) and then quote the title and hit the 'Link Button' (🔗) -- then if I feel that I must do more I'll type markup so it looks like: ... For more info see: "How to do it" (Alt. Links: #2, #3). --- As for putting in an Archive Org URL into the question/answer that's something that the Server at SE could do (Community User) if we want or if SE finds it 404 several months down the road. Authoring archive links is last choice.
    – Rob
    Nov 22 '18 at 22:12
  • @allo It looks like they haven't actually stopped doing it. Just now I found out I was unable to access the old TrueCrypt website as a result of it explicitly blocking IA archiver in robots.txt.
    – forest
    Jan 3 '19 at 4:15
1

The link you describe is a shortened link. Link shorteners are officially frowned upon and would be outright blocked if it were technically feasible to do so.

There is nothing wrong with linking to the Internet Archive, or other archival services, but the full, original URL should either be a component of the address (e.g. https://web.archive.org/web/TIMESTAMP/URL), or a separate link to the original should be provided. This way, if your preferred archival service dies, we still have the original URL, and maybe we can even find the content on a different archival service.

I don't know the folks who run http://archive.fo from Adam. Maybe they're perfectly competent and have long-run continuity of operations plans just like the Internet Archive people do. Maybe they don't. Including the original URL removes the need to trust third parties.

1
  • 1
    It is not a link shortener. The fact that the link it generates is not long is irrelevant. Your point that not containing the original URL in the case that the service dies is valid however.
    – forest
    Nov 30 '18 at 3:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .