Preservation Of Knowledge

[Alex Libman]

This forum is underground.  Too far underground!  No one can see it!  The billions and billions of people all over the world cannot learn the wisdom that's been posted here over the years.  All this because:  It's not on Google!  Or any other search engine for that matter.  The robots.txt file on this site [RTFM] explicitly prevents all formal search engines them from spidering / crawling / indexing this site.  Even archive.org is prevented from preserving these writings for posterity!

I of course can archive it independently (as I've done in the past), but that is not a very good solution - and perhaps would do more harm than good by discouraging a more reliable and objective archiving solution.  As a tax resister living in a tent in the woods on near-zero income, I should not be committing myself to any perpetual Web-hosting expenses.  More importantly, I am not an objective impartial third party witness: any alleged snapshots / mirrors / archives of this forum coming from me (as from any other individual) should be suspect.

There's a great free service at archive.IS that (ever since the FTL BBS libricide) I use for third party snapshots of all my public online works, and it doesn't obey the robots.txt restriction, but this Web-site deliberately blocks it as well!

So the question I've been asking for the past few years is WHY?!

This is NOT a privacy / security issue, because anyone can access this site if they have a direct link.  This will not significantly increase traffic to this Web-site.  I can provide technical assistance if needed.

The Early Christians worked hard and risked much to preserve and duplicate their sacred scrolls, and still much has been lost.  This forum, on the other hand, creates artificial barriers to our knowledge being found, accessed, duplicated, and preserved...

This forum holds many pieces of the jigsaw puzzle of libertarian history, which I am struggling to preserve for posterity (when I hope our efforts would be far more appreciated than they are today).  People who wish to live their life on the basis of evidence and reason should want the intellectual history of their movement preserved.  Without the ability to archive it, the forum is just a honeypot for hijacking knowledge into a place where it can be destroyed on a whim!

To me, this (along with other instances of disrespect for knowledge utility in the broader FSP community) has been a HUGE disappointment, and the #1 reason why I never made the move, instead choosing to do my activism as my NJ hometown's sole libertarian...

[Tom Sawyer]

Yeah I've thought the same thing... posterity shouldn't be denied my pearls of wisdom.

[Jim Johnson]

Yes, well.... I'm sure that even the people here don't know what's going on here.

[Pat K]

Hey! What's going on here !

[Alex Libman]

Yes, well.... I'm sure that even the people here don't know what's going on here.

Excellent point!  Even for people who are regulars on this forum, finding a specific thread / post can be difficult.

Using Google can often be better than the built-in forum search, but there are even better search services that specialize in message forums.  One of them is BoardReader.com.  It spiders a few forums, like BitcoinTalk.org for example, providing many useful tools.  It's still in the very initial stages, so it doesn't seem to cover smaller forums (like anything FSP-related) yet, but it would never crawl this site because this site explicitly forbids it...

Being able to search this site via third party Web-sites would mean LESS strain on the nhunderground.com server.

[Free libertarian]

  As the dark ages loom in front of us, we must hide the secret scrolls !  Does anybody remember where we put them?

[WithoutAPaddle]

Fortunately, I memorized the 9th verse of the 23rd scroll.

[Russell Kanning]

good point .... so how do I change it?

[Alex Libman]

good point .... so how do I change it?

Option 1:

• Step 1 - Find a nerd that you trust.

• Step 2 - Remember your Web Hosting service info.

• Step 3 - Give said nerd your Web Host login & password.   

• Step 4 - Done.

Option 2:

        (To fix the restriction blocking out ALL robots, like Google, Bing, DuckDuckGo, archive.ORG, etc, etc, etc.)

• Step 1 - Remember your Web Hosting service info.

• Step 2 - There's probably a browser-based admin panel that lets you manage files - find it.  Or use FTP, etc.

• Step 3 - Find nhunderground.com's root folder and delete the robots.txt file (or replace contents with just "Crawl-delay: 5").

        (To fix the explicit forum ban of my beloved archive.IS, which robots.txt doesn't affect.)

• Step 4 - Read these SMF docs for managing bans (LINK) to learn how to browse and remove bans.

• Step 5 - My best guess is that archive.is is banned via IP or hostname wildcard.  If you have lots of things banned, it can be hard to figure out which one.  You can look at the history of its snapshot attempts of your site (LINK) (ignoring my archive-of-archives work-around research) and match it against the forum IP bans log based on time (account for time zone difference).  At the moment it seems to be using the IP address: "46.166.139.120".  Also look for bans of IP wildcards (ex. "46.166.*") or its cloud host-name ("zayo.com").  Unban it.  If you have a forum plug-in that automatically adds the bans of known cloud service addresses, you'd need to white-list it with this plug-in too...

[Alex Libman]

The overly restrictive /robots.txt has been deleted, yaay!     

(archive.is is still banned, but that's a lesser problem.)

[Russell Kanning]

yah

[Mark D. Jacobsen]

  As the dark ages loom in front of us, we must hide the secret scrolls !  Does anybody remember where we put them?

Yes. The scrolls are at Nag Hamadi.