Re: [squid-users] open webpage category database proposal

From: Bgs himself <bgs@dont-contact.us>
Date: Tue, 29 Jul 2003 15:16:21 +0200 (MEST)

 Hi !

> Due to the sheer size of a decent ratings databases you will need to
> implement your own DNS master servers however, using some form of
> replicated backend database with a DNS interface frontend. The
> standard DNS servers is not suitable for these volumes of master
> information or rates of updates required.

Agreed. A backend database creates the DNS db that can work at the highest
speed possible. My first choice would be tinydns. It has a very simple and
easily generatable db and lack a lot of (here) totally useless code.

> Exacly. And this is THE major technical problem to solve for this to
> become a reality. How to get someone to rate the pages with a
> reasonably good accuracy. With web pages coming and disappearing at a
> rate of god know how many hundreds of thousands per month in is a
> quite big problem to solve.

Yes this is the major problem. This is where the 'open' part comes into
play. Managing such a db on an admin or even on a company level is a huge
work. But if there are let's say a dozen 'center' around the world with
admins and enough user contribution, this whole thing could work.

As you pointed out, a good classification system is vital. We should find
a system that allows minimal subjective influence. Let the end user decide
on the final classification (good-bad).

> How to get the ratings distributed is minor and not really a problem.
> There is many ways to do this, where DNS is one possible framework
> which fits very well given a correct application of DNS properties to
> the distribution problem.

Yep. AXFR, direct db exchange, etc.

> Then you ask the system to have them reclassified as pornography, and
> if the system is good in design it allows for both classifications if
> the content indeed fits both classifications.

Right :)

Bye
Bgs
Received on Tue Jul 29 2003 - 08:11:43 MDT

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 17:18:20 MST