Members

Post by **Marc van Arc** » 19 Jun 2007, 09:22

The subject title is not very appropriate, but given the fact(?) that spammers don't read other posts it might do.
I noticed a lot of those get-a-life-persons use nonsensical letter combinations.
Is there no way to deny access for such user names?
Or is it just a thing we have to learn to live with?
Sorry to bother you again on this subject. I know you're working very hard to get these %$$@%$$ out. So why not just don't let them in?

MatsP · Post by **MatsP** » 19 Jun 2007, 10:45

The ones I've seen have fairly "sensible" names - not necessarily something that can be related to Catfish or a persons name, but very often they don't have any "obvious sign of being a spammer".

I know it's annoying (I remove 3-4 spams a day, and I'm sure that Silurus and the other Mod's catch a few too).

If it was trivial to spot a spammer, I think the solution is good, but I don't think there's any really good way to tell the difference between a spammer and a non-spammer.

--
Mats

Dave Rinaldo · Post by **Dave Rinaldo** » 19 Jun 2007, 14:14

I recognise the spams by the subject line and, that 90% of the time, they are posted in the Loricariidae forum.

I don't open them.

It's rare that one of the Mods isn't around to delete them in a faily short amount of time.

Post by **Silurus** » 19 Jun 2007, 14:26

A very high percentage of the spam you see are written by programs anyway. I usually try to pre-emptively ban/delete spammer accounts before they post, but I cannot be on the forum page all the time.

One possible solution would be to block email address domains most frequently used by spammers

MatsP · Post by **MatsP** » 19 Jun 2007, 14:38

Silurus: Are you saying that the spam is by "robots"? I think Jools have stated many times that it's not "robots" that post the spam.

Either way, I think there's no easy solution.

Some intelligent scanning of the posts would be beneficial, but I'm sure it's hard to get the balance right between letting spam past and non-spam being stopped.

Since most of the spam-links are "moving targets", it's hard to mechanically detect the difference between a personal web-site with fishes and a spam-link to a porn-site.

Detecting certain words such as viagra would be a good start.

But then again, it's a bit of a mess to do that, since it would require additional patches on top of the existing phpBB, which means that those patches have to be maintained separately from phpBB.

I think version 3.0 of phpBB is in the works (and it has further anti-spam measures), and I think it's not much point in doing changes to the 2.x version that is currently used by PlanetCatfish.

--
Mats

Post by **Silurus** » 19 Jun 2007, 15:21

Most of the spam that shows up are posted by users that do not show up on the active list at the bottom of the page (nor are they hidden users). A human poster would show up on the active list (wether as a hidden user or not). To me, such a phenomenon looks like the work of a program, rather than a human poster.

MatsP · Post by **MatsP** » 19 Jun 2007, 15:30

I don't agree - I've seen several spam users on the "who's logged in".

A program that doesn't appear on the "who's logged in" would also mean that the "post" would have to be inserted directly into the phpBB data-base. If they can do that, then I don't think anything can prevent spammers - but to do that, they'd either need a backdoor into the database, or know the password for the database.

--
Mats

YouTube · Post by **Jools** » 22 Jun 2007, 10:04

Here's some background.

0. I wasn't aware that robots could get past the visual identification code. If there are robots doing this, can someone tell me how?

1. As it only took about an hour of work, I made the location field mandatory at registration time, any robot would have to be modified to ensure it always puts in a location for an automated registration request to work.

2. The response to registration goes to the email address used at the time of registration. If that isn't valid, is full or whatever, then activation of the account cannot happen. Accounts can only be activated by clicking on a link in the activation email.

3. You can't 90%, let alone 100%, spot human spammers. Programmatically, I can't personally see a way to significantly reduce this.

Jools

MatsP · Post by **MatsP** » 22 Jun 2007, 10:58

0: "OCR"?

3. I think we could (possibly) add a filter to look for certain patterns in the post - such as LOTS of links to other websites, names of certain medecines (I don't think it's necessary to give Viagra to fish) or "porn" words.
Weigh in if the poster is a first-time poster (or low-number of posts).

But it's still not going to be 90% sure - it will IMPROVE the detection rate, but not make it fool-proof.

Also, at one point, I did a search for a particular username (it looked "interesting"), and found the same user in 15 or so forums (that I looked at, Google returned LOTS of pages). All registrations where SIMILAR but not the same - which means either a very clever robot or a human is the registrator.

It may well be that the actual POSTS after the name has been registered is done by robots.

--
Mats

YouTube · Post by **Jools** » 22 Jun 2007, 11:50

MatsP wrote:0: "OCR"?

They are meant to be OCR proof, heck I get one email a week (or so) because humans can't read them!

MatsP wrote:3. I think we could (possibly) add a filter to look for certain patterns in the post - such as LOTS of links to other websites, names of certain medicines (I don't think it's necessary to give Viagra to fish) or "porn" words.
Weigh in if the poster is a first-time poster (or low-number of posts).

True, but that's more work (in keeping a word list up-to-date) and what are you then going to do once you ID the post.

There is LOTS of discussion about this over at http://www.phpbb.com but the fact is that the only foolproof way appears to be to switch to manual registration (i.e. admins only can approve registrations).

Jools

MatsP · Post by **MatsP** » 22 Jun 2007, 12:53

Jools wrote:
MatsP wrote:0: "OCR"?

They are meant to be OCR proof, heck I get one email a week (or so) because humans can't read them!

MatsP wrote:3. I think we could (possibly) add a filter to look for certain patterns in the post - such as LOTS of links to other websites, names of certain medicines (I don't think it's necessary to give Viagra to fish) or "porn" words.
Weigh in if the poster is a first-time poster (or low-number of posts).
True, but that's more work (in keeping a word list up-to-date) and what are you then going to do once you ID the post.

Auto-delete [or move to quarantine] posts that are considered spam. Repeated offense: auto-block the account.

If the list of words is in a database (rather than a program form) and there are a combination of "positive" and "negative" words - e.g. mentioning certain words give you a "plus" score, other words a "negative" would improve the chances of detecting the right things. So if someone says "I gave my bristlenoses some <food> and it seems like it was fish-viagra, because they have spawned now", that would still be accepted becasue it contains the positive words "Ancistrus", "fish", "spawned" as well as the negative "viagra" -> score = +2.

But I agree that it's a lot of work - and to make it generic so that it works for all sorts of forums would be even more difficult, as a photo-forum would have different set of words that are "good" than a "fishy" forum, for example.

There is LOTS of discussion about this over at http://www.phpbb.com but the fact is that the only foolproof way appears to be to switch to manual registration (i.e. admins only can approve registrations).

I'm sure that's not 100% sure either - unless you actually know the person behind the e-mail address - and I wouldn't know 10% of the (active) members of this forum by their e-mail address.

Which one of these users would be allowed in, "ugnix" or "argaiv"? Both are (by immediate appearance) "unsensible names" - but the first one would be the one I should let in, but not the second one. [1]

Not to mention users caled "JohnSmith33" or "FishyLover" (these are just names I just made up).

So some users will slip through anyways. Silurus filters out many of the users that register to post spam based on whatever criteria (looking at the "www" address in their profile is a good start - but doesn't work every time).

The only REALLY secure way is to have admin approve the first several posts - perhaps admin approval for all posts until the user has "proven" that they are a valid user [so posting a dozen "nice fish" or similar won't trigger a "you're now free to post"].

On a different forum, we sometimes get spam posts in the form of a comment to an existing thread. I haven't seen that here.

Jools

[1] That's "Xingu" and "Viagra" backwards, a pattern copied from Neil from Pier Aquatics name [anyone know who I talk about - I thought his name was the product of a vegetarian burger factory until I triggered on the "backwards spelling"]... There are so many users that have names that don't appear very sensible.

--
Mats

YouTube · Post by **Jools** » 22 Jun 2007, 14:08

I have to say that I think that's a lot more work that the effort that primarily HH, but everyone generally puts in to keep an eye on things.

It also becomes redundant work when we upgrade the forum.

If you can find a forum mod that can be achieved in under 2-3 hours work, I'll look at it but this kind of integration would take days maybe even weeks to build.

Jools

Post by **Marc van Arc** » 22 Jun 2007, 14:51

MatsP wrote: "ugnix" or "argaiv"? Both are (by immediate appearance) "unsensible names"

I see what you mean, yet I was specifically refering to names such as FGYYGFFGYYGF, DHTDHT and so on. So literally letter combinations that make no sense.
They always come with a website and strange interests.

But it is indeed a hard job to keep them out. Thanks for the efforts you're making to do so.

MatsP · Post by **MatsP** » 22 Jun 2007, 15:32

Jools wrote:I have to say that I think that's a lot more work that the effort that primarily HH, but everyone generally puts in to keep an eye on things.

It also becomes redundant work when we upgrade the forum.

If you can find a forum mod that can be achieved in under 2-3 hours work, I'll look at it but this kind of integration would take days maybe even weeks to build.

Jools

Let me first say that I don't see the current situation as something that really needs something doing to fix it - yes, we do get the odd bit of spam, and we (the moderators) remove it most of the time within minutes or at least a few hours.

And with regards to the effort, I said earlier:

I think version 3.0 of phpBB is in the works (and it has further anti-spam measures), and I think it's not much point in doing changes to the 2.x version that is currently used by PlanetCatfish.

So this would only make sense to do once we've moved to the new 3.0 version, and only if it's still a big problem.

There's also a risk that the overhead in scanning each new post is going to add to the load on the system (although much less than if we had to scan each post on READ) and this may increase the cost of hosting the forum. Currently Jools doesn't pay the moderators, so the cost for this is lower than increased system usage.

It may be better to add another moderator, if it's really a problem at certain times that isn't covered by the regular moderators.

--
Mats

MatsP · Post by **MatsP** » 22 Jun 2007, 15:43

Marc van Arc wrote:
MatsP wrote: "ugnix" or "argaiv"? Both are (by immediate appearance) "unsensible names"
I see what you mean, yet I was specifically refering to names such as FGYYGFFGYYGF, DHTDHT and so on. So literally letter combinations that make no sense.
They always come with a website and strange interests.

But it is indeed a hard job to keep them out. Thanks for the efforts you're making to do so.

The problem with detecting the difference between a valid name and an invalid one is that you can't really make a MACHINE tell it. You and I that understand English and a few other human languages can tell what looks like a good name and what doesn't. But a machine needs to have strict rules that it can use to determine this. The rather trivial rule of "more than three letters with only vowels or no vowels" is a possibility, but I'm sure that's not particularly accurate anyways.

--
Mats