Azure Jane Lunatic (Azz) đş (![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
azurelunatic) wrote2011-04-03 03:32 pm
azurelunatic) wrote2011-04-03 03:32 pm
Entry tags:
- lj,
- meta,
- metaquoted,
- spam
Dodgy Analogy Theatre: LJ vs. Spammers vs. Spamhaus
Welcome to azurelunaticâs Dodgy Analogy Theatre, in which she attempts to explain the Outsiderâs View of the Current Mail Not Delivered/Spam Problem at LiveJournal! (The information here is gathered from news posts and discussions in news comments and such, and the analogy gets dodgier the further it's strung along.)
First, understand that I picture spammers not as businesspeople who need re-training on how to use the computer, nor as members of a shady underworld, nor even as the stereotypical cave-dwelling geek gone bad. Even though all three of the former are involved in spamming, thatâs not my mental image. I picture them as birds -- in looks, a cross between the worst aesthetic properties and personal habits of the vulture and the chicken: the teenage chicken whose voice is breaking and still retains the shrillness of its chickhood peep, but the full volume and power of its adult voice, and nigh unto zero control of its voicebox. It makes inexperienced theremin players sound tuneful. They favor nesting in the concrete notches of Brutalist architecture, but are opportunists who make their homes anywhere.
Now picture LiveJournal as a home-based business: mostly family, a lot of kids, a few external staff members who come and go. They have a very large back yard with blueberry bushes in it, and they make blueberry jam and ship it off to people who order it off the internet. (These are the comment notifications.)
To make this analogy work, they somewhat unwisely leave the addressed boxes sitting out on the back porch. Most of the time, the box gets its jar of jam, and is closed up and shipped off.
Other times, a spammer (remember, weâre talking birds here) lays its egg in the box, and closes it up safely. And the teenagers doing the shipping runs figure that any closed box is jam ready to go, slap some tape on it, and ship it off, so some very surprised and annoyed customers sometimes get a spammerâs egg delivered when they were expecting a pot of very nice blueberry jam.
All number of different sorts of birds nest in the blueberry bushes, some of them rare and endangered, some of them common but inoffensive, but a lot of them are spammers.
Now, the spammers nesting here are chiefly of two varieties. One of them has interbred with cuckoos. They like to kick the eggs of other birds out of their nests. These are the spammers who see an empty box and lay in it. The other has interbred with pigeons. Their nests are in the bushes, making it hard for anyone whoâs trying to pick blueberries, but they mostly do not bother the open boxes. Instead, they like to fly around the neighborhood. And. Well. Anyone whoâs lived near pigeons knows what happens to the neighborhood.
So sometimes there are customer complaints about the spammer eggs being in the boxes that were supposed to hold jam (comment-spam). When that happens, they send a couple of the kids to the yard with BB guns to pick off some spammers. And occasionally they lay out bait and traps, but theyâve got to be careful, on account of the endangered species also nesting there.
There have been problems in the past with packages containing jam not getting delivered because someone has complained too many times to the post office about getting spammer eggs from LJ. Which, true, they are getting boxes with spammer eggs from LJ, but not because LJ is a business that sends out spammer eggs on purpose instead of jam, but that LJ has spammers nesting in the backyard, and sometimes their eggs get in the boxes by accident. (It doesnât help that the jars for the jam are egg-shaped, either.)
So when too many people complain to the post office about getting spammer eggs from LJ, various local post office branches stop delivering ANY packages from LJ. Which doesnât work so well, and pisses off people even more. (When a person complains to LJ about the spammers, things do get better overall for a while, but the spammers that get hit with the BB guns are the ones flying above the bushes with a clear shot, not necessarily the ones down further in more hidden, and itâs anyoneâs guess whether the one that laid the egg in your box will get hit.)
Now, the Spamhaus situation. Weâve been talking here about the cuckoo-type spammer. The pigeon-type spammer flies about the neighborhood, crapping everywhere (making entries on LJ and emailing totally unrelated people links to the entries). So people complain, and the city gives LJ a notice: Look, there are bushes in your backyard, we understand you canât tear them down on account of the jam and the rare parrots and all, but theyâre infested with spammers that are crapping on peopleâs cars and mailboxes and everything, and if you donât do something about them, fast, weâre going to have to take action.
âYou think we donât know?â LJ says. âYou think we like this?â
The city doesnât really care what LJ thinks, and points out that there have been complaints for years about the pigeon-spammer crap on the neighborhood, and LJâs already been shown the roosting poles of at least fifteen different spammers, but those spammers are still there, night after night.
So LJ diverts more of the teenagers from hauling boxes of jam to sitting with BB guns picking off spammers, but there are far more spammers than teenagers, and in any case the pigeon-spammer hybrids are off on someone elseâs mailbox during the day, and only come back to LJ to roost, and by that time itâs bedtime for the teenagers.
So the city tells the post office: Look, we know that a lot of the packages you deliver from LJâs return address, theyâre legit, they have jam. But theyâre not doing anything about the pigeon-spammer crossbreeds that nest there, and we need to put the hurt on them so theyâll listen and do something. Stop delivering packages with their return address.
Lots of the post offices stop.
FedEx and UPS donât stop. They have a good relationship with LJ way back, and even though LJ sends out a ton of packages, they know itâs a legit business despite the fact that theyâve sometimes got spammer eggs inside.
LJâs customers, the ones who order the jam, mostly stop getting their jam. They are confused and angry. (The ones using FedEx and UPS ((Gmail, AOL)) donât stop getting it, and are confused when all their neighbors start ranting about never getting promised deliveries from LJ.)
LJ realizes that this is serious, and the nets and traps just arenât cutting it anymore. So they scramble to figure out what they can do. Meanwhile, the customers are getting angry, and the post office still isnât delivering.
The customers eventually figure out what is going on. Some are sympathetic; some are mad as hell. At least one customer recounts their own tale of woe involving a spammer infestation and the post office: they had a bunch of pigeon-hybrid spammers in a single tree in their yard, they got a notice to clear them out, they broke out the high-pressure water hoses and soon there was not a single spammer to be seen; the city said âGreat, weâll send an inspector out next month, and if everything is OK, weâll see about getting your packages delivered again.â That was not at all workable, the spammers were gone, and that customer simply started using the return address from their gate on the other side of the block. It wasnât their front door, but it was still a legit return address, and it wasnât an address that the post office was looking for, and since the packages werenât covered with spammer guano, there was no reason to suspect anything was dodgy. And that worked for them.
So someone in LJâs shipping department realized, hey, weâve got a very big lot, and itâs got an edge on another street, so yes indeed we are entitled to use the address from that street! But the bushes were still full of spammers, and while there was a dent, a sizeable dent, in the spammer population, there were still a lot of them, and now the spammer eggs in the undelivered packages at the post office were starting to be a bit of a problem. (Donât leave spammer eggs sitting out. They go very bad very fast.)
At this point, a lot of the people relying on the postal service for their packages started getting theirs, since the secondary address wasnât on the post office watch list. However, there was a problem with the UPS customers. See, LJ had negotiated things with UPS a long time ago, and UPS knew that yeah, LJ sent a lot of packages, and a good number of those had spammer eggs... but they were a legit business, and the spammer eggs were a shipping error, not what was meant to be sent. But now UPS was getting a flood of boxes from a different address, and people started reporting that they had spammer eggs in âem ⌠so what does UPS do? Why, they stop delivering them, is what they do. Spammer eggs, unfamiliar address... sounds dodgy to me!
*facepalm*
So that part did not go as well as it could have gone. But people started getting mail again, and LJ talked with UPS and presumably worked things out.
And LJ realized that one of the new guys theyâd brought in after some of their old people had left -- this new guy was a frickinâ DALEK. (In which Oceanplexian and his default userpic are used as a synecdoche for all the excellent people working on this.)
âHey, dude, can you EXTERMINATE the spammers?â LJ management asked.
âEX-TER-MIN-ATE!!â said the Dalek with great enthusiasm.
âBut carefully! There are RARE PARROTS in those bushes!â
âDIFF-ER-EN-TI-ATE!!â said the Dalek, and began tuning his death ray to only vaporize spammers.
So now LJâs figuring out what to do with a quarter of a million exterminated spammers (that piece is more winceworthy the older I get, but I found it hilarious at age fifteen). They donât make very good eating: all gristly, tasting vaguely of roadkill, and not much meat on âem. Theyâre still shipping out the back-door address, and still waiting on the city to come and say that the spammer infestation is much reduced and theyâre clear again.
And meanwhile, the spammers that were missed in the death ray sweep keep breeding and trying to build new nests...
First, understand that I picture spammers not as businesspeople who need re-training on how to use the computer, nor as members of a shady underworld, nor even as the stereotypical cave-dwelling geek gone bad. Even though all three of the former are involved in spamming, thatâs not my mental image. I picture them as birds -- in looks, a cross between the worst aesthetic properties and personal habits of the vulture and the chicken: the teenage chicken whose voice is breaking and still retains the shrillness of its chickhood peep, but the full volume and power of its adult voice, and nigh unto zero control of its voicebox. It makes inexperienced theremin players sound tuneful. They favor nesting in the concrete notches of Brutalist architecture, but are opportunists who make their homes anywhere.
Now picture LiveJournal as a home-based business: mostly family, a lot of kids, a few external staff members who come and go. They have a very large back yard with blueberry bushes in it, and they make blueberry jam and ship it off to people who order it off the internet. (These are the comment notifications.)
To make this analogy work, they somewhat unwisely leave the addressed boxes sitting out on the back porch. Most of the time, the box gets its jar of jam, and is closed up and shipped off.
Other times, a spammer (remember, weâre talking birds here) lays its egg in the box, and closes it up safely. And the teenagers doing the shipping runs figure that any closed box is jam ready to go, slap some tape on it, and ship it off, so some very surprised and annoyed customers sometimes get a spammerâs egg delivered when they were expecting a pot of very nice blueberry jam.
All number of different sorts of birds nest in the blueberry bushes, some of them rare and endangered, some of them common but inoffensive, but a lot of them are spammers.
Now, the spammers nesting here are chiefly of two varieties. One of them has interbred with cuckoos. They like to kick the eggs of other birds out of their nests. These are the spammers who see an empty box and lay in it. The other has interbred with pigeons. Their nests are in the bushes, making it hard for anyone whoâs trying to pick blueberries, but they mostly do not bother the open boxes. Instead, they like to fly around the neighborhood. And. Well. Anyone whoâs lived near pigeons knows what happens to the neighborhood.
So sometimes there are customer complaints about the spammer eggs being in the boxes that were supposed to hold jam (comment-spam). When that happens, they send a couple of the kids to the yard with BB guns to pick off some spammers. And occasionally they lay out bait and traps, but theyâve got to be careful, on account of the endangered species also nesting there.
There have been problems in the past with packages containing jam not getting delivered because someone has complained too many times to the post office about getting spammer eggs from LJ. Which, true, they are getting boxes with spammer eggs from LJ, but not because LJ is a business that sends out spammer eggs on purpose instead of jam, but that LJ has spammers nesting in the backyard, and sometimes their eggs get in the boxes by accident. (It doesnât help that the jars for the jam are egg-shaped, either.)
So when too many people complain to the post office about getting spammer eggs from LJ, various local post office branches stop delivering ANY packages from LJ. Which doesnât work so well, and pisses off people even more. (When a person complains to LJ about the spammers, things do get better overall for a while, but the spammers that get hit with the BB guns are the ones flying above the bushes with a clear shot, not necessarily the ones down further in more hidden, and itâs anyoneâs guess whether the one that laid the egg in your box will get hit.)
Now, the Spamhaus situation. Weâve been talking here about the cuckoo-type spammer. The pigeon-type spammer flies about the neighborhood, crapping everywhere (making entries on LJ and emailing totally unrelated people links to the entries). So people complain, and the city gives LJ a notice: Look, there are bushes in your backyard, we understand you canât tear them down on account of the jam and the rare parrots and all, but theyâre infested with spammers that are crapping on peopleâs cars and mailboxes and everything, and if you donât do something about them, fast, weâre going to have to take action.
âYou think we donât know?â LJ says. âYou think we like this?â
The city doesnât really care what LJ thinks, and points out that there have been complaints for years about the pigeon-spammer crap on the neighborhood, and LJâs already been shown the roosting poles of at least fifteen different spammers, but those spammers are still there, night after night.
So LJ diverts more of the teenagers from hauling boxes of jam to sitting with BB guns picking off spammers, but there are far more spammers than teenagers, and in any case the pigeon-spammer hybrids are off on someone elseâs mailbox during the day, and only come back to LJ to roost, and by that time itâs bedtime for the teenagers.
So the city tells the post office: Look, we know that a lot of the packages you deliver from LJâs return address, theyâre legit, they have jam. But theyâre not doing anything about the pigeon-spammer crossbreeds that nest there, and we need to put the hurt on them so theyâll listen and do something. Stop delivering packages with their return address.
Lots of the post offices stop.
FedEx and UPS donât stop. They have a good relationship with LJ way back, and even though LJ sends out a ton of packages, they know itâs a legit business despite the fact that theyâve sometimes got spammer eggs inside.
LJâs customers, the ones who order the jam, mostly stop getting their jam. They are confused and angry. (The ones using FedEx and UPS ((Gmail, AOL)) donât stop getting it, and are confused when all their neighbors start ranting about never getting promised deliveries from LJ.)
LJ realizes that this is serious, and the nets and traps just arenât cutting it anymore. So they scramble to figure out what they can do. Meanwhile, the customers are getting angry, and the post office still isnât delivering.
The customers eventually figure out what is going on. Some are sympathetic; some are mad as hell. At least one customer recounts their own tale of woe involving a spammer infestation and the post office: they had a bunch of pigeon-hybrid spammers in a single tree in their yard, they got a notice to clear them out, they broke out the high-pressure water hoses and soon there was not a single spammer to be seen; the city said âGreat, weâll send an inspector out next month, and if everything is OK, weâll see about getting your packages delivered again.â That was not at all workable, the spammers were gone, and that customer simply started using the return address from their gate on the other side of the block. It wasnât their front door, but it was still a legit return address, and it wasnât an address that the post office was looking for, and since the packages werenât covered with spammer guano, there was no reason to suspect anything was dodgy. And that worked for them.
So someone in LJâs shipping department realized, hey, weâve got a very big lot, and itâs got an edge on another street, so yes indeed we are entitled to use the address from that street! But the bushes were still full of spammers, and while there was a dent, a sizeable dent, in the spammer population, there were still a lot of them, and now the spammer eggs in the undelivered packages at the post office were starting to be a bit of a problem. (Donât leave spammer eggs sitting out. They go very bad very fast.)
At this point, a lot of the people relying on the postal service for their packages started getting theirs, since the secondary address wasnât on the post office watch list. However, there was a problem with the UPS customers. See, LJ had negotiated things with UPS a long time ago, and UPS knew that yeah, LJ sent a lot of packages, and a good number of those had spammer eggs... but they were a legit business, and the spammer eggs were a shipping error, not what was meant to be sent. But now UPS was getting a flood of boxes from a different address, and people started reporting that they had spammer eggs in âem ⌠so what does UPS do? Why, they stop delivering them, is what they do. Spammer eggs, unfamiliar address... sounds dodgy to me!
*facepalm*
So that part did not go as well as it could have gone. But people started getting mail again, and LJ talked with UPS and presumably worked things out.
And LJ realized that one of the new guys theyâd brought in after some of their old people had left -- this new guy was a frickinâ DALEK. (In which Oceanplexian and his default userpic are used as a synecdoche for all the excellent people working on this.)
âHey, dude, can you EXTERMINATE the spammers?â LJ management asked.
âEX-TER-MIN-ATE!!â said the Dalek with great enthusiasm.
âBut carefully! There are RARE PARROTS in those bushes!â
âDIFF-ER-EN-TI-ATE!!â said the Dalek, and began tuning his death ray to only vaporize spammers.
So now LJâs figuring out what to do with a quarter of a million exterminated spammers (that piece is more winceworthy the older I get, but I found it hilarious at age fifteen). They donât make very good eating: all gristly, tasting vaguely of roadkill, and not much meat on âem. Theyâre still shipping out the back-door address, and still waiting on the city to come and say that the spammer infestation is much reduced and theyâre clear again.
And meanwhile, the spammers that were missed in the death ray sweep keep breeding and trying to build new nests...
no subject
But ouch at the figure. I never asked how many, and I can't believe what the abuse team must now be going through.
no subject
!!!!
no subject
no subject
no subject
no subject
It's made by
no subject
no subject
ETA because I can't spell.
no subject
*feels rich*
no subject
no subject
no subject
no subject
no subject
Sure! Spammers really do need to be exterminated.
no subject
no subject
no subject
no subject
Tracking the stats on LJ since June '05, originally posted by
Stats today:
# ... updating in last 30 days: 1210325
# ... updating in last 7 days: 543697
# ... updating in past 24 hours: 168508
About the same number updating in the last 30 days as in May of 2006. (Anyone think LJ has as much real activity now as it did in mid-06? Is ONTD really that popular?) If the spam has been artificially inflating LJ's usage numbers, making it more appealing to advertisers, that would explain why they haven't dealt with the problem before now.
Other posts:
LiveJournal is shrinking. In new faux-academic style! in Aug 2007 (just after
Strikethrough)Livejournal population statistics, round 2. Mar 2008.
no subject
ONTD itself probably wouldn't count for much updating, though, as that's only a single comm.
Pure speculation, but another reason why they didn't deal with it is possibly because it's expensive for them to do so, so they didn't until there was a compelling reason to.
no subject
I think anecdotal evidence, especially as much widespread agreement as there is for "LJ is slowing down," is useful, but needs to be taken with a large grain of salt, and can't be used for any more specific purposes.
possibly because it's expensive for them to do so
Well, yes--but they have been getting complaints about the spam journals and spam comments for years. Allowing things that are against the TOS, and that make paying customers unhappy, is a bad idea--unless they're helping the business somehow. Like by providing the illusion of high activity levels.
They didn't just now discover the tools to deal with mass spammers; they just now decided it was worth implementing them. It wasn't worth doing so when they only cluttered up the Latest Posts and content search pages, no matter how useless that made the pages.
no subject
Well, I just think it isn't useful if we have reason to believe that a majority of the activity on LJ is Russian, though.
Yes, but probably the more relevant metric for advertisers is the viewers, not the number of journals. LJ definitely has stats on the number of people viewing.
Even if the tools are not new, it doesn't mean that there aren't extra costs. I don't think there is compelling evidence that LJ is using the spammers to provide the illusion of high activity elevels.
no subject
I do not like that development culture one bit. It's exhausting and demoralizing; one knows that there are plenty of more important things to do, and occasionally things to make it better do happen -- but they happen at a slow enough rate that the complaints about putting fish and whistles first are (at least to my perspective) justified.
I do not know what other tools were/are available to LJ's spamwhackers, but DW's antispam system as inherited from LJ without modification was usable, but painful. While I was a volunteer, there was talk of things that were being done to address spammers on a level more advanced than picking them off with BB guns -- but there were no details discussed in front of the general volunteer population, and even if there had been, it would be the sort of thing that I wouldn't feel comfortable repeating. I know there was a certain amount of "Right, so this one email address/domain just registered a bazillion accounts, zapping all of them!" but ... still BB territory.
I remember one volunteer suggesting that maybe the sorts of spammers that just made spam entries in their own journals without bothering actual users were not so bad; I remember Tupshin quite memorably ranting on the theme of NOT ON MY DAMN SERVERS and the general feeling that the person who had suggested this should go wash their keyboard off with soap.
Without having been a member of the abuse team ... it felt like volunteer time was cheap, ops time was expensive, and volunteer time had historically "worked" (for values of "worked" that meant there wasn't much yelling in news posts, or whatever) to keep spam down to a dull roar.
It just felt like the sort of "sure, the support team is complaining about this, but what do they know, lowest priority" brushoff.
I don't know how it is there now, or if this is in any way accurate other than my emotional reactions, but.
no subject
no subject
no subject
(Don't get me started on my job's data archiving policies & procedures. Just aaagh.)
I do understand that it's not easy. There is no anti-pigeon-net that lets parrots through. There is no poisoned pidgeon chow that parrots won't eat by accident. Kids with bb guns is the most *accurate* way to address the problem; it's just not effective, once your pigeon population reaches a certain level.
Part of me thinks they could fix something by requiring email verification; I thought that wasn't part of the LJ signup process. (Didn't need it when invite codes were around.) Or maybe requiring captcha verification for posts, or for the first post in a journal, or more than 5 in a day, or something. (Can't use captcha for post-by-mail, so I don't know how that would work. OTOH, I don't know if any of the spambots are posting by mail.)
I'd like to believe anything I could come up with has already been thought of by people who actually understand the code, and been considered by Those Who Make Important Decisions, and the reason they're not being done is because they won't work for some reason. However, after watching LJ's Important Decisions for the last few years, I'm a lot less convinced of this.
no subject
They already require email verification for a lot of things, but I think you may be right and it should probably be required for posting to a journal, too. But, considering I watch communities that get entry spam and see comment spam (which requires email verification) I know that it's not too high of a bar to clear.
no subject
http://www.livejournal.com/support/faqbrowse.bml?faqid=11
no subject
no subject
And yeah, I figure that the obvious things--Captcha, verification--have fairly easy automated workarounds of some sort. I'm kind of clutching at straws, trying to think of something that could help, and realize that no, I don't know enough about how the process works to come up with anything useful.
no subject
no subject
no subject
no subject
There's no need for them to deliberately cultivate spam in order to keep their activity figures high. They just don't care if all the American and international users complain, or even if they all, to a woman, walk out in disgust. Russian users I think are less bothered by spam (and advertising) because they're used to a web experience that is littered with all kinds of dodgy stuff. Even if Russian users care about spam, they almost certainly don't care about the pigeon kind of spammers who don't interrupt their discussions.
So I'm pretty clear that LJ has been incompetent or apathetic here rather than evil. I have seen the suggestion that sites that host (anti-)social games are easier targets for DDoS attacks. Also that the latest round of outages is revenge by the big-league spammers for LJ finally cracking down on them in response to Spamhaus. Both of those seem more plausible than LJ actively encouraging spam!
no subject
I absolutely *do not* believe LJ sought or in any way encouraged spam journals. I only think they might not have been diligent about seeking ways to avoid them, until they were blocked by Spamhaus. I strongly believe they've been motivated by apathy with a small side of greed--"How much would it cost us to fix this? Versus, wait, its giving how many hit counts per day? And it would cost what to stop?" (No, LJ, you cannot pitch the constant flyovers of the pigeons as "organic sunscreen.")
I don't think they sought spam. I think that, when people first started complaining (fairly soon after invite codes went away), they shrugged it off as "not really bothering anyone, and besides, hard to figure out how to code to stop" and when it grew to a level that (1) it was bothering people and (2) would be very expensive & difficult to code to stop, they looked for simple solutions, found none, and then looked for excuses not to fix it.
Excuse #1: Damn, that's a lot of work for nebulous benefit. "Better user satisfaction" is not a line item we can bring to the stockholders.
Excuse #2: It's not causing any *real* problems. I mean, so journals are full of spam ... who cares? They eat a little bit of server space and that's all. So people get the occasional spam comment ... tell 'em they can set their posts to not allow anon comments and those'll go away.
Excuse #3: Um... at least it's site activity? It brings visibility?
Which is both weak and rather ridiculous, but I've seen dumber ideas brought up in corporate meetings, on the theory that, "if it's hard to fix, redefine the problem as an advantage and maybe it won't be a problem anymore."
no subject
I think it's more that the content on Russophone LJ so compelling (especially with the way staffers pick through the content to highlight the best posts, and with all the well known political, journalistic, and media figures blogging on LJ), and that's why the tolerate the spam.
I actually think a similar phenomenon exists on LJ. People are staying in spite of the spam (and the sometimes intrusive advertising).
no subject
I took it as my displayed name at some point during my administration of LJ Suggestions.
Maybe "The rule is, jam to-morrow and jam yesterday - but never jam to-day" -- although that's for the promised thing that will never arrive, except the timeframe for the promise is a day, not three years.
no subject
no subject
Yeah, it does not surprise me that the latest ones have been part of that.
no subject
no subject
no subject
no subject
no subject
no subject
no subject
no subject
But I really do think of spammers as very obnoxious and ugly birds.
no subject
Edit:
PRO-CRE-ATE!
no subject