azurelunatic | "not currently in violation" - what #Twitter really needs to do about its abuse problem

Entry tags:

"not currently in violation" - what #Twitter really needs to do about its abuse problem

Background: So there's a current up-in-arms regarding really skeevy crap on Twitter. It goes like this:

Someone (often female) says something that gets the attention of abusive asshats.
Abusive asshats (often male) say things on Twitter that are entirely possibly legally actionable.
Their target complains, usually to Twitter, with screenshots and links.
Support volume being what it has to be, it takes a while to get notice.
The abusive asshat cleans up their account in the interim.
Twitter comes back and says that Abusive Asshat's account is "not currently in violation" of Twitter's terms of service.
This is remarkably unhelpful to the person who's been the target of all this abuse.

Now.

I have never been a member of LiveJournal's Abuse Prevention team. I am not a member of Dreamwidth's Terms of Service team. (I am a Dreamwidth spamwhacker, which is a partner department.)

From my experience in conversing with various then-current and former members of LiveJournal's Abuse team, I can say quite firmly that accepting accuser-sourced screenshots of content that is against the Terms of Service of a website is not, and can never be, a form of evidence that can be solely admissable when enacting penalties against an offending account.

Why? Screenshots can be faked.

I am as certain as I can be without having been personally there and witnessed the whole thing go down that 99% of the women on Twitter reporting that jacked-up asshats are promising to enact various forms of appalling violence to them (most of it rapey) have legit complaints. I've seen enough of it happening to know that it's happening and not being exaggerated a large majority of the time. It's got to be against the rules.

But the jackholes in question are sometimes canny enough to make their violations disappear from their Twitter accounts before it gets taken official notice of, and then all Twitter has is the word of the complainant and the screenshot.

There's a technical solution for this, and it's not a "report abuse" button that can be gamed by someone with a huge following on their side.

The technical solution for this is a "preserve and report tweet" button that caches the offending tweet on Twitter's servers, and initiates the reporting process, where the complainant fills out the appropriate forms, making reference to the secured and admissible cached tweet.

After this, no matter if the offender cleans up his account, there is still a record that he said this thing, assuming someone initiated the reporting process. Furthermore, the complainant could be given a Twitter case number to give to law enforcement, and law enforcement could then request testimony from Twitter that the offending tweet was made, in case it's something deserving of criminal or civil charges. The cached copy would remain accessible to Twitter's speaker-to-cops department even after Twitter suspended the account for violations.

Flat | Top-Level Comments Only

As a former APT member, I definitely concur with your reasoning on screenshots being non-actionable evidence. From my experience in the warranty/insurance industry, I can tell you that most people can't forge their way out of a paper bag -- but there are probably people who can fake a screenshot convincingly enough (actually, all it takes is a modified HTML file...)

However, I'm not sure how well a function to preserve user data for abuse prevention purposes would fit within Twitter's Terms of Service and Privacy Policy, as well as various privacy laws (especially in Europe). There doesn't seem to be anything in their TOS that grants or implies an absolute right to delete data, but there's also nothing that grants Twitter the right to indefinitely preserve user data (even the license granted by the user in section 5 isn't irrevocable).

Their privacy policy specifies that you can delete your account (and thus your data) from their servers, but does mention a 30-day delay. It's possible a "preserve for evidence" feature could work with a maximum retention of 30 days or something like that.

Or, of course, they could just rewrite their TOS.

I suspect that a complainant-initiated "preserve this shit" button, rather than an "all deleted tweets will be preserved" policy, would mollify many privacy complaints.

Data retention. Such a fascinating topic.

but there are probably people who can fake a screenshot convincingly enough

Fake screenshots of tweets are so abundant (especially on Tumblr) it's not even funny. Standard fonts, identical formatting for every tweet, it's not ten minutes of work in any graphic editor.

(Added a line at the bottom because suspension & cops)

All I can think of now is "The Cop Whisperer"... which isn't even A Thing.

One of the most bizarre endearments/epithets I've ever received was "our little techie" from the Anchorage, Alaska cops.

This is an elegant suggestion, practical and meaningful. Therefore I doubt it'll be implemented. I have no faith in the system.

My thoughts exactly.

THIS IS SMART AS FUCK

From an on-Twitter discussion involving

eskoala and

lalonde; some paraphrasing and expansion:

Would it be fair to hold people accountable for everything they've said in haste and repented at leisure?

For one, it's already been said, and if it's been complained about, it's had an effect on other people -- is it fair to let someone get off scot-free for saying something really awful if they delete it out of a fear of getting in trouble?

Anything that's both been complained about and found in violation of the rules ought not to have been said in the first place.

The rules should include consequences that acknowledge that a tweet being deleted is, on balance, probably better than it staying up, and account for isolated incidents and first offenses as well as accounting for situations where there's a clear pattern of abuse. "We saw you do that; don't do it again, under any of your accounts" is a legitimate consequence.

"Would it be fair to hold people accountable for everything they've said in haste and repented at leisure?"

Also, would it be fair to hold people accountable for money they stole from their companies and then paid back? Or if they attacked people physically but didn't leave marks?

If someone commits a legally actionable act then destroys the evidence in a forest when only their victim is watching, is it still legally actionable?

...Yes. Yes, yes it is. And if they just said something stupid and hasty one time, not a pattern of illegal behaviour, that'll come out in the investigation.

As I briefly mentioned on Twitter, I'm pretty sure that deleted tweets are not beyond the reach of Twitter, from a technical perspective at least.

I'd be very surprised if the people they have doing abuse requests have direct access to any of their tech - they probably just get a couple of extra privs to allow them to see locked accounts and flag accounts as suspended. This is a pity, since a more technically-minded abuse team would be able to fish these things out themselves.

In the end, I suspect Twitter just don't care - if there's nothing bad showing when they look, that's sufficient for them to brush it to one side. I wouldn't hold your breath for any meaningful improvement in this area. Apparently part of privacy being dead is that guys get to threaten and abuse women online and that's okay. :o(

You're probably right, there. As a tech company insider, I'd be a little surprised if they didn't keep a lot of their data by default, at least for a certain amount of time. It's in the nature of sysadmins to be paranoid about disaster recovery, at least to a certain extent, so lazy deletion is often the case. That said, it's also in the nature of building these systems that the user interface for finding stuff that hasn't yet been lazily deleted is mainly along the lines of "open a command window, ssh to this host, run this program and enter this password, and then you get an XML file dump that contains approximately 100 times the amount of data you actually need. Oh, and if you don't have access to a framework to interpret that data, you may need to write a perl script yourself."

Also, there's the Twitter firehose feed, which several external companies pay for, and which ships off every. single. tweet. for, well, whatever the other company wants to do with it (private accounts possibly excepted). I know that Yahoo search used this for a while to display trending topics and celebrity tweets in their search results. Point being, if they have this available for external access, it's exceedingly unlikely that they haven't built some kind of in-house data warehousing and analysis system using it, or something even more closely tied to the online system.

In theory, they should be deleting the data when a user deletes the message. This isn't an issue of 'best practices', you understand, it's an issue of EU law, which is pretty strict on this.

In practice, it's impossible to really know without actually working there. Certainly it would be unreasonable to expect a 'click delete and it's instantly gone from everywhere' policy, given Twitter use a bunch of technologies that involve clustering, caching, and eventual consistency, but that's not a meaningful way of preserving the data.

Not everyone who pays for firehose access actually gets the full feed - they discriminate on pricing based on what percentage of the feed you actually get. They also don't seem to offer replay options (that I've seen), and while I'm sure they're doing interesting analytics work on their stored data their legal obligations re: deleted messages wouldn't change.

Here's something that I've not seen discussed, though: the US Library of Congress maintains an archive of all (public) tweets. I suspect this isn't delivered via the firehose, but there's certainly no way they could honour later delete requests. (See: http://blogs.loc.gov/loc/2013/01/update-on-the-twitter-archive-at-the-library-of-congress/)

Yeah, the various jurisdictional legal stuff gets complicated, and there's also the possibility for internal abuse if they made "deleted" tweets easy to access by abuse or support personnel. Probably what it boils down to is you'd have to sue Twitter to get any kind of traction on the matter (which certainly may happen someday, but probably will take a much larger case than this would be, and the lawyers would have a field day with those jurisdictional issues).

Also, I was under the vague impression that the LOC archive was fed by the firehose, but I wouldn't want to say for certain.

There's also that this proposal is being made by a transphobe who, no doubt, wants to use it to silence trans women by reporting their tweets as abuse. The problem with a tool like this is that abusers will inevitably use it against abused people... technology doesn't change the power dynamic there.

The fact that abusers can use it against abused people doesn't mean that their reports will be given any merit, unless the abused are also abusing others themselves.

Personally, I think Azz's idea is great.

Well, except when the abused group is systematically disbelieved and denied credibility.

My idea here is in conversation with a proposed Twitter report-abuse button that would be vulnerable to the death of the thousand clicks -- anyone who got massively unpopular all at once would run the risk of suspension. That idea is horrible unless there's human oversight baked in, which very much not so much.

Given PayPal's hair-trigger suspension of anyone mentioning "Iran" or "Persia," I don't have a lot of hope.

Yes export-control and ritualized hate of the designated-enemy-of-the-day is not the same as harassment, but no, not a lot of hope here that it will be used for anything but a let-us-appear-to-be-seen-doing-something tool.

If you wouldn't mind to clarify, which proposal by whom?

This one: http://www.change.org/en-GB/petitions/twitter-add-a-report-abuse-button-to-tweets?utm_campaign=twitter_link_action_box&utm_medium=twitter&utm_source=share_petition

This proposal should be implemented as-is.

...oh, wait, this isn't another Suggestions post? But it's such a good idea!

Interestingly, just the other evening, I was reading something that reflects on Twitter's organizational culture which pertains.

I had just learned about the late lamented @Anti_Racism_Dog, and googled about it, and turned up this fascinating article by Malcolm Harris from 2012. It contains, as he puts it, four case studies, of which @Anti_Racism_Dog was one. Far more interesting to me was the one about his prosecution:

On 11 October 2011, I was arrested along with over seven hundred other people on the Brooklyn Bridge as part of an Occupy Wall Street mobilisation. The Manhattan District Attorney charged me with disorderly conduct, the lowest possible violation on the books. But despite the low stakes, the DA’s office in the course of their investigation subpoenaed Twitter for my account details, seeking to verify and introduce as evidence against me tweets that the prosecutors allege conflict with my anticipated defence. The state has argued that the account does not belong to me and therefore I have no right to intervene on my own behalf to dispute the subpoena. At the time of writing, the judge has agreed with the prosecution, though Twitter has stepped in to support my right to defend the account from the prying eyes of government.

[...]

When Twitter forwarded me the first subpoena (which the DA’s office asked the company not to disclose), it was targeted at the account
@destructuremal, which was said to be operated by one Malcolm Harris. But one of the main reasons the prosecutors filed a subpoena (rather than just printing out pages of my unprotected tweets and waving them in my face come my trial date) was to verify the account was mine through the user details that aren’t publicly available. Just because the account has my name and picture on it doesn’t mean it’s mine – after all, celebrities attract parody accounts that display their names and pictures, too.

After tweeting out the subpoena document, I dropped the user name @destructuremal and switched my account name to @getsworse. Another user picked up @destructuremal, kept my name but changed the picture to an attractive woman’s torso, and started tweeting what I have to admit was solid mockery of me. Twitter allows you to change your user name without losing any of the rest of your account history, so I could maintain my followers while changing the one useful piece of information that the state actually had. In fact, Twitter allows you to change all the information the user has access to without actually changing the account. There must be, for lack of a better word, a soul to every Twitter account, a piece of information that doesn’t change even if everything else about the account does. But these numbers are hidden somewhere on a server, and if the government wants them, it has to know what to ask for.

Weeks after I changed my account name, the DA’s office issued another subpoena that was more narrowly tailored with regard to the information it was seeking (it specified only public tweets and verifying account details, not private DMs), but this time it was targeted at @getsworse [...] Twitter forwarded me the second subpoena [...] But during this whole time, the court was still ruling on the subpoena directed at the account @destructuremal. So if both my and Twitter’s motion and appeals fail and the court compels Twitter to turn over the requested account details, I’m not confident anyone can tell me what will happen. There’s a decent possibility that prosecutors could be left with nothing more than a bunch of pictures of an unknown woman’s breasts. As it stands, there’s nothing in the document that would lead Twitter to my current account any more than any of the accounts operated by various Malcolm Harrises, including a famous fashion designer and a lady-killing teenage soccer phenom. Unless the social media giant is over-eager to cooperate with the prosecution – and so far they’ve been significantly less than so – I don’t know how any prosecutor’s office could successfully subpoena a Twitter account that has no permanent accessible characteristics.

Now I found this fascinating, and looked further into this case. I found at one point a tweet from before the article, from some random dude mocking Harris that Twitter wasn't going to go to the mat for him. But that is exactly what Twitter did. Twitter only finally complied with the subpoena after quite the court battle.

This is in accord with the characterization of Twitter in this Reason article:

Twitter, in contrast, governs in much less proscriptive fashion. “All Content, whether publicly posted or privately transmitted, is the sole responsibility of the person who originated such Content,” its TOS reads. “We may not monitor or control the Content posted via the Services and, we cannot take responsibility for such Content. Any use or reliance on any Content or materials posted via the Services or obtained by you through the Services is at your own risk.” In its Rules section, Twitter reaffirms this hands-off policy: “We do not actively monitor user’s content and will not censor user content, except in limited circumstances.”

Those limited circumstances mostly involve impersonating other people or disclosing their private and confidential information, committing trademark violations or copyright infringement, and posting “direct, specific threats of violence against others.” Harvey elaborates: “You cannot say to a specific person, ‘I’m coming over to your house right now with a baseball bat to kill you.’ ”

[...]

Twitter, on the other hand, seemed content to let users fend for themselves. “Twitter recognizes that it is not skilled at judging content disputes between individuals,” company co-founder Biz Stone explained on a message board where people were discussing the matter. “Determining the line between update and insult is not something that Twitter, nor a crowd, would do well.”

But if Twitter’s newly articulated policy failed to protect its users from other users, it also effectively protected users from Twitter itself. Though the company “reserve[s] the right at all times (but will not have an obligation) to remove or refuse to distribute any Content” and “suspend or terminate users,” it constrains its ability to exercise these powers by characterizing itself in the way it does. Indeed, when you proclaim that you’re “the free speech wing of the free speech party,” as CEO Dick Costolo often does, that creates certain expectations. When you regularly assert that you do not mediate content, users will assume that you do not mediate content, even in cases when that content is highly objectionable.

[...]

Over time, Trust & Safety and Twitter’s legal department (which now oversees Trust & Safety) have developed policies that allow Twitter to comply with country-specific free speech laws and yet still put users at the forefront. “Our mission statement within Trust & Safety is to ensure user trust, protect user rights, and craft and enforce policies to reduce legal risk,” Harvey says. “And that’s legal risk for Twitter and for users.”

Thus, when the company receives a court order or subpoena to disclose user information, it alerts the user in question, so that he or she has an opportunity to contest the disclosure before it happens. When local laws compel Twitter to delete posts or ban accounts in a given country, it does so in a narrow, transparent fashion, blocking the material locally rather than globally, and replacing the blocked material with a grayed-out alert box indicating that an act of officially mandated government censorship has taken place.

And when third parties aren’t armed with subpoenas, court orders, or other valid legal processes, Twitter typically does not take action. In September 2012, for example, Al-Shabaab, a Somali-based affiliate of Al Qaeda, tweeted photos showing the corpses of several Kenyan soldiers it took credit for killing in combat. A few weeks later, seven Republican members of Congress sent a letter to the FBI urging it to make Twitter remove Al-Shabaab’s account and those of various other “Specially Designated Global Terrorist entities.” As of early January, the five accounts the lawmakers complained about continue to function on Twitter. A letter to the FBI, even by influential types on Capitol Hill, does not carry the force of a court order. Twitter was unmoved.

[...]

But so far, Twitter’s laissez-faire attitude toward online discourse has been its greatest business proposition, so much so that Twitter’s chief legal counsel, Alex Macgillivray, told The New York Times that he views Twitter’s commitment to free speech as a “competitive advantage.”

[...]

I recommend reading the whole article.

I don't think you're going to get this, because I don't think Twitter is interested in enforcing any more of its ToS than it absolutely has to, for a gun-to-the-head definition of "has to". More than merely not interested, actively opposed as a matter of both moral principle and business plan.

That would be an excellent way of preserving those things when they happen, and hopefully prevent asshattery and abuse of the abuse system.

Ooh. I really like this and have added it to my storify about related stuff. https://storify.com/andustar/twitter-moderation/

Flat | Top-Level Comments Only

"not currently in violation" - what #Twitter really needs to do about its abuse problem

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject