"not currently in violation" - what #Twitter really needs to do about its abuse problem

[azurelunatic]

Background: So there's a current up-in-arms regarding really skeevy crap on Twitter. It goes like this:

Someone (often female) says something that gets the attention of abusive asshats.
Abusive asshats (often male) say things on Twitter that are entirely possibly legally actionable.
Their target complains, usually to Twitter, with screenshots and links.
Support volume being what it has to be, it takes a while to get notice.
The abusive asshat cleans up their account in the interim.
Twitter comes back and says that Abusive Asshat's account is "not currently in violation" of Twitter's terms of service.
This is remarkably unhelpful to the person who's been the target of all this abuse.


Now.


I have never been a member of LiveJournal's Abuse Prevention team. I am not a member of Dreamwidth's Terms of Service team. (I am a Dreamwidth spamwhacker, which is a partner department.)

From my experience in conversing with various then-current and former members of LiveJournal's Abuse team, I can say quite firmly that accepting accuser-sourced screenshots of content that is against the Terms of Service of a website is not, and can never be, a form of evidence that can be solely admissable when enacting penalties against an offending account.

Why? Screenshots can be faked.

I am as certain as I can be without having been personally there and witnessed the whole thing go down that 99% of the women on Twitter reporting that jacked-up asshats are promising to enact various forms of appalling violence to them (most of it rapey) have legit complaints. I've seen enough of it happening to know that it's happening and not being exaggerated a large majority of the time. It's got to be against the rules.

But the jackholes in question are sometimes canny enough to make their violations disappear from their Twitter accounts before it gets taken official notice of, and then all Twitter has is the word of the complainant and the screenshot.

There's a technical solution for this, and it's not a "report abuse" button that can be gamed by someone with a huge following on their side.

The technical solution for this is a "preserve and report tweet" button that caches the offending tweet on Twitter's servers, and initiates the reporting process, where the complainant fills out the appropriate forms, making reference to the secured and admissible cached tweet.

After this, no matter if the offender cleans up his account, there is still a record that he said this thing, assuming someone initiated the reporting process. Furthermore, the complainant could be given a Twitter case number to give to law enforcement, and law enforcement could then request testimony from Twitter that the offending tweet was made, in case it's something deserving of criminal or civil charges. The cached copy would remain accessible to Twitter's speaker-to-cops department even after Twitter suspended the account for violations.

Comments

[ryan]

As a former APT member, I definitely concur with your reasoning on screenshots being non-actionable evidence. From my experience in the warranty/insurance industry, I can tell you that most people can't forge their way out of a paper bag -- but there are probably people who can fake a screenshot convincingly enough (actually, all it takes is a modified HTML file...)

However, I'm not sure how well a function to preserve user data for abuse prevention purposes would fit within Twitter's Terms of Service and Privacy Policy, as well as various privacy laws (especially in Europe). There doesn't seem to be anything in their TOS that grants or implies an absolute right to delete data, but there's also nothing that grants Twitter the right to indefinitely preserve user data (even the license granted by the user in section 5 isn't irrevocable).

Their privacy policy specifies that you can delete your account (and thus your data) from their servers, but does mention a 30-day delay. It's possible a "preserve for evidence" feature could work with a maximum retention of 30 days or something like that.

Or, of course, they could just rewrite their TOS.

[azurelunatic]

(Added a line at the bottom because suspension & cops)

[amadi]

This is an elegant suggestion, practical and meaningful. Therefore I doubt it'll be implemented. I have no faith in the system.

[chagrined]

THIS IS SMART AS FUCK

[azurelunatic]

From an on-Twitter discussion involving eskoala and lalonde; some paraphrasing and expansion:

Would it be fair to hold people accountable for everything they've said in haste and repented at leisure?

For one, it's already been said, and if it's been complained about, it's had an effect on other people -- is it fair to let someone get off scot-free for saying something really awful if they delete it out of a fear of getting in trouble?

Anything that's both been complained about and found in violation of the rules ought not to have been said in the first place.

The rules should include consequences that acknowledge that a tweet being deleted is, on balance, probably better than it staying up, and account for isolated incidents and first offenses as well as accounting for situations where there's a clear pattern of abuse. "We saw you do that; don't do it again, under any of your accounts" is a legitimate consequence.

[wibbble]

As I briefly mentioned on Twitter, I'm pretty sure that deleted tweets are not beyond the reach of Twitter, from a technical perspective at least.

I'd be very surprised if the people they have doing abuse requests have direct access to any of their tech - they probably just get a couple of extra privs to allow them to see locked accounts and flag accounts as suspended. This is a pity, since a more technically-minded abuse team would be able to fish these things out themselves.

In the end, I suspect Twitter just don't care - if there's nothing bad showing when they look, that's sufficient for them to brush it to one side. I wouldn't hold your breath for any meaningful improvement in this area. Apparently part of privacy being dead is that guys get to threaten and abuse women online and that's okay. :o(

[tim]

There's also that this proposal is being made by a transphobe who, no doubt, wants to use it to silence trans women by reporting their tweets as abuse. The problem with a tool like this is that abusers will inevitably use it against abused people... technology doesn't change the power dynamic there.

[aedifica]

This proposal should be implemented as-is.

...oh, wait, this isn't another Suggestions post? But it's such a good idea!

[siderea]

Interestingly, just the other evening, I was reading something that reflects on Twitter's organizational culture which pertains.

I had just learned about the late lamented @Anti_Racism_Dog, and googled about it, and turned up this fascinating article by Malcolm Harris from 2012. It contains, as he puts it, four case studies, of which @Anti_Racism_Dog was one. Far more interesting to me was the one about his prosecution:

On 11 October 2011, I was arrested along with over seven hundred other people on the Brooklyn Bridge as part of an Occupy Wall Street mobilisation. The Manhattan District Attorney charged me with disorderly conduct, the lowest possible violation on the books. But despite the low stakes, the DA’s office in the course of their investigation subpoenaed Twitter for my account details, seeking to verify and introduce as evidence against me tweets that the prosecutors allege conflict with my anticipated defence. The state has argued that the account does not belong to me and therefore I have no right to intervene on my own behalf to dispute the subpoena. At the time of writing, the judge has agreed with the prosecution, though Twitter has stepped in to support my right to defend the account from the prying eyes of government.

[...]

When Twitter forwarded me the first subpoena (which the DA’s office asked the company not to disclose), it was targeted at the account
@destructuremal, which was said to be operated by one Malcolm Harris. But one of the main reasons the prosecutors filed a subpoena (rather than just printing out pages of my unprotected tweets and waving them in my face come my trial date) was to verify the account was mine through the user details that aren’t publicly available. Just because the account has my name and picture on it doesn’t mean it’s mine – after all, celebrities attract parody accounts that display their names and pictures, too.

After tweeting out the subpoena document, I dropped the user name @destructuremal and switched my account name to @getsworse. Another user picked up @destructuremal, kept my name but changed the picture to an attractive woman’s torso, and started tweeting what I have to admit was solid mockery of me. Twitter allows you to change your user name without losing any of the rest of your account history, so I could maintain my followers while changing the one useful piece of information that the state actually had. In fact, Twitter allows you to change all the information the user has access to without actually changing the account. There must be, for lack of a better word, a soul to every Twitter account, a piece of information that doesn’t change even if everything else about the account does. But these numbers are hidden somewhere on a server, and if the government wants them, it has to know what to ask for.

Weeks after I changed my account name, the DA’s office issued another subpoena that was more narrowly tailored with regard to the information it was seeking (it specified only public tweets and verifying account details, not private DMs), but this time it was targeted at @getsworse [...] Twitter forwarded me the second subpoena [...] But during this whole time, the court was still ruling on the subpoena directed at the account @destructuremal. So if both my and Twitter’s motion and appeals fail and the court compels Twitter to turn over the requested account details, I’m not confident anyone can tell me what will happen. There’s a decent possibility that prosecutors could be left with nothing more than a bunch of pictures of an unknown woman’s breasts. As it stands, there’s nothing in the document that would lead Twitter to my current account any more than any of the accounts operated by various Malcolm Harrises, including a famous fashion designer and a lady-killing teenage soccer phenom. Unless the social media giant is over-eager to cooperate with the prosecution – and so far they’ve been significantly less than so – I don’t know how any prosecutor’s office could successfully subpoena a Twitter account that has no permanent accessible characteristics.

Now I found this fascinating, and looked further into this case. I found at one point a tweet from before the article, from some random dude mocking Harris that Twitter wasn't going to go to the mat for him. But that is exactly what Twitter did. Twitter only finally complied with the subpoena after quite the court battle.

This is in accord with the characterization of Twitter in this Reason article:

Twitter, in contrast, governs in much less proscriptive fashion. “All Content, whether publicly posted or privately transmitted, is the sole responsibility of the person who originated such Content,” its TOS reads. “We may not monitor or control the Content posted via the Services and, we cannot take responsibility for such Content. Any use or reliance on any Content or materials posted via the Services or obtained by you through the Services is at your own risk.” In its Rules section, Twitter reaffirms this hands-off policy: “We do not actively monitor user’s content and will not censor user content, except in limited circumstances.”

Those limited circumstances mostly involve impersonating other people or disclosing their private and confidential information, committing trademark violations or copyright infringement, and posting “direct, specific threats of violence against others.” Harvey elaborates: “You cannot say to a specific person, ‘I’m coming over to your house right now with a baseball bat to kill you.’ ”

[...]

Twitter, on the other hand, seemed content to let users fend for themselves. “Twitter recognizes that it is not skilled at judging content disputes between individuals,” company co-founder Biz Stone explained on a message board where people were discussing the matter. “Determining the line between update and insult is not something that Twitter, nor a crowd, would do well.”

But if Twitter’s newly articulated policy failed to protect its users from other users, it also effectively protected users from Twitter itself. Though the company “reserve[s] the right at all times (but will not have an obligation) to remove or refuse to distribute any Content” and “suspend or terminate users,” it constrains its ability to exercise these powers by characterizing itself in the way it does. Indeed, when you proclaim that you’re “the free speech wing of the free speech party,” as CEO Dick Costolo often does, that creates certain expectations. When you regularly assert that you do not mediate content, users will assume that you do not mediate content, even in cases when that content is highly objectionable.

[...]

Over time, Trust & Safety and Twitter’s legal department (which now oversees Trust & Safety) have developed policies that allow Twitter to comply with country-specific free speech laws and yet still put users at the forefront. “Our mission statement within Trust & Safety is to ensure user trust, protect user rights, and craft and enforce policies to reduce legal risk,” Harvey says. “And that’s legal risk for Twitter and for users.”

Thus, when the company receives a court order or subpoena to disclose user information, it alerts the user in question, so that he or she has an opportunity to contest the disclosure before it happens. When local laws compel Twitter to delete posts or ban accounts in a given country, it does so in a narrow, transparent fashion, blocking the material locally rather than globally, and replacing the blocked material with a grayed-out alert box indicating that an act of officially mandated government censorship has taken place.

And when third parties aren’t armed with subpoenas, court orders, or other valid legal processes, Twitter typically does not take action. In September 2012, for example, Al-Shabaab, a Somali-based affiliate of Al Qaeda, tweeted photos showing the corpses of several Kenyan soldiers it took credit for killing in combat. A few weeks later, seven Republican members of Congress sent a letter to the FBI urging it to make Twitter remove Al-Shabaab’s account and those of various other “Specially Designated Global Terrorist entities.” As of early January, the five accounts the lawmakers complained about continue to function on Twitter. A letter to the FBI, even by influential types on Capitol Hill, does not carry the force of a court order. Twitter was unmoved.

[...]

But so far, Twitter’s laissez-faire attitude toward online discourse has been its greatest business proposition, so much so that Twitter’s chief legal counsel, Alex Macgillivray, told The New York Times that he views Twitter’s commitment to free speech as a “competitive advantage.”

[...]

I recommend reading the whole article.

I don't think you're going to get this, because I don't think Twitter is interested in enforcing any more of its ToS than it absolutely has to, for a gun-to-the-head definition of "has to". More than merely not interested, actively opposed as a matter of both moral principle and business plan.

[silveradept]

That would be an excellent way of preserving those things when they happen, and hopefully prevent asshattery and abuse of the abuse system.

[andustar]

Ooh. I really like this and have added it to my storify about related stuff. https://storify.com/andustar/twitter-moderation/