A story of Gandhi, Racism, and Curiosity.

Or: How a non-coder ends up writing a Twitter bot

Image for post
Image for post
This portrait of Ghandi is made up of 25,000 people on Twitter who claimed not to be racist… But…

Discovering Twitter

I signed up for Twitter in December 2006, and was dabbling on the platform here and there, but didn’t really figure out what it was really for until I signed up for an account for my photography blog two years later. That worked out rather better — and that account is now my main account. It’s doing pretty well, and has over 50,000 followers.

“I bet 99% of people who start a sentence with ‘I’m not racist, but…’ are actually about to say something racist”

One evening, I had a drunken evening with a friend in the pub, and behind us, we overheard a conversation. It started “I’m not racist, but…” and then proceeded to say something horrifically racist. My friend pointed out “Hey, I bet that 99% of people who start a sentence with ‘I’m not racist, but…’ are actually about to say something racist”.

“People who tweeted something racist were, on average, 2% more likely to misspell words.”

Methodology

  • I coded a little PHP script that used the Twitter Search API to find all Twitter posts that have the words “not”, “racist” and “but” in them, in that order.
  • The script ignores all retweets and @replies (or, in fact, any posts with the @ symbol in them), because I wanted to ensure that only ‘original’ tweets were counted.
  • I set up an oDesk account, and asked a freelancer to help me sort the tweets, flagging them either as racist (“I’m not racist but Obama needs to go back to the jungle” etc), anti-racist (“I hate people who say ‘I’m not racist’ and then say something racist’), or unknown (i.e. nonsensical, in another language, or a joke such as ‘I’m not racist but I love brown bread’ etc)
  • When a tweet was tagged as definitely racist, the twitter user’s user image was downloaded and stored on a server.

“Based on our data set, it turns out that around 81% of people who start a tweet with ‘Not racist but…’ are tweeting something racist.”

Image for post
Image for post
We analysed 33,690 tweets. In total, 81% of them were racist.

Turning racism into art

Well, I wanted to try and create something creative, and turn the barrage of racism into something vaguely constructive. I created a second script, which used procedural (!) PHP to analyse the content of Twitter user photo, to find the overall brightness of the photograph. This data could then be used to create a photo mosaic. Of course, there’s plenty of software out there that does this for you, but given that I wanted to learn some coding myself, I decided to give it a shot.

Image for post
Image for post
A portrait of Martin Luther King, built up of 5,000 Twitter profile images of people who claimed they weren’t racist, but then went on to prove themselves wrong
Image for post
Image for post
25,000 Twitter profile pictures making up a portrait of Mahatma Gandhi
Image for post
Image for post

Did you do any other statistical analysis?

I played around with the data I gathered quite a lot, both in SPSS and with little tools I coded myself.

Image for post
Image for post

Share this post

Like this post? Please tweet: 81% of people who start a tweet with ‘Not racist but…’ are tweeting something racist by clicking here (don’t worry, you can edit the text before you tweet). Thanks!

So, what did I learn?

  • It turns out that if you have a specific goal in mind, you can teach yourself to code.
  • People can be pretty horrible.
  • Digging into Twitter traffic is awesome: People tweet about pretty much everything, and digging into things and doing statistical analysis is fun!
  • This project inspired me to code a proof of concept for a series of tools for the Metropolitan Police, that can be used as early warning systems for unrest, accidents, etc.
  • Twitter’s APIs are pretty easy to work with, as far as APIs go.
  • This was all a lot of fun and I learned a lot.

Written by

CEO of Konf, pitch coach for startups, enthusiastic dabbler in photography.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store