Post 6 — Twitter Data Scraping

The issue of asylum seekers coming to Australia is a highly debatable topic, particularly on social media platforms. Twitter in particular allows people to express their thoughts and opinion in a quick 140 character message. Using the Actor Network Theory to understand the social systems and networks surrounding the topic of asylum seekers, Twitter can also be seen as a non-humanistic actant that connects people all across the world. Human actants can read, reply and write tweets using their laptop or mobile device.

Hashtags are used for keywords/phrases to broadcast and categorise tweets, helping them to appear in Twitter searches. Advanced searches allows you to filter your results by setting specific parameters using keywords, locations, people, places and dates. This information can be exported into table format with the combined use of Google Sheets.

Over the week, I have been collecting Twitter data to show patterns of human actants and their thoughts and behaviours. I experimented with different sets of parameters to reveal more about attitudes towards asylum seekers and how they are formed and swayed. I drew on the words collected from week 4’s class task, where we were to intuitively write down 25 words that resonate with us that are often used by the media and in daily discourse about Australia’s asylum seeker/refugee issue.

Class Task — Words heard in daily discourse regarding the topic of asylum seekers

I began by entering some of these words in a very broad search using the Google Sheets Twitter archive. However, I found that it was difficult to find any underlying patterns as there was an overload of data that was difficult to analyse (over 4,000 tweets). I realised that if I wanted to find data that is relevant to my specific area of interest, I needed to suggest a somewhat broad hypothesis so that I know what patterns/variables/trends I am looking for.

data_scrapeI hypothesised that people from the same regions may have similar attitudes. This vague assumption dictated the keywords that I entered into the Google Sheets Twitter Archive and Twitter’s Advanced Search. The Twitter archive did not allow me to specify the location of the tweets, thus I needed to manually filter through locations within Australia. I was also able to obtain qualitative data from the user descriptions, providing an insight of the underlying reasons or factors that may influence these results. I repeated this process using different keywords and compared and analysed the results. I obviously could not go through all of the results, thus I selected a random sample group of 175 people to analyse which can be seen here [].


Unfiltered results (Tweets from international locations)
Filtered results (Tweets from locations in Australia)
tweet locations
Number of tweets from different locations in Australia (these results were collected from a random sample group of 175 tweets)

From this data scraping research, my hypothesis was not entirely validated as the data showed that the vast majority of people on Twitter have similar attitudes, irregardless of where they are from. Most people tweeting about the issue of asylum seekers in Australia condone the offshore processing policies and detention centres. However, these results only reflect a tiny percentage people on twitter and an even smaller percentage of the Australia public.

In conjunction with tweet locations, I also needed to identify the attitudes/opinions presented in the tweets. I considered how one may frame their statements, focusing on how the word ‘illegal’ is used. By simply changing the linking verb ‘are’ and ‘is’, I was able to collect tweets with very different stances on the issue.

are illegal_01are illegal_02is illegal_02is illegal_01

I tried to understand the background of this sample group, as that would give an insight to why they have these attitudes. From observing the bio descriptions, most people who have tweeted on the subject of asylum seekers are fairly well educated and/or are advocates for social justice.


Summary Points:

  • People from CBD’s (particularly Melbourne) are a lot more vocal about asylum seeker issues on social media than people from regional areas. In saying this, one must consider that capital cities have higher populations than regional areas).
  • Most people who have tweeted on the subject of asylum seekers are fairly well educated and/or are advocates for social justice.
  • Tweets tend to focus on one specific aspect of the issue (though they are restricted to 140 characters).
  • Most twitter users do not have the ability to mobilise change. Rather, they act as intermediaries that retweet provocative posts made by influential mediators.
  • Twitter data scraping is better suited for numeric data, rather than measuring trends in opinions. I found it to be tedious as I had to manually find and organise tweets that came from locations in Australia.

Potential Concept:
A Twitterbot would ideally distinguish peoples attitudes based on how the post has been worded (i.e. ‘is illegal’ and ‘are illegal’) and match the tweets with opposite attitudes. I found that many twitter posts tended to focus on only one aspect of the issue whilst ignoring equally important concerns. This concept could lead to either a mass of heated arguments and hurling of insults, or possibly provide people with a bit of perspective and alternative insights. I would hope the latter would be the predominant outcome and hopefully force people to see the issue in a broader context.



%d bloggers like this: