Blog Post 6: Scraping the Web for Data

In looking through various platforms of social network, I wanted to take a closer look at the general public awareness and the trending perspectives regarding the current situation of refugees in the Australian political context. I chose to look at the opinions expressed on Twitter, considering its reputation as a social network platform encouraging people to share their thoughts and opinions, connecting people from all over the world to communicate with each other. With its limited text capacity prompting a much more direct and blunt opinions to be expressed, Twitter also enables people to see what issues are trending through hashtags, and using filters the users can also narrow down their findings to results of desired relevance.

In beginning the data scrape from twitter, to get a broad overview I searched for the term ‘refugees,’ and ‘asylum seekers,’ specifically. The results were overwhelming – over 3000 tweets were collected, all discussing the global refugee crisis from Syria, Europe to Australia. I wanted to know more about what people thought about the refugee crisis in Australia, in particular within the current political context. So to narrow it down, I included the term ‘queue jumper’ / ‘queue jumping’ and ‘Australia,’ as part of the searched terminology.

The results were disappointing – only a single tweet was retrieved, favourited by three people, commenting on the detrimental effects of using the terms ‘illegal’ and ‘queue jumpers’ to refugees. So I searched again, this time without geographic limitation, searching for terms ‘refugees’, ‘queue’ and #auspol. Luckily this time I had around 2000 tweets to look through, but unlucky at the same time because a lot of the posts were retweets of the same articles. A good majority of the individuals behind the tweets were from Australia, understandably due to the #auspol context, and it was interesting to see how their responses developed consistently as updates to the conditions in Nauru and Manus Island were reported.

General Scrape.jpg

In looking through these tweets, I was able to gather a good amount of idea of what the general public opinion was in regards to the refugee issue in Australia, and learned more about the politics in Australia that I didn’t know about from earlier.

Tweet 1.jpgOne particular tweet I noted highlighted Australia’s recent political manoevure that had been rather quietly implemented, reported by only The Guardian, a media news source leaning left that is not owned by the notorious media mogul Murdoch. While a lot of other tweets criticised Australia’s unchanging reluctance and targeted the figureheads like Dutton and Turnbull, a few accounts like the example above focused on monitoring not just the political circumstances but the slight but significant changes made in the environment. I noticed that the author Jane McAdam was also the author of one of the two scholarly articles I studied earlier in the earlier blog posts. Jane McAdam is a Scientia Professor of Law and Director of the Andrew & Renata Kaldor Centre for International Refugee Law at UNSW. It was interesting to note how McAdam’s voice on twitter was onw of the most retweeted, considering her position and credibility as a reliable source of information related to the refugee in crisis in Australia.

In further looking through the gathered tweets, a considerable number of right winged opinions were found also, name calling the activists as “Australian Loony-Left”, a lot of the users who retweeted this lived in either Australia or the US. Although a majority of the users in this scrape were Australian, there were still a considerable number of people from different countries around the world actively involving themselves in the Australian debate of refugees.

tweet 2.jpg

Five Summary Points:

  1. A lot of the searched tweets tend to be retweets of articles and criticisim against Australia and their key political leaders.
  2. A lot of the tweets were generated within Australia, probably due to the hashtag AUSPOL, but in the first general scrape covering Australia and refugees as key terms, the scrape showed a number of users were from overseas. Political engagement in a global sense is being encouraged through twitter.
  3. Twitter is useful to keep up to date with what political aspect the users are currently engaged with, and raising awareness to recent / significant developments – but only to that extent.
  4. Twitter users try to spread their own tweets by adding/editing more relevant hashtags on to the posts – in the scrape, the most prominent tags were #asylum seekers #refugees, #Nauru, #Manus, #auspol, #ClosetheCamps and more – hashtags on twitter in regards to refugee matters seems to be constantly changing and growing.
  5. Opinions and tweets from people with credible backgrounds/people with extreme opinions tended to be the most retweeted – politically vindictive and name calling key figures in particular trended a lot, as to further promote awareness amongst people. i.e. “RT @LegacyOfJack: John Howard: small-minded lying bigot, legacies include structural deficit, pointless war and punishing refugees for political gains.”

Following the end of blog post 6, I want to direct more of my later research in not just finding out how to raise awareness in the political landscape among the young Australian audiences, but also to learn more about the resettling process of refugees, and find out more about their experiences in the Australian refugee processing. In terms of possible forms of visualisation, charting/mapping the emotional responses from both side of the argument could work to produce an interesting result.


McAdam, J. 2016,Australia is the only country to have deleted references to the Refugee Convention from its law ‘, Twitter post, 5 September, viewed 7th of September, <;

Twitter user “Real Palestinian” (shim_rational) 2016,’The Australian Loony-Left wants to flood Australia with refugees, just like Europe did via ‘, Twitter post, 9 September, viewed 9th of September, <;