Scrapping the Web

Blog Post 6 by Wiana Inthapanya


The use of data scraping tools are interesting mediums to search for specific content from the internet. Scraping the web for such specific data is useful to see the social outcomes of your focused area and what thoughts are being generated in today’s media.

After my probe, I wanted to explore more about the affects of mental health on university students. From personal experiences, I use to update my social media status depending on my random mood swings of being stressed and under pressure from my years at uni. I actually wondered if I was the only one who did this as a method of relief and procrastination.

Using the advanced search tool on Twitter, I was able to find the current issues on mental health on university students and their social status updates.


  • I can honestly say that this minor exercise was actually quite an enjoyable one as it opened by eyes to realise that I was not the only over whelmed student emotionally expressing my thoughts to the world. Even though today I don’t express myself on the internet as much as I use to when I initially started uni, these status updates certainly still resonates with me today as I could understand how they feel
  • This search provides me with an understanding that factors such as work, personal life, relationships and social commitments are highly valued by individuals due to the amount of anxiety that is expressed in these concerns. These factors may contribute to the stress build up that affects the mentality of university students
  • Most of the posts are from students who are about to commence their uni semester and are already under this so much pressure on their first week. This determines the level of anxiety on people and how sensitive they are to stress

From here, I want to further investigate how many tweets there actually were regarding stress and anxiety in university students. To be able to do this, my next advance search of scrapping the web for data required me to download an archiver using Google Spreadsheets that connected to the world of Twitter. This tool allowed me to narrow down my search further with accuracy. Below were my results.



This image shows the three main hashtags I’ve utilised to search for the data. All data using the hashtags #university, #stressed and #exams were located on twitter. With such a broad search rule, 8785 tweets were found.


I then narrowed it down to those same specific hashtags, however set the location to Sydney and created another rule where only the tweets that were retweet more than 5 times would show up. This allowed me to further refine the results which interestingly dropped to just 11 data sources! It’s amazing what you can do with advance searches and data tools!


And lastly, I removed the location rule to reveal all the tweets from Australia using the same hashtags. In total, there were 221 tweets regarding university stress. Out of the 221, only 11 tweets were from Sydney alone.




%d bloggers like this: