Post 6: Scraping the web or data
By Angela Tam
The process of web scraping was quite time consuming in terms of finding the right phrases or words to search in order to have data to then analyse. I looked at Twitter, Facebook and Instagram to see what social media had to say about affordable housing.
Using a Twitter Archive connected to Google Sheets, this method of finding data allowed for a lot of data to be found and stored neatly to analyse. It took some tweaking of key words and advanced searching to find a large quantity of data related to this issue. Searching with Twitter Search Rule: “affordable housing” innovative OR living OR housing lang:en, produced 10694 results. This was overwhelmingly successful however realistically it would be quite difficult to thoroughly analyse the data. I made the search in “affordable housing” more specific with key words such as ‘innovative living’, ‘Sydney housing’ and ‘affordable’, producing a set of data from 549 Twitter posts (Twitter Search Rule: “affordable housing” innovative OR living OR affordable OR housing OR sydney #affordablehousing, OR #innovativeliving). Surprisingly, only two of these were accounts from Sydney. Either way I was still curious to see what people around the world were saying about this issue in the context of their cities, and I was interested to see what specific people said, so I decided to disregard Twitter accounts that were organisations or companies. Of these people, majority of the tweets were of news articles rather than a distinct subjective thought on the issue.
At this stage, it seemed like there wasn’t much discussion regarding this topic on Twitter – especially not in Sydney. Perhaps I wasn’t looking at the right key words for searching but I didn’t find that this method of web scraping was very useful for me. I decided to try searching more specifically for Twitter thoughts on inclusionary zoning. Out of the 101 responses, 6 (highlighted) were from NSW.
I’m unsure whether it is the form of media, or if it is because the concept of inclusionary zoning is not really talked about on social media in Sydney, but it was difficult to find relevant data to explore further. Not really knowing how to use Twitter either, I thought it would be helpful to turn to more familiar social media platforms such as Instagram and Facebook to see what was said there.
An advanced search into affordable housing through Facebook produced much more interesting results.
The range of data collected here was much more interesting for me as a visual learner. With a clear and diverse set of data ranging from articles, videos and photos, this way of web scraping was a lot more engaging. One particular example of innovative design from Sydney that is aiming to impact the city is known as Big World Homes. Founded by Alexander Symes in 2014, the lack of opportunity for young people to live in Sydney at an affordable price was frustrating. From a background in architecture, Symes designed a transitional home that re-conceptualises ‘space’ and ‘wealth’, offering a solution to those currently struggling with home-ownership (Big World Homes, n.d.) You can read more about it here.
Scrolling through the search results, it seemed majority of the content was quite positive and driven by change – conveying Sydney as a place that is ready for action and offering solutions to approach this issue. Posts that suggested this from conferences and forums including the National Housing Conference 2017 and Vinnies NSW’s Affordable Housing Forum.
Similar to the post about Big World Homes, Facebook user J. Haines also posted about another innovative solution about shipping containers being repurposed to create affordable housing. Following this, I searched “innovative housing” with the tagged location of Australia. I found a post regarding an upcoming talk “Design Affordability: Quicker, Smarter, More Efficient Housing Now” by the Faculty of Architecture, Design & Planning, University of Sydney that is on this upcoming weekend. Another noteworthy search result was the “Wikkelhouse” – A house made of cardboard built in Amsterdam.
Finally, I conducted an Instagram hashtag search, looking at #sydneyhousing and #theaustraliandream in particular. From previous group mind mapping explorations, the expectations of the ‘Australian Dream’ has gradually changed overtime. Being a generation that posts frequently online, I was curious to see what was shown on this platform.
Results varied – #sydneyhousing only had 35 posts with two that were related to the issue rather than real estate advertisting images. In contrast, #theaustraliandream consisted of 198 posts, with only six relating to housing. Other images included other elements of the aspirational lifestyle one hopes to attain in Australia. Four of the six images highlighted the struggle that is portrayed in news articles and forums/blogs.
I decided then to focus more on two particular hashtags on Instagram that had a range of imagery that I could potentially dissect emotive and poetic data from. Through choosing 12 images from the hashtag that represented it well, I picked three of the main colours in the image to form a colour scheme for that hashtag. I also extracted some emotions found through reading the captions users posted with their image and documented other hashtags found along with that image.
I think this way for collecting data is quite interesting and visually engaging – what are people trying to say through what they post on social media? Already with this initial search, the colours of #australiandream are much more bright and evoke a sense of happiness than #rentingsucks. What other hashtags could I use to find images that relate to housing affordability and what would their colour schemes be like? How can these collective communities of hashtags represented by colour tell a story?
- The issue of housing affordability is not widely discussed on social media in a personal context unlike how gender equality or mental illness may be discussed, thus suggesting that this issue is perhaps:
- not yet a huge priority in young Australians
- ignored due to lack of immediate solutions
- not really something you talk about as it relates to your financial status
- Owning a house is not widely represented on social media as part of the Australian Dream – has the priority for savings and spending changed in the younger generations?
- There are various innovative living solutions
- Housing affordability is an issue that is gradually becoming more talked about through influential organisations and community groups on a global scale
- In terms of process, I found that it took some refinement in key word searching to create interesting data sets. It is good to be specific but not too specific that you only get three posts about it. It takes some practice and a lot of patience!
Big World Homes n.d., Homepage, Big World Homes, viewed 3 September 2016, <http://bigworldhomes.com/>.
cynthias.adventures 2016, Pretty much all anyone can afford in this market, Instagram, viewed 3 September 2016, <https://www.instagram.com/p/BGeH75JN6KG/?tagged=sydneyhousing>.
jlynagh 2016, Home Sweet Home!, Instagram, viewed 3 September 2016, <https://www.instagram.com/p/BELPgjNNxJt/?tagged=theaustraliandream>.
ne0ncreature__ 2015, Instagram, viewed 3 September 2016, <https://www.instagram.com/p/4-i9TcBpyl/?tagged=sydneyhousing>.
organicallydanica 2016, A house for $500 000???, Instagram, viewed 3 September 2016, <https://www.instagram.com/p/BELv50FCUAG/?tagged=sydneyhousing>.
Wikkelhouse 2016, Wikkelhouse, Vimeo, viewed 3 September 2016, <https://vimeo.com/156852806>.