Post 6: Scraping the web for data

To understand the growing issue of housing affordability and where we as designers can make an impact – and meaningful changes that stick – we need to look at both hard data relevant to the problem of housing affordability and social media data that reflects public perception of the problem. I feel an effective design response would be one that changes prevailing perception of the problem through incorporating or presenting the analytical data in an engaging way – a call to action.

As part of the Queensland Government’s commitment to open government, the Department of Housing and Public Works releases datasets to the public. The department’s open datasets include: National rental affordability waitlists, Social housing registers, RentConnect data, Allocations to community housing, and many more.


I decided to focus in on the Allocations to community housing, a dataset delivered by registered community housing providers during the 2014-2015 financial year.


This data, while still raw, demonstrates a few key insights. It lists level of housing needs and location data. It also shares specific household information which I found surprisingly moving. Our typical response to datasets is an unempathetic one, yet this data when summarised is able to put a ‘face’ to who community housing is important for. Below are some totals from the dataset.

Of the 1034 households listed:

482 had at least one homelessness flag.

432 had no data listed.

120 had no homelessness flags.


351 had a disability.

433 had no data listed.

250 had no diability.


175 were indigenous households.

434 had no data listed.

425 were non indigenous households.


I then decided to look at Twitter data – collating tweets that used either #housingaffordability or #publichousing, between August 20th and September 5th, 2016. I made the decision to not limit the search by location so I could then explore the global location data. My hypotheses being that Australians would be talking more about this issue than other nations. By grouping the total number of tweets from Australia I found this to be true.


80 of the 323 tweets came from locations in Australia, with Sydney alone being responsible for 32. In contrast, New York, a famously overcrowded city, had just 12 tweets in the dataset.


– Even analytical data can evoke empathy. Data on the sheer number of disabled and once homeless people that community housing helps is moving.

– When it comes to tweets about affordable housing, Australians are vastly overrepresented. Is the state of our housing market worse than other nations or are we just more vocal?

– Most tweets link to articles, indicating that housing affordability is not a hot-button-issue that can be summed up in 140 characters – but a complicated issue that requires deep reading. How then can we make housing affordability engaging and digestible to a young audience?