Post 6 – Data Scraping

Rekha Dhanaram
The subject has seen me engage with numerous tasks that have helped me gauge an understanding on my chosen topic of Asylum Seeker and Refugees. The most recent task saw me go deeper and engage with online research through a unique means being data scraping. Data scraping allowed me to see the bigger picture but also the individual context of how people are reacting and engaging with this issue. With social media being a primary form of communication, data scraping is useful in understanding the different perspectives in regards to this issue.
I chose the platform of twitter to conduct my data-scraping tasks. Twitter is a social network, where people post or send a ‘tweet’, a short message of maximum 140 characters. With instant feedback either as a twitter follower or the tweeter, this service essentially allows people to reach countless others instantaneously. Furthermore the dialogue generated evidences it as a medium where people voice their opinions on varied issues. With this in mind twitter provided an appropriate platform to examine the public opinions around Asylum Seekers and Refugees.
Defining my research proved to be the most difficult part. Reflecting on the past few weeks, I’ve looked at this issue through a broad lens tapping into areas of media transparency, politics, processing environment and activism. Whilst its hard to narrow my focus, I decided to look into the legal context and activism through refugee experiences for this specific task.
Initially I did a very broad search on terms such as ‘refugees’, ‘asylum seekers’ and ‘Australia’. However the results were varied and whilst I was trying to get insight on the Australian society, I found that the tweets were from all over the world in response to the Australian situation. Thus I decided to go back and focus on an area I wanted to know more about. This led me to the hashtag #BorderForceAct.
The Border Force Act sees that anyone who ‘gains “protected information” during their employment service for the Border Force is barred from revealing this information without authorisation. The penalty for doing so is two years in prison.’ Whilst there have been no penalties charged under the BorderForceAct till date, it is still under constant criticism. I wanted to understand the stakeholders views in regards to this disputed issue which is highly relevant to the treatment of refugees and asylum seekers.
Looking at the tweets and images that were accompanied with the hashtag revealed the various stakeholders in regards to this issue including campaigners, the general public and event works at asylum seeker centres. What was interesting to note in some of these posts was that they spoke of other issues that arise front the Border Force Act as seen through the use of the hashtags #Bringthemhere and more contrastingly #WhitePriviledge. These tweets and uses of hashtags are a form of activism or campaigning. And whilst they stand alone as individual campaigns, it was interesting to see how they intersect. More often than not the hashtags are used together to support multiple campaigns centred around the one issue of refugees and asylum seekers. Overall, these tweets were overwhelmingly taking a negative stance in regards to the issue.
Asylum Seeker Resource Centre CEO and founder Kon Karapanagiotidis and other tweets which raise the issue of White Privilege alongside the Border Force Act.
Screen Shot 2016-09-05 at 11.23.12 pm
The #BringThemHere hashtag was equally common.
Screen Shot 2016-09-05 at 11.23.38 pm
The tweets gave insight into the various stakeholders including workers, the public, doctors and nurses. 
Screen Shot 2016-09-05 at 11.32.48 pmScreen Shot 2016-09-05 at 11.32.58 pm
Whilst most of these were from an Australian perspective, I came across a news like twitter page called ‘Public Concern at work’ which constantly tweeted quotes and statistics from articles that related to this issue. Whilst it is based in UK, it was interesting to note that it frequently tweeted posts using this hashtag.
Screen Shot 2016-09-05 at 11.23.49 pm
Furthermore when exploring the images, I found that there was a pattern of photographs of different protests peppered with political and satirical cartoons. I found it interesting that cartoons, which are traditionally in newspapers, are really popular in the ‘twittersphere’.
Screen Shot 2016-09-05 at 11.37.14 pmScreen Shot 2016-09-05 at 11.38.37 pm

 

Whilst the twitter search revealed interesting insights, I do believe that it would be better to add more parameters to gauge. I did however try to define the tone of posts, but unfortunately the results weren’t very accurate and in defining whether the tweet was opposing or supporting the Border Force Act.

Summary:

  1. Hashtags in the twitter sphere can act as a form of activism.
  2. Whilst certain hashtags support specific campaigns, many users combine them with other Hashtags centred around the bigger issue. It highlights the interconnectedness of campaigns and the overall stance taken towards that issue.
  3. #BorderForceAct is quite ironic in nature as people are voicing their opinions on a law that prohibits  many from speaking.
  4. The Border Force Act received commentary from an international platform.

 

 

 

Advertisement

Dehumanising the homeless through language

Post 6 by Alice Stollery

Web Scraping Tools

Twitter has become both a social and professional platform allowing dissemination of anything from random thoughts, ignorance and pointless memes to breaking news and public opinion. With 313 million active global users recording and sharing their thoughts, feelings and experiences, twitter is a real-time source of information and has become a medium in which people can keep up with those they know and those they don’t. The 140-character limitation on posts makes it a perfect tool for researching public opinion and scraping the platform for data, without being overwhelmed.


Web Scraping Process

Twitter archiver was my main source when scraping the web, as it allows you to create a search rule and, unlike other tools such as twitter advanced search or facebook, it collates the results into a single excel spreadsheet. This feature became invaluable in gaining insights from the results as it allowed me to sift through the information using specific terms, further refining my results. Insights often came as a result of tangents, originating in previous search results.

FullSizeRender 40
A simple visualisation of my process of navigating web scraping and twitter archiver. I found I went off on tangents from previous results.

Not being a twitter user myself, I stumbled at the start. I created a twitter account and downloaded twitter archiver and linked it to my gmail account. Initially, I ran searches that were very specific, in an attempt to find information on stereotyping and the technological divide within homelessness. I set the location to Sydney, however, this failed as I did not receive a single result. So, I changed the location and widened the search to Australia. Again, I was being too specific with the number of words I was using and twitter archiver came back with nothing.

Screen Shot 2016-09-05 at 23.26.30
A simple visualisation of my process of navigating web scraping and twitter archiver. I found I went off on tangents from previous results.

Findings

I then decided to change my approach and create a simple search of ‘homeless’ and ‘stereotyping’ which produced 14, 931 results. I then used the search tool to search for key words from the group issue mapping class within the data set. This helped me to break down the vast amount of information and to see what people were saying about issues within homelessness. This became a very interesting process. The initial words I searched within the results were quite basic as I was still getting use to using the software. However, they still provided insights and I have listed the most frequent words below.

Screen Shot 2016-09-06 at 00.08.42
A simple visualisation of my process of navigating web scraping and twitter archiver. I found I went off on tangents from previous results.

Interestingly, homeless men were referred to 4,279 times compared to only 251 times for homeless women. This is in line with the statistics revealed through one of my earlier posts that stated 82% of Sydney’s homeless are male and only 17% are female. As these posts were not Sydney specific, it is interesting to see that these statistics may also be a good indication of the situation in other parts of the world. I am curious as to why the numbers of homelessness between sexes differs so significantly. And can begin to understand why previous sources outlined a lack of services tailored to homeless women. I can only assume that this is due to less demand for them..? I could look further into the specific causes of male homelessness, contrast it to female homelessness and see if there is an opportunity to intervene with my design response.

Another observation was that the word ‘help’ only appeared in 1,137 posts out of 15,000. Looking further into these posts I found, ‘help’ took different forms. Some were genuine posts about helping the homeless with information of individuals lending a hand or community projects. It was really interesting to see how others were approaching and tackling the issue on a personal level.

genuine help
A simple visualisation of my process of navigating web scraping and twitter archiver. I found I went off on tangents from previous results.

Other twitter users made genuine offers help. In his case twitter was used as a form of communication to reach people in a particular area to help assist in helping those in need. This highlighted the possibility of technological responses that act as a connection between those with something to offer and those in need. Service design could bridge the gap between say, small businesses with food or accomodation available and the homeless.

Screen Shot 2016-09-06 at 00.28.26Screen Shot 2016-09-06 at 00.28.35

While each of the above insights were interesting, perhaps the most interesting insight was not only the presence of stigma in these posts but stigma as a result of the frequent misuse of the word homeless. While there were 15,000 posts that included the word homeless, I found that more often than not people were using it as part of casual conversation, to describe their lack of dress sense or effort invested in the appearance of friends or celebrities.

Screen Shot 2016-09-05 at 23.46.30
A simple visualisation of my process of navigating web scraping and twitter archiver. I found I went off on tangents from previous results.

Screen Shot 2016-09-06 at 20.57.23

It seems that, no matter how hard I try to broaden my understanding of the issue, I always seem to arrive back at stigma, perceptions of the homeless and ignorance towards their circumstances.After seeing these tweets, I decided to look into how the misuse of language is hindering our ability to tackle homelessness. As a result, I then ran a search on the term hobo to understand how often it was being used within the twitter sphere. This search found 10,732 results with the term hobo included in it.

“RT @emmaabel_: I am going to try and make myself look decent tmmr and not like a hobo”

“@Lizbeth_923: First Date And I Look Like A Hobo 🙂”

I believe the ubiquitous misuse of language surrounding homelessness is dehumanising the homeless, and ultimately, taking away from the issue. This is not only present in social conversations and personal online interactions, but is also reinforced by the fashion industry as seen below with the release of the ‘hobo’ bag.

Screen Shot 2016-09-06 at 00.59.19Shockingly, the term hobo is also being used as a design response to the issue, evident in the iHobo app. The virtual pet app that puts a homeless person in your pocket for you to feed and take care of. Forget to feed your hobo and he dies or runs off to get drugs. I think using gaming to tackle peoples perceptions is an interesting idea but I can’t help but feel this approach is distasteful, to say the least, and is reinforcing the publics stereotypical and often negative perceptions on the issue, not to mention dehumanising those in need.

Screen Shot 2016-09-06 at 18.50.39Screen Shot 2016-09-06 at 18.51.11Screen Shot 2016-09-06 at 18.52.20

To further investigate this area, I could also try searching the term ‘tramp’ and other common names used to describe the homeless. The above tweets highlight a severe lack of empathy among the general population for those suffering from homelessness. Homelessness does not seem to be a topic that people are talking passionately about. The homeless have fallen by the wayside and we are all so desensitised to the issue that homelessness has become commonplace in daily language for all the wrong reasons.


Design Response

Design responses could enable change in this area, and I would like to focus on the role language plays in the issue. I think there are already a number of individuals and NGO’s working to directly help the homeless so I would like to instead create a design response that tackles the wider issue and aims to influence the views people have of the homeless. Tackling this wider problem of perception, assumptions and language would aim to influence the common vernacular  rather than direct action on a smaller scale. This would hopefully result in a knock on effect, creating empathy and engagement among the wider population, to ultimately generate positive outcomes on a wider scale.

In terms of it’s form, I could create a twitter bot that calls people out on their misuse of particular words. However I think this would breed hostility rather than empathy. Language will be an important element and in order to generate a feeling of empathy I think the design would be suited to a poetic response that encompasses feelings and a the contradiction of meanings.

In an attempt to reveal relationships between language and the the number of homeless people, I could visualise the frequency of misuse of vital words. Perhaps I could plot the locations of these misuses and correlate this data with the number of homeless people in that particular area to see the relationships between the two and to discover how local attitudes affect the issue. However I am hesitant to do that as I do not think data will generate an empathetic response in the way that I am hoping.


Five Point Summary

  1. Twitter and twitter archiver are both very effective tools in scraping the web for data to understand how the wider population are feeling towards homelessness.
  2. It is important to remain open to outcomes outside your initial understanding. I went into this process with a focus on stigma and the technological divide, yet ended up delving further into the role language plays in creating barriers to a solution.
  3. Those offering food, services or accommodation on a personal level have great difficulty in finding the right people to help. Perhaps a service could be designed to bridge this disconnect.
  4. More often that not, conversation around homelessness is not referring to the issue at all and is used more so in casual conversation to describe appearances. The misuse of homeless terminology is rife among the online community and has seeped into the common vernacular ultimately resulting in a lack of empathy towards sufferers.
  5. Homelessness does not seem to be a topic that people are talking passionately about. The homeless have fallen by the wayside and we are all so desensitised to the issue that homelessness has become commonplace in daily language for all the wrong reasons.

References

AlaynaaaMarie. 2016, ‘I just added this to my closet on Poshmark: Michael Kors Julian Chain Exotic Hobo’, Twitter post, 2 September, viewed 2 September 2016,https://twitter.com/AlaynaaaMarie/status/771826144198873088>.

Alex. 2016, ‘I look like a hobo walking to class but idgaf’, Twitter post, 1 September, viewed 2 September 2016,< https://twitter.com/alexj0ness/status/771405203354550272>.

Bryant, M. 2010, iHobo app puts a homeless man in your pocket, The Next Web, viewed 31 August 2016,<http://thenextweb.com/apps/2010/05/10/ihobo-app-puts-a-homeless-man-in-your-pocket/#gref&gt;.

Dan Kenny, 2016. ‘Can’t think of any but I’ll happily help give it out in the streets after service, be done 11:30ish’, Twitter post, 2 September, 2 September 2016,< https://twitter.com/DanKenny29/status/771812145558347779>.

Dream Closet. 2016, ‘Always torn between getting ready and looking cute or being lazy and looking like a homeless person’, Twitter post, 2 September, viewed 2 September 2016,< https://twitter.com/TheDreamCIoset/status/771684469287845892>.

Emmabel. 2016, ‘I am going to try and make myself look decent tmmr and not like a hobo’, Twitter post, 1 September, viewed 2 September 2016,<https://twitter.com/emmaabel_/status/771505054201241600>.

EWapo. 2014, ‘I’m playing a game called iHobo where you look after a tramp and I’m legit checking up on him every 5 minutes, I’m here for you trampy’, Twitter post, 10 January, viewed 3 September 2016,<https://twitter.com/EWapo/status/421761288617091072>.

FryGuy84. 2015, ‘who remembers that app iHobo when you had to look after a tramp like a tamagotchi’, Twitter post, 29 January, viewed 3 September 2016,< https://twitter.com/fryguy84/status/560837097612140545>.

Hungry Hoss, 2016. ‘At a wedding in Brighton & there’s a shit load of food left over… does anyone know a homeless shelter or similar who’d make use?’, Twitter post, 2 September, viewed 2 September 2016,<https://twitter.com/HungryHoss/status/771804636298809345>.

Lizbeth_936. 2016, ‘First date and I look like a hobo’, Twitter post, 1 September, viewed 2 September 2016,< https://twitter.com/Lizbeth_936/status/771517651130601472>.

NinjaRonnie. 2013, ‘Shit I forgot about my hobo’, Twitter post, 9 June, viewed 2 September 2016,< https://twitter.com/NinjaRonnie/status/343751075221872640>.

Pissedoff99. 2016, ‘You ignorant description kind of shows how insensitive people involved with fashion industry can be with social issues’, Twitter post, 3 September, viewed 3 September 2016,<https://twitter.com/PissedOff99/status/771977859829202944>.

Prank Videos, 2016. ‘He gave his ‘winning’ lottery ticket to a homeless man, gave the store owner the money to pretend its real, amazing’, Twitter post, 11 March, viewed 3 September 2016,< https://twitter.com/30SecPranks/status/708500013349900288>.

Smith, C. 2016, Twitter Statistics and Facts, DMR Stats and Gadgets, viewed 2 September 2016,<http://expandedramblings.com/index.php/march-2013-by-the-numbers-a-few-amazing-twitter-stats/>.

Type 1 Diabetes: A Twitter Data Scrape

 

Post 6 by Lucy Allen

Twitter has been defined as a sort of ‘microblogging’ in allowing Its users to post short, 140 character posts on the ‘twitter-sphere’. The limited word count often means that witty and somewhat blunt opinions and statements are made and shared with followers and available to a worldwide community.

The appeal of Twitter is not only the freedom of opinion and expression but the ability to read content quickly and engage with users world-wide. An array of accounts manned by different companies, individuals and businesses means that live events are responded to and can be tracked in real time.

Screen Shot 2016-09-05 at 2.10.23 PM
(Buzzfeed, 2014)

Due to the array of users and audience members there are many different reasons and opinions expressed in ‘tweets’. The different identities and values of the different users are mapped out below:

new doc 6_1
Mapping the values and identity of different Twitter users (Lucy Allen, 2016)

The tone of voice used in a Tweet very much comes down to the user themselves and The type of user they are. For example, a business company’s tweet about it’s recent successes would most likely have a much more formal and matter-of-fact tone of voice compared to that of a young adult on the topic of a music festival. In saying this companies and even governments/politicians sometimes utilise a more informal and humorous tone of voice over social media platforms to attract a younger and more varied audience.

graham-tweet-Syria
(Lindsey Graham, 2013)

 

 

With over 313 monthly million users and 500 millions tweets per-day, Twitter is a treasure trove of information and opinion. The development of data scraping tools such as the ‘Advanced Search’ Twitter function allows anyone to discover the data and information across the enormous platform. This allows people to examine and explore this data, pulling out interesting findings and insights.  I undertook my own data scraping task to investigate the presence of discussions about Type 1 Diabetes, excited see what I could deduce from my process and findings.

Process 

My process for data scraping utilised the Twitter Advanced search function as an add-on of Google Sheets. Whilst I began with this special function I ended up using the Advanced Search on the Twitter to enable more thorough data scrapes.

new doc 7_1.jpg
Process map (Lucy Allen, 2016)

Results

My first search was very broad and included all the related words I had brainstorm in regards to Type 1 Diabetes. This broad search resulted in 170 pages of data and over 10, 000 results. From a brief glance at the results I did immediately recognised that there was a real mix of personal posts from people living with the disease, mixed in with news reports and research findings. I realised however that I needed to refine my data scrape if I was going to be able to properly analyse the data.

Screen Shot 2016-09-03 at 1.45.43 PM
Twitter Advanced Search 1

My next search involved searching fewer words but still keeping the terms broad by searching ‘Type One Diabetes’, ‘Type 1 Diabetes’ as well as those terms as hashtags. I got 450 results back, a much more realistic set of data however still to big to analyse and draw insight from.

Screen Shot 2016-09-03 at 1.57.42 PM
Twitter Advanced Search 2

Using the same search terms I wanted to narrow the date field of the data, limiting it to the past 30 days. I discovered that the Twitter Advanced Search add-on for Google sheets didn’t enable a date-based advances search. Instead I ran the advanced search on the Twitter website which did allow me to refine my search by date and time frame. The results weren’t as neatly collated as that of using the search in conjunction with Google Sheets however it did enable me to see entire posts, even those that included images and videos. I was able to spend time looking into the search results and made some insightful discoveries.

new doc 4_1
Search 3:  Tallying the types of Twitter posts on Type 1 Diabetes

I went through the collection of posts and started to tally what type of posts they were. I wasn’t surprised to see that most posts on the topic of Type 1 Diabetes were personal ones that saw people expressing day-to-day struggles of living with the disease. There were certainly fewer news and research based posts than I was expecting and far too many posts about Nick Jonas, a celebrity living with Type 1 Diabetes (I had to give him his own category…). There were of course a few ‘really?!’ tweets where people obviously had no idea about the disease as seen in the example below.

Screen Shot 2016-09-03 at 3.02.01 PM
Is Type One Diabetes Real… (Twitter, 2016)

 

Probably the most incredible discovery I made throughout my data scrape was the discovery of so many support networks and platforms for people living with Type 1 Diabetes to connect with. I was completely blown away and can’t wait to look into these further and discover the digital community that exists.

After I’d completed this area of investigation I was interested in looking at the results I got when searching Type 1 Diabetes imagery on Twitter, again with the Advanced search tool. After creating my own image archive last week I was really intrigued to see what types of images would come up. I found that most images were in regards to personal achievements of those living with Type 1 Diabetes or fundraising or campaigning. I noticed again and as discussed in my recent blog post ‘Type 1 Diabetes: Stakeholders and Visual Representation’ that there was a lack of images that can completely sum up what it is like living with Type 1 Diabetes.

Key findings

  1.  People are using social media to speak out about what It’s like living with Type 1 Diabetes
  2.  Social media platforms such as Twitter are acting as a digital support network for people living with diseases as well as other issues and concerns
  3.  There are still lots of people who are completely unaware of Type 1 Diabetes or lack any proper understanding of the disease
  4.  Most ‘tweets’ about Type 1 Diabetes came from Westernised countires
  5.  People living with Type 1 Diabetes are becoming increasingly frustrated with a lack of understanding and stereotypes present in society

Reflection

In reflection I believe that there may have been more creative potential if I searched some less obvious terms such as ‘insulin pump’ or ‘Autoimmune disease’ etc as part of my data scrape. I also would have liked to have spent more time analysing and looking into the image archives of Twitter however this came as an afterthought to my initial process. I noticed when going back over my search results you can also generate positive or negative responses. It’d be interesting to gather and compare this data, looking at the different tone-of-voice these opposing posts take.

Screen Shot 2016-09-05 at 9.53.43 PM.png

I would also love to have the skill and time to create a Twitter Bot as Chris Gaul talked about in his lecture. It would be fascinating to interact with people posting on this topic and even pose questions to them or provide them with knowledge on Type 1 Diabetes. These bots have so much potential in engaging with the issue through a tweet and therefor engaging with users and people from all walks of life.

Like with all data there is so much potential in how it can be visualised and broken down. A fantastic pre-existing example of this is the #NoPricks campaign image which visualises the average amounts of injections a person living with Type 1 Diabetes has in a month.

20140721132615-1yr_nopricks.png
#NOPRICKS Campaign Poster (Nopricks, 2016)

With the data and content I’ve collected It would be really great to visualise the breakdown of the types of tweets people post about Type 1 Diabetes as I think it says a lot about social media and sharing and support platform. It’d also be interesting to delve into this further by breaking down the types of personal posts and analysing the reasoning behind why someone may want to post this and the response/support each post gets. This then got me thinking about the possibility of mapping the online support network of sufferers living with Type 1 Diabetes to see the great extent of this network. An example of what this type of data visualisation could look like is seen below:

new doc 8_1.jpg
(Lucy Allen, 2016)

References

Gaul. C, 2016, ‘Twitter Bots’, Lecture, accessed 1st of September 2016, <https://online.uts.edu.au/bbcswebdav/pid-1384637-dt-content-rid-8098732_1/courses/87831/Week4Lecture_ChrisGaul_TwitterBots.pdf&gt;

Gil, P. 2016, ‘What Exactly is Twitter’, About Tech, accessed 1st of September 2016, <http://netforbeginners.about.com/od/internet101/f/What-Exactly-Is-Twitter.htm&gt;

Heugel. A, 2016, ‘Lindsey Graham Tweet’, 30 Hilarious Political Tweets, accessed 1st of September 2016, <http://twentytwowords.com/hilarious-political-tweets-that-will-make-the-internet-great-again/>

Muhammad M. U., Muhammad. I , Hassan. S, ‘Understanding Types of Users on Twitter’, Lahore University of Management Sciences, Lahore, Pakistan1 Qatar Computing Research Institute, accessed 1st of September 2016, <hsajjad@qf.org.qahttp://www.qcri.org.qa/app/media/4858>

NoPricks, 2016, ‘Campaign Image’, accessed 2nd of September 2016, <http://www.prweb.com/releases/2014/08/prweb12069199.htm&gt;

Sayce, D. 2016, ’10 Billion Tweets‘, David Sayce, viewed September 1st 2016, <http://www.dsayce.com/social-media/10-billions-tweets/&gt;

Twitter Advanced Search, Twitter, accessed 3rd of August, <https://twitter.com/search-advanced?lang=en&gt;

The Huffington Post, 2016, ‘Best Tweets’, accessed 4th of September 2016, <http://www.huffingtonpost.com/news/best-tweets/>

Tholepin. C, 2016, ‘Is Type One Diabetes Real’, Twitter, accessed 2nd of September 2016

 

Post 6 — Twitter Data Scraping

The issue of asylum seekers coming to Australia is a highly debatable topic, particularly on social media platforms. Twitter in particular allows people to express their thoughts and opinion in a quick 140 character message. Using the Actor Network Theory to understand the social systems and networks surrounding the topic of asylum seekers, Twitter can also be seen as a non-humanistic actant that connects people all across the world. Human actants can read, reply and write tweets using their laptop or mobile device.

Hashtags are used for keywords/phrases to broadcast and categorise tweets, helping them to appear in Twitter searches. Advanced searches allows you to filter your results by setting specific parameters using keywords, locations, people, places and dates. This information can be exported into table format with the combined use of Google Sheets.

Over the week, I have been collecting Twitter data to show patterns of human actants and their thoughts and behaviours. I experimented with different sets of parameters to reveal more about attitudes towards asylum seekers and how they are formed and swayed. I drew on the words collected from week 4’s class task, where we were to intuitively write down 25 words that resonate with us that are often used by the media and in daily discourse about Australia’s asylum seeker/refugee issue.

words_all
Class Task — Words heard in daily discourse regarding the topic of asylum seekers

I began by entering some of these words in a very broad search using the Google Sheets Twitter archive. However, I found that it was difficult to find any underlying patterns as there was an overload of data that was difficult to analyse (over 4,000 tweets). I realised that if I wanted to find data that is relevant to my specific area of interest, I needed to suggest a somewhat broad hypothesis so that I know what patterns/variables/trends I am looking for.

data_scrapeI hypothesised that people from the same regions may have similar attitudes. This vague assumption dictated the keywords that I entered into the Google Sheets Twitter Archive and Twitter’s Advanced Search. The Twitter archive did not allow me to specify the location of the tweets, thus I needed to manually filter through locations within Australia. I was also able to obtain qualitative data from the user descriptions, providing an insight of the underlying reasons or factors that may influence these results. I repeated this process using different keywords and compared and analysed the results. I obviously could not go through all of the results, thus I selected a random sample group of 175 people to analyse which can be seen here [https://s9.postimg.io/5gci9fw65/table_all.jpg].

 

international
Unfiltered results (Tweets from international locations)
national
Filtered results (Tweets from locations in Australia)
tweet locations
Number of tweets from different locations in Australia (these results were collected from a random sample group of 175 tweets)

From this data scraping research, my hypothesis was not entirely validated as the data showed that the vast majority of people on Twitter have similar attitudes, irregardless of where they are from. Most people tweeting about the issue of asylum seekers in Australia condone the offshore processing policies and detention centres. However, these results only reflect a tiny percentage people on twitter and an even smaller percentage of the Australia public.

In conjunction with tweet locations, I also needed to identify the attitudes/opinions presented in the tweets. I considered how one may frame their statements, focusing on how the word ‘illegal’ is used. By simply changing the linking verb ‘are’ and ‘is’, I was able to collect tweets with very different stances on the issue.

are illegal_01are illegal_02is illegal_02is illegal_01

I tried to understand the background of this sample group, as that would give an insight to why they have these attitudes. From observing the bio descriptions, most people who have tweeted on the subject of asylum seekers are fairly well educated and/or are advocates for social justice.

bio

Summary Points:

  • People from CBD’s (particularly Melbourne) are a lot more vocal about asylum seeker issues on social media than people from regional areas. In saying this, one must consider that capital cities have higher populations than regional areas).
  • Most people who have tweeted on the subject of asylum seekers are fairly well educated and/or are advocates for social justice.
  • Tweets tend to focus on one specific aspect of the issue (though they are restricted to 140 characters).
  • Most twitter users do not have the ability to mobilise change. Rather, they act as intermediaries that retweet provocative posts made by influential mediators.
  • Twitter data scraping is better suited for numeric data, rather than measuring trends in opinions. I found it to be tedious as I had to manually find and organise tweets that came from locations in Australia.

Potential Concept:
A Twitterbot would ideally distinguish peoples attitudes based on how the post has been worded (i.e. ‘is illegal’ and ‘are illegal’) and match the tweets with opposite attitudes. I found that many twitter posts tended to focus on only one aspect of the issue whilst ignoring equally important concerns. This concept could lead to either a mass of heated arguments and hurling of insults, or possibly provide people with a bit of perspective and alternative insights. I would hope the latter would be the predominant outcome and hopefully force people to see the issue in a broader context.

 

 

{post 6} the scraping of data.

data scraping. analysis. judith tan.

background texture
(MFT 2013) ‘Data scraping’ is a new term for me. Before, this, I was not aware that the word ‘scrape’ can mean more than just the physical, tangible meaning, e.g. to scrape paint off a surface.

The next phase of research was to scrape data from the web. I chose Facebook as my research platform, as I wanted to gather data regarding the effects of different methods of communication and media, including text, image and links to external sites. I gathered data with a focus of seeking to identify how the public views the issue of homelessness. I also wanted to identify what methods are more effective in engaging an audience.

Continue reading “{post 6} the scraping of data.”