POST 6: Scraping the web for data

Twitter was foundered by Jack Dorsey in 2006 and has become social networking sensation. The media source allows individuals to send and receive short messages that are referred to as Tweets. The social media platform is popularly used to keep up to date with global news sources, celebrities’ and other high profile personalities. The application encourages users to interact and engage with others, however limits them to only 100 characters. Providing a unique new forum that is both restrictive and innovative in its practice. By limiting character count users are able to deliver messages in a clear and concise manner. Enabling individuals to share articles and content that substantiates their ‘tweet’ message. Twitter allows users to follow their desired accounts enabling easy access to current affairs, pop culture and contemporary events of their interest.

Flowchart describing my process of scraping data

Through the use of an automated process I was able to forage through twitter to find articles surrounding the topic of gender equality and female empowerment. By crediting words such as gender, empowerment, equality, violence and rape, I was introduced to a spreadsheet of 762 findings.


Screen Shot 2016-09-04 at 9.32.42 pm

I synthesized my results by selecting articles from users with 10,000 followers and above, as this provided more legitimate articles and issues surrounding gender equality and female empowerment. Through this synthesis I was repeatedly directed to twitter pages that were devoted to the issues surrounding gender equality and female empowerment. Four pages repetitively appeared on the spreadsheet, @HIRED_UK, @RoomtoRead and @RosaForWomen.

Hired UK explores gaps that are prevalent within workplaces, with a large focus on gender pay gaps. Directing me to an article that explores reasons why gender equality is being held back. [Figure 1.]

Screen Shot 2016-09-04 at 7.30.41 pm

The article identifies “two findings that unavoidably coexist on the spectrum of attitudes about gender and the relative values of certain kinds of work.” (R. Bellis, 2016) The findings communicate that the home front is still seen as feminine and 11% of U.S. employees don’t believe in equal pay. The author identifies that these ingrained ideas are resisting the pursuit of gender equality.


Room to Read promotes literacy and girls’ education across Asia and Africa and guided me to an article that provides predictions on the resolution and closure of gender inequality.

Screen Shot 2016-09-04 at 7.50.39 pm

The article gives insights into the progression of equal pay, stating that “by 2018 companies will be required by law to publish how much they pay men and women” (S. Wadmore, 2016). Whilst also predicting ‘Masculism’ a future movement that will arise as a repercussion of feminism. Giving men the opportunity to redefine what being a man is.


Rosa For Women is the only UK-wide Fund for Women and Girls. The empowering page lead me to an article published by the Guardian titled, 12 Steps to Achieve Gender Equality in Our Lifetimes. The article reviews necessary steps that will help achieve gender equality, whilst also providing insight into current unequal trends such as sexual harassment, child marriage and traditional gender roles.

Screen Shot 2016-09-04 at 7.58.51 pm

From this process I was directed to informative and current content based on gender equality and female empowerment.When refining and scraping the data collected I ensured to carefully analyse the verification and reliability of the page, enabling trustworthy content. This form of researching allowed me to find reliable sources that I would not have been exposed to without the twitter archiver tool.Whilst, also directing me to twitter activists for gender equality, providing me with an abundance of material and insights on the issue.


5 point summary

  • The more specific the search rules = the more precise the sources
    • Specific: #rape #rapeculture #violenceagainstwomen #sexualharrassment
    • Widespread: #GenderEquality #Womensempowerment #feminism
  • You can legitimise your findings by reviewing the verification of the source
  • Data scraping allows you to find patterns within your searches
    • Pages repetitively discussing the same topic
    • Repeated articles
    • Location of users
  • Crediting words such as gender, empowerment, equality, violence and rape lead to articles primarily focused on equal pay and education
  • In the future I will provide more specific search rules, in order to narrow my findings



Written by Zara Hartwig



Reference List:

A. Leach, 2016, 12 Steps to Achieve Gender Equality in Our Lifetimes, Date Accessed: 03.09.2016, <;

R. Bellis, 2016, 3 Overlooked Reasons Why Gender Equality is Being Held Back, Date Accessed 03.09.2016, <;

S. Wadmore, 2016, The Future of Gender Equality, Date Accessed: 03.09.2016, <;


Gender Equality – -Gender equality- Gender OR equality OR empowerment OR violence OR rape OR culture #genderequality O

%d bloggers like this: