Bram's Dev Blog

home

100 Days of Code Day 75 - Scholar Bot Detection

07 Nov 2018

Switching up the user agents

Not sure if it will make a difference, but to rule it out as a possibility i’m switching to a different user agent string.

Sleeping 5 seconds between homepage request and search query

Right now, I first request the homepage to get (and set) the Google Scholar cookies. My query for the item results comes immediately after that. I’m putting my thread to sleep for 5 seconds to see if this makes a difference.

Unfortunately, it didn’t

Allowing users to manually set whether full text and metadata was found in Google Scholar

Even if I would be able to get passed the captcha today, changes are big that it would break again at an arbitrary point in the future. So now I’m empowering users to change the settings themselves for whether metadata or full text show up in google scholar after a manual check.

Shipped this change !

Day 76 Plan

  1. Clean up CSS for listed items, both on the repository page, as well as for the paragraph text on the item pages. Font should be darker
  2. Render the titles of the items on a repository page in different colours, depending on whether metadata and/or full text have been found.

After that, ask my colleagues for assistance to access the assets and hopefully get a successful test run in place for the DSpace 7 Angular language switch.

Future days - DSpace 7 Angular

Future days - Analyzer.atmire.com work

Future Days - Productivity

Future days - Jekyll http://bram-atmire.github.io/ site

Future Days - Atmire.com work

Investigate and work on search engine optimization (SEO) for the main atmire.com website.

Future Days - Learning just for learning

Sustainability challenge - Finish before Christmas

If I continue like October, I could hit day 68 by end of October and day 98 by end of November.