Track working papers by monitoring website change
date
Nov 20, 2023
slug
webmonitor
status
Published
tags
Tech
summary
type
Post
ID
104
Unsolved/Solved Issue:
BCC (Blind carboncopy) Block recipient from seeing each other’s email address (29 Nov 2023)
Sometimes track very tiny updates 😅
Report also changed text and text near changed text (29 Nov 2023)
Fix the function of tracking and reporting changes
Track also R&R papers and forthcoming papers
Send only one email every time (22 Dec 2023)
Also track the number of working papers? (May 2024)
Run regularly on server (now on AWS, every Monday June 2024)
Change from EC2 to AWS Lambda to save costs
Keeping track of the work of famous scholars is beneficial as it helps you to grasp the cutting-edge advances in research. Following their latest publications and working papers is certainly a good way to do this.
There are four possible ways to track the latest (working) papers, attending conferences and seminars, getting distributions from friends and colleagues, following SSRN/Researcher APP updates and searching scholars' personal websites. Each of these approaches undoubtedly has many drawbacks, including randomness, inefficiency and incompleteness.
I have tried to use the following way to comprehensively track the work of renowned scholars. Specifically, I locally deploy a program to track the change of Research Section of finance scholars’ personal websites (probably means they have new publishes or working papers). What means the Research Section of personal websites? Like these:
Basically, my program will send a brief email to you if any changes are detected on these pages (I set a trigger of more than 3 words 😅 hope it works). The email is from my personal Gmail, looks like this:
Then you can click the link and see whether there is something new 🤩. You can join/quit by filling this form📩.
Also, I show you the changed text (at the current stage, it is not very clear) and the number of R&R papers and forthcoming papers, which I believe is worthwhile looking at. And you should not worry about your privacy as I use BCC (Blind Carbon Copy) to send emails.
This program repeats itself everyday Monday (on AWS). Code is here (updates are on Github) so that you can make your own tracker (please reload again and again until code appears, I don’t know why 😅 or go to my Github).