I only vaguely understand the last half of your third paragraph, even though I think I've been in a similar position. One of my jobs at the Stanford library was to periodically run the URLs attached to catalog records (online versions of the texts, supplementary material, etc.) through a URL checker and update the ones that had changed. This required jumping through some amazing hoops to format the data, some of which hoops I had to invent myself, and I remember thinking as I wrote up the procedure prior to leaving the job, "Will anybody understand why they have to do it this way? Will anybody continue to do this job at all?"