I wish AI leaders would fund more data archival projects, instead of hoarding it themselves. They could build a lot of goodwill, but the current state of the art is destroying old books and denying anyone access to them.
These archival projects have an enormous impact, though they are chronically underfunded. I wish there were a strong economic incentive to support them.
Anthropic just learned to the tune of a 1.5 billion dollar settlement that making available for human consumption, even internally, instead of hoarding for AI training exclusively is a fraught exercise.
Two events: Anthropic did end up paying a $1.5B settlement for a case involving the use of pirated books. See Bartz v. Anthropic.
Second, apparently archiving content and providing it to others is risky business. It is often ruled as piracy, see Hachette v. Internet Archive. It went badly for Internet Archive, against common sense.
Everything is "chronically underfunded". There has never been a thing in the world that was "well funded" because people's objectives increase with the amount of money they have. No one has ever said "Wikipedia has enough now" or "The Internet Archive doesn't really need any more money" or "Firefox has collected enough money from Google that it can now operate independently forever".
Besides, this so-called goodwill is an illusion. Opposition to AI in the PR space is entirely dominated by motivated reasoning. Attempting to satisfy people with random things is simply going to shift complaints to "take the commons, chop it up, and give it back to us"; "Should AI companies be the arbiters of what is worth preserving?"; and other such things. These kind of public-relation gains are entirely fictive and neither the supposed beneficiary nor the supposed benefactor is fooled by the theory.
"Oh so they suck up all our water and burn our planet down and we're supposed to be happy that they chop up people's books (that humans spent years creating) just so they can give us digested slop back?". Everyone knows that this sentence is waiting in the wings, and that the entire effort is clearly bait so that someone is foolish enough to pull a Google Books and get sued into oblivion for it.
And that's saying nothing about this so-called "hoarding" of things in a world where we pulp tons of these hoarded things every day and no one cares to scan them.
Hmmm, there are definitely people saying "Wikipedia has enough now". They have enough cash to keep the site going for a long time, but they still beg for donations.
Firefox is similar, people complain about Mozilla's side projects distracting from firefox development all the time, and that they could runa lot leaner if they just focused on the browser.
I was very excited to see this project. I went to the site and requested processing of the maps for my small town. It also inspired me to make a donation to OSM.
I should have mentioned this earlier, but folks can make direct tax-deductible donations to OldInsuranceMaps.net, which is now affiliated with OSM US's 501(c)(3) status.
Awesome project! Would definitely love to see more overlap with Allmaps. It’s pretty trivial to utilize centralized gcps for both server and client applications and maintain single sources of truth. I’ve been meaning to dive a bit deeper on an old project anyway, good excuse.
This is something we need to add to the FAQ. There is some cross-compatibility already, in that OIM has endpoints to spit out Allmaps-compatible annotations from the database, e.g. https://oldinsurancemaps.net/iiif/resource/88127/. You can paste that right into Allmaps, which will in turn pull tiles from the Library of Congress IIIF server and warp them based on the annotation information. That's just the beginning though, more to come...
Seems like a very interesting challenge for automation of raster to vector scanning/software, and QC/checking that the raster-to-vector pipeline hasn't hallucinated something or joined two shapes together that should be discrete objects.
I would presume they're starting with very high res lossless raster scans and storing those in some sort of index, then making them available for different pipelines of processing?
That's basically what's going on, but it's all a manual process so far. The goal is scoped really just to the actual georeferencing and mosaic creation, no post-processing or OCR is performed on the maps within this platform (people are certainly downloading the final rasters and doing that with them though).
If you like OldInsuranceMaps, check out another OSM charter project, Yesterdays! Originally created by a team in Richmond, Virginia [0], with one more deployment covering the Inland Empire [1]. They plan to merge soon. Yesterdays imports Sanborn maps to help place historical photos as accurately as possible.
Absolutely, this would be incredibly cool, and if they can do it with burnt ancient scrolls it should be a cakewalk with these... Even just shining very strong light behind the pages you can see all the old layers. However, most of the LoC collection is original copyright deposits, so much of what's on OIM doesn't have pasted-on slips.
On visiting the actual oldinsurancemaps.net, I was met with:
"OldInsuranceMaps.net is a Charter Project of OpenStreetMap US that provides an open, online workflow for creating georeferenced layers of historical maps, with a special focus on making seamless mosaics from multi-page atlases in the Sanborn map collection at the Library of Congress."
Sanborn was an insurance adjacent company that made detailed paper maps of many towns in the US in the late 1800s and early 1900s.
This project is scanning those maps and adding the information that modern mapping software needs to show them alongside other geographic information. Where the old paper maps overlap, they work to cleanly stitch them together rather than showing the edges of the paper.
Later: The project is sort of for people looking for the old maps, so I can see why the introduction doesn't spend a life of time explaining them.
They are building computer programs that other people can use to then post online maps from old paper sources. For example, from the Sanborn map collection.
Those maps can be browsed as "overlays" on top of other maps, so you can easily compare how several maps show the same places on Earth.
These archival projects have an enormous impact, though they are chronically underfunded. I wish there were a strong economic incentive to support them.
Second, apparently archiving content and providing it to others is risky business. It is often ruled as piracy, see Hachette v. Internet Archive. It went badly for Internet Archive, against common sense.
not really
they were operating under model when they remotely lended the same amount of books they had (effectively operating as a library)
then they started doing it without any limits whatsoever, which in fact is piracy and illegal
some people with common sense begged them to stop and stay in legal operating area
they refused, got (predictably) sued as publishers were unhappy already about remote library and (as predictably) lost case
Besides, this so-called goodwill is an illusion. Opposition to AI in the PR space is entirely dominated by motivated reasoning. Attempting to satisfy people with random things is simply going to shift complaints to "take the commons, chop it up, and give it back to us"; "Should AI companies be the arbiters of what is worth preserving?"; and other such things. These kind of public-relation gains are entirely fictive and neither the supposed beneficiary nor the supposed benefactor is fooled by the theory.
"Oh so they suck up all our water and burn our planet down and we're supposed to be happy that they chop up people's books (that humans spent years creating) just so they can give us digested slop back?". Everyone knows that this sentence is waiting in the wings, and that the entire effort is clearly bait so that someone is foolish enough to pull a Google Books and get sued into oblivion for it.
And that's saying nothing about this so-called "hoarding" of things in a world where we pulp tons of these hoarded things every day and no one cares to scan them.
Firefox is similar, people complain about Mozilla's side projects distracting from firefox development all the time, and that they could runa lot leaner if they just focused on the browser.
Also here's a past HN thread with some more context: https://news.ycombinator.com/item?id=46788588
I should have mentioned this earlier, but folks can make direct tax-deductible donations to OldInsuranceMaps.net, which is now affiliated with OSM US's 501(c)(3) status.
https://openstreetmap.app.neoncrm.com/forms/donate-oim
https://cartes.gouv.fr/explorer-les-cartes/
you can add and remove layers by clicking on the map icon on the top right.
I would presume they're starting with very high res lossless raster scans and storing those in some sort of index, then making them available for different pipelines of processing?
0: https://yesterdays.maprva.org/
1: https://yesterdays.inlandempire.place/
"OldInsuranceMaps.net is a Charter Project of OpenStreetMap US that provides an open, online workflow for creating georeferenced layers of historical maps, with a special focus on making seamless mosaics from multi-page atlases in the Sanborn map collection at the Library of Congress."
What the hell does that mean? Anyone to ELI5?
This project is scanning those maps and adding the information that modern mapping software needs to show them alongside other geographic information. Where the old paper maps overlap, they work to cleanly stitch them together rather than showing the edges of the paper.
Later: The project is sort of for people looking for the old maps, so I can see why the introduction doesn't spend a life of time explaining them.
They are building computer programs that other people can use to then post online maps from old paper sources. For example, from the Sanborn map collection.
Those maps can be browsed as "overlays" on top of other maps, so you can easily compare how several maps show the same places on Earth.