As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification
I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our corpo-political masters". It is surprising to see how much they have stripped from our view - long tail results, actual results for product reviews and not ad spam, no preference for 20 page recipe sites.
There are still illegal streaming sports and movie sites everywhere (who knew) and all other seedy corners of the internet that have been neatly erased by Google. It makes me nostalgic for that brief window of time when the web was truly uncontrolled, when page rank had meaning and you didn't know if your search would return 0 results or 4,000 pages, which you could actually browse.
> There are still illegal streaming sports and movie sites everywhere (who knew) and all other seedy corners of the internet that have been neatly erased by Google
This is such a strange position to take. Apple doesnt allow all sorts of apps on the appstore, but that is never said as "Apple is erasing illegal streaming". Google is simply not showing the results. They are not taking down, or banning, or doing anything to the websites.
The pain being made is: If you use a search engine as your eyes to see what exists “on the internet”, then, absolutely, whatever Google hides from its results or fails to index is “erased” from “your” experience of the internet.
I see piracy sites just fine on Google. Almost always the top result. That's why I go there to find the next domain after a previously working one gets shut down.
If the companies weren't trying to destroy trust and take everything away from everyone these things would never have much traction but now they deserve to have more than ever.
> Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our corpo-political masters".
As long as you don't search anything related to Russia itself, or to Russian interests elsewhere (like their invasion of Ukraine). Then it's very heavily biased, priority is given to state ran propaganda mills, independent media is hidden from results, etc.
And before anyone starts thinking about whatabouting, Yandex are based in a country where journalists are openly and publicly assasinated to intimadate. It would be delusional to expect any sort of press and related (like search engine or aggregation) freedom, or to try to compare this to anything in any other developed country.
Yep, plenty, and tbf it's entirely logical. The people working at Yandex do not want find themselves dying of a nerve agent or polonium, which is a real and acute danger for people who displease the ruling regime.
I was suspicious when they started obfuscating URLs in their own browser, then on their SERPs, and now this...
For many years, I had my filtering proxy rewrite the URLs in the way mentioned in the article.
Almost exactly a year ago, Google stopped working without JS. I stopped using Google.
Now they're upping the game, and as the article (which is a bit of marketing itself) admits, those who have the resources can still blast through these obstacles while those who don't are locked out.
Since the article brings up "AI scrapers", I'll just point it out as being the latest scare-tactic for coercing people to give up the privacy, anonymity, and (browser) freedom of an open interoperable Internet.
I've been using DuckDuckGo for years now, ever since it became noticeable that two different people searching for the same search term would get two different results back from Google. Meaning they were no longer completely reliable: they might show one person a result that they hide from the other person by burying it on page 3 where few people ever look.
DDG's search results have been poorer recently than they used to — I often see completely unrelated results (to the point of my saying "Why in the world did that come back as a search result??!?") starting from page 2. And yet, I still use them, simply because they aren't Google.
I don't know how global consistency is actually useful, but I can easily think of ways it is less convenient.
For example, DST starts and ends on different dates in the UK and the US. Which dates should google return when someone from either country searches for just "DST end date?" Someone lives in Orange County and searches for "Orange County Sheriffs," which one of the eight Orange Counties should google return?
These are both examples where a localized — not even customized, just localized based on IP addresses — search results will easily help reduce headaches.
I wouldn't mind that one. That a man in the US and a woman in Japan would get different results for a search for "sushi restaurant" is perfectly reasonable (even if the woman in Japan was searching in English).
It's when two people in the same neighborhood got different results for the same search that I said "wait a minute, they're personalizing search results now for ad-targeting purposes" and ditched them. The potential for them to deliberately hide things from you was too great.
It means they're capable of burying news stories that would contradict your worldview and pushing news stories that support your pre-existing biases, leading to more engagement from you (a win from their point of view) but also burying you in an echo chamber. And unless you were in the habit of doing the occasional search in Incognito Mode, you wouldn't know. (And even then, they probably would be able to put together enough clues to figure out your identity even without your Google login cookie).
Yes, the fact that they could do it does not prove that they were doing it, not right away. I ditched them as soon as I found out that they could do it, because I was absolutely certain that eventually, they would end up doing it. And I wanted neutral search results, not biased ones, even ones biased towards my own point of view.
I subscribe to The New York Times, if I am searching for a news event, I’d appreciate a site that’s not paywalled and I trust be the first result if it is reasonable.
The age of internet search is over. The age of Cloudflare has begun. It wouldn't be possible to build a search engine now if they wanted to... and there wouldn't be anything to search for anyway. The non-corporate internet withered into dust and blew away in the wind.
If you could find what you want, how would they ever sell you what they want you to buy? And I'm not just talking merchandise, though that too. Your political narratives, your values, opinions, everything. And everyone likes it so much they just sit there scrolling and swiping and tapping.
> Sometimes, these redirect URLs take a perceivable amount of time to load, which is very irritating.
Great. On top of my on-going battle with Windows + Firefox + DNS/TLS resolution sometimes stalling for seconds at a time, another few second server-side stall is introduced.
I swear that every day modern computing scenarios get slower and slower instead of snappier and snappier.
Wow - this exact same bug has been happening to me too. I gave up on troubleshooting it after the first few attempts came up with nothing, assumed it was just unique to me.
The link you followed when you clicked hasn't a direct link for years, decade afaik (they mangle so they can see what's followed). The page used to show the direct on the search text but now it shows some stand in for it - sometimes. You can see the direct link on the bottom of the screen when you hover - sometimes (and sometimes you see a mangled link). Sometimes the google link contains the original link in the center also[1].
The situation seems to vary from result to result even on the same page of the same search - at least on the test search I just did. You can figure out what happening to an extent but this very inconsistency seems to speak to a dystopian quality to today's information gatekeepers.
It's horrible. You almost can't share something from a pixel phone without it being a googlified link. You can but you have to jump through hoops.
If I want to share something from the feed I hit the share button. Which creates a fucking nonsensical Google link. The only way is to "open in Chrome browser" and copy the link from there.
You literally have to sell your soul and give and infect the recipient with a golden cookie. It's a deliberate effort to keep their shitty ads business alive.
At least they seem still provide results for my searxng instance. I mean sure, they are horrible but duckduckgo just blocks most queries (and I'm the only person using the ip / seraxng instance)...
Next i'll do is to implement tavilly, exa, tinyfish etc. as search engines for searxng. No agents, no mcp, just their search api endpoint.
It's sad that instead of searching things other people put up, we're basically asking sam or dario oracle to tell us the truth. The people should be furious. But we've internalized this idea that they are somehow better.
Dont worry in a year or two, Google wont even redirect you to the actual true url, instead everything will be a page with all links rewritten so all http is tunnelled thru them.
> Combined with earlier moves like removing &num=100
Removing this made google search horrible to use. I often use command+f to quickly identify relevant search results, but doing it on 10 results at a time is so laborious that I just don't bother using Google search, resulting in less searches and use of other tools instead.
Kagi is what I settled on a couple of years ago. I do think sometimes the fawning is over the top, but it is a solid search engine and tended to give me a bit better results than Google out of the box. The real win, though, is that you can give various sites a weight, so the search results will prefer or avoid sites according to your desires. Once I had that going, my search results tended to be much better than Google.
Yeah, that's the thing for me: filtering out the SEO crap that Google happily serves up.
Google's actual search results are a waste of time visiting, both for the mindless CEO content and the ad-laden, analytics-happy, javascript-heavy websites.
So I tend to use the AI overview. But plugging myself into the all-seeing corporate oracle, that grew on all the web's content, and now seeks to supplant it seems unseemly.
It's just sad that kagi will likely only be a fringe thing, and google will continue to promote these foul, foul, mindless websites and then supplant them with its AI.
Because I wouldn't want Google to know what links I click on. When I used to use Google, and they added such redirects that included the destination URL as a parameter, I'd edit the link to make it just that URL before resolving it. This new scheme would make that impossible.
I use a Firefox extension to rewrite those Google redirects into plain links, to remove Google tracking of which links I click. The extension is broken now
Blocking URL shorteners and google-ad links yes! Personally for me it's also the fact that this is effectively an unresolvable URL shortener, store that link somewhere and it will most likely be dead. Can't copy link anymore and paste it on a notepad or chat app to check it out later as there isn't a guarantee it will load at all (ie: the problem with url shorteners).
Well just as a regular user, I think it is pretty annoying because if I look up anything on Google while in Incognito Mode and hover over a search result, I can see that maybe the top result is maybe Wikipedia, or Instagram, or some other less-known website depending on what I'm searching for. Now, that's all very obfuscated because I don't actually know where I'm going to land for sure.
They were already tracking everything you click but for example if you want to send a link to someone you can't copy the link from the Google result and send it to them, you'd either send them the Google tracking link or go to the website yourself.
It's primarily relevant because it makes scraping search results much more expensive, solidifying Google's effective monopoly on Internet search.
Google has previously tried to prevent scraping of search results using legal means, but courts correctly think that scraping of Google's search results should be legal, just as Google's scraping of the whole Internet is legal. This is Google's reaction to that.
This is the best explanation. They’ve been doing the same in Google News. Each entry comes not with a URL to the source, but with a hash. To resolve it, you must send requests to Google’s servers. Anyone who wants to create a list of URLs of sources automatically can therefore be blocked by Google now on two levels rather than one - the search for a list of results, and identifying the source URL for each result.
In effect, they’re removing attribution from the content they quote from other people’s websites. It would be interesting to see if courts object to that. It is one thing to crawl other people’s websites and display snippets of their work as your search results when each result is properly and transparently attributed. But if the text is quoted and the source is not there alongside it in plaintext, replaced only by a vague promise that, if you ask, we may or may not tell you where this piece of content is from, that is a very different deal.
You can opt out from Google scraping you though? In theory you can opt out of anyone scraping you (if people were well behaved). Google should get to opt out of being scraped too.
> the reasons are so they can track who you are and sell your profile advertising.
What? Like they weren’t doing this before? Obviously Google’s telemetry is tracking every link you click regardless; there’s no extra tracking benefit to this.
The reason they’re doing this seems to be to stop competitors from scraping their search results.
it sounds like it primarily matters if you are a customer of this company, one that is building a search index off of urls scrapably hardcoded (or at least so as to be easily unencodable in non-realtime, it sounds like?) inside google search result redirects. in theory, there could be noticeable consumer user impact, but ... it would have to be a pretty large theory
The security implications of this are very severe when you consider the amount of people who google government websites, banking, crypto and others. And google will happily serve you a phishing website either in ads or results.
I... Don't see it? It's the result page right? I just search some random string on Google and the results are all direct URLs. Do they get resolved via javascript after page load and replaced automatically? Or am I looking at something else?
What I see now, and it's been like this for a while, is this:
You get the results and they do have direct URLs. But then, if you do some things with the link, e.g. right click to open it in a new tab, it swaps the URL to the indirect one. The idea is that initially you see a normal link, with a normal URL which will be displayed correctly when you hover the mouse over it, but right before you click it, it's swapped for the indirect one.
So, they have been doing stuff like this for a while and it has been somewhat fluid, because the swapping can occur on different events and I have also seen it load with all the links pre-swapped to the indirect ones, sometimes.
So, yes, what you see may be different and you may get the indirect URLs swapped at different stages.
Yeah seems pretty obvious to me that most people are not going to be using a search engine in 5 years. In the sense of searching for something and combing through the results to find the answer.
Whenever I've tried this, it seems okay if you need an answer to a question, but plain bad if I'm looking for a specific page.
e.g. I'm just now looking for the menu for a local restaurant. "restaurantname menu" in Kagi (Google would presumably be similar) returns a link to the menu as the first result in about a second. Or "restaurantname menu !" goes directly to the menu in about a second.
Meanwhile, searching "restaurantname menu" in chatgpt takes about 5 seconds to return an embedded map from mapbox showing the location of the restaurant. If I click the restaurant pin on the map, there's no menu link, the 667 reviews have no link or way to view, and the restaurant description literally says "I don't have enough information to identify which local business <restaurantname> refers to."
Below the map there's some text: "If you mean <restaurantname> in <place>, here’s the current menu. <restaurantname>". The <restaurantname> link just opens the same card as clicking the pin on the map.
After that there's a bullet point list of the menu that ommits a ton of detail and options.
After that there's finally a link... that I can click to open up a popup at the bottom of the page with an actual link to the menu.
This was literally the first thing that popped into my head, I didn't have to put any effort into finding a query where chatgpt falls on its face.
ISPs can presumably correlate the Google query string with the request following the response to the goto and so make a search index? I guess they would charge too much.
Do any large ISPs use visit data to feed into a search index?
Unless you're using DNS over HTTPS they can see the unencrypted DNS traffic.
There's also Encrypted Client Hello, but they can also see which IP you're connecting to.
why is this such a bad thing? it's not really any different from using a uuid as a user facing key, which basically everyone does.
and trying to protect your moat isn't automatically a bad thing. they clearly feel it's helping competition, so they're closing a hole. competition is good doesn't mean help your competitors.
- coming from someone who's been using fastmail as my personal for ~10 years because i don't want my emails to be backprop fodder
I guess someone made a website which google crawled and adding a senf made uuid to it is like google trying to own it rather than just being a true search engine just having index to it.
It hasn’t provided direct URLs for decades? Not exactly new behaviour.
I’ve got something that will blow your mind. Google now has tracking analytics, for get this, your business’s phone number. Some “Adsense partner” convinced our web admin to install a little script which changes your phone number on your website so they track phone call enquires back to search engine leads / advertising spend.
Yeah no thanks, that was creepy as hell and had it rolled back ASAP. You’ve got to realise the power these tech companies hold over your business. Don’t show up in the search results, someone lists your business as closed in maps, a tracking phone number goes dead so they can’t call you, you might as well have shut up shop and ceased to exist.
Would that company then sell that data back to Google? The consolidation of control of information under a single actor is certainly a factor, but it's not that simple, and it's not the only factor. You shouldn't be so eager to outsource as much of your business intelligence as possible.
I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our corpo-political masters". It is surprising to see how much they have stripped from our view - long tail results, actual results for product reviews and not ad spam, no preference for 20 page recipe sites.
There are still illegal streaming sports and movie sites everywhere (who knew) and all other seedy corners of the internet that have been neatly erased by Google. It makes me nostalgic for that brief window of time when the web was truly uncontrolled, when page rank had meaning and you didn't know if your search would return 0 results or 4,000 pages, which you could actually browse.
It's really not free. You're paying with your data.
This is such a strange position to take. Apple doesnt allow all sorts of apps on the appstore, but that is never said as "Apple is erasing illegal streaming". Google is simply not showing the results. They are not taking down, or banning, or doing anything to the websites.
As long as you don't search anything related to Russia itself, or to Russian interests elsewhere (like their invasion of Ukraine). Then it's very heavily biased, priority is given to state ran propaganda mills, independent media is hidden from results, etc.
And before anyone starts thinking about whatabouting, Yandex are based in a country where journalists are openly and publicly assasinated to intimadate. It would be delusional to expect any sort of press and related (like search engine or aggregation) freedom, or to try to compare this to anything in any other developed country.
https://hal.science/hal-03217497
https://euvsdisinfo.eu/yandex-from-tech-innovation-to-inform...
https://pmc.ncbi.nlm.nih.gov/articles/PMC10130930/
https://misinforeview.hks.harvard.edu/article/a-story-of-non...
For many years, I had my filtering proxy rewrite the URLs in the way mentioned in the article.
Almost exactly a year ago, Google stopped working without JS. I stopped using Google.
Now they're upping the game, and as the article (which is a bit of marketing itself) admits, those who have the resources can still blast through these obstacles while those who don't are locked out.
Since the article brings up "AI scrapers", I'll just point it out as being the latest scare-tactic for coercing people to give up the privacy, anonymity, and (browser) freedom of an open interoperable Internet.
DDG's search results have been poorer recently than they used to — I often see completely unrelated results (to the point of my saying "Why in the world did that come back as a search result??!?") starting from page 2. And yet, I still use them, simply because they aren't Google.
For example, DST starts and ends on different dates in the UK and the US. Which dates should google return when someone from either country searches for just "DST end date?" Someone lives in Orange County and searches for "Orange County Sheriffs," which one of the eight Orange Counties should google return?
These are both examples where a localized — not even customized, just localized based on IP addresses — search results will easily help reduce headaches.
It's when two people in the same neighborhood got different results for the same search that I said "wait a minute, they're personalizing search results now for ad-targeting purposes" and ditched them. The potential for them to deliberately hide things from you was too great.
Yes, the fact that they could do it does not prove that they were doing it, not right away. I ditched them as soon as I found out that they could do it, because I was absolutely certain that eventually, they would end up doing it. And I wanted neutral search results, not biased ones, even ones biased towards my own point of view.
If you could find what you want, how would they ever sell you what they want you to buy? And I'm not just talking merchandise, though that too. Your political narratives, your values, opinions, everything. And everyone likes it so much they just sit there scrolling and swiping and tapping.
The base64 data appears to consist of a very basic protobuf structure, containing a long string of bytes in field 2 which presumably identify the URL.
Sometimes, these redirect URLs take a perceivable amount of time to load, which is very irritating.
Great. On top of my on-going battle with Windows + Firefox + DNS/TLS resolution sometimes stalling for seconds at a time, another few second server-side stall is introduced.
I swear that every day modern computing scenarios get slower and slower instead of snappier and snappier.
The situation seems to vary from result to result even on the same page of the same search - at least on the test search I just did. You can figure out what happening to an extent but this very inconsistency seems to speak to a dystopian quality to today's information gatekeepers.
[1] Example. https://www.google.com/url?sa=t&source=web&rct=j&opi=8997844...
If I want to share something from the feed I hit the share button. Which creates a fucking nonsensical Google link. The only way is to "open in Chrome browser" and copy the link from there.
You literally have to sell your soul and give and infect the recipient with a golden cookie. It's a deliberate effort to keep their shitty ads business alive.
Next i'll do is to implement tavilly, exa, tinyfish etc. as search engines for searxng. No agents, no mcp, just their search api endpoint.
[0]: https://news.ycombinator.com/item?id=49665572
Removing this made google search horrible to use. I often use command+f to quickly identify relevant search results, but doing it on 10 results at a time is so laborious that I just don't bother using Google search, resulting in less searches and use of other tools instead.
Google's actual search results are a waste of time visiting, both for the mindless CEO content and the ad-laden, analytics-happy, javascript-heavy websites.
So I tend to use the AI overview. But plugging myself into the all-seeing corporate oracle, that grew on all the web's content, and now seeks to supplant it seems unseemly.
It's just sad that kagi will likely only be a fringe thing, and google will continue to promote these foul, foul, mindless websites and then supplant them with its AI.
Google has previously tried to prevent scraping of search results using legal means, but courts correctly think that scraping of Google's search results should be legal, just as Google's scraping of the whole Internet is legal. This is Google's reaction to that.
In effect, they’re removing attribution from the content they quote from other people’s websites. It would be interesting to see if courts object to that. It is one thing to crawl other people’s websites and display snippets of their work as your search results when each result is properly and transparently attributed. But if the text is quoted and the source is not there alongside it in plaintext, replaced only by a vague promise that, if you ask, we may or may not tell you where this piece of content is from, that is a very different deal.
As others have elaborated, the reasons are so they can track who you are and sell your profile advertising.
What? Like they weren’t doing this before? Obviously Google’s telemetry is tracking every link you click regardless; there’s no extra tracking benefit to this.
The reason they’re doing this seems to be to stop competitors from scraping their search results.
Brave has its own independent index which is cool.
You get the results and they do have direct URLs. But then, if you do some things with the link, e.g. right click to open it in a new tab, it swaps the URL to the indirect one. The idea is that initially you see a normal link, with a normal URL which will be displayed correctly when you hover the mouse over it, but right before you click it, it's swapped for the indirect one.
So, they have been doing stuff like this for a while and it has been somewhat fluid, because the swapping can occur on different events and I have also seen it load with all the links pre-swapped to the indirect ones, sometimes.
So, yes, what you see may be different and you may get the indirect URLs swapped at different stages.
Maybe Google is observing this?
e.g. I'm just now looking for the menu for a local restaurant. "restaurantname menu" in Kagi (Google would presumably be similar) returns a link to the menu as the first result in about a second. Or "restaurantname menu !" goes directly to the menu in about a second.
Meanwhile, searching "restaurantname menu" in chatgpt takes about 5 seconds to return an embedded map from mapbox showing the location of the restaurant. If I click the restaurant pin on the map, there's no menu link, the 667 reviews have no link or way to view, and the restaurant description literally says "I don't have enough information to identify which local business <restaurantname> refers to."
Below the map there's some text: "If you mean <restaurantname> in <place>, here’s the current menu. <restaurantname>". The <restaurantname> link just opens the same card as clicking the pin on the map.
After that there's a bullet point list of the menu that ommits a ton of detail and options.
After that there's finally a link... that I can click to open up a popup at the bottom of the page with an actual link to the menu.
This was literally the first thing that popped into my head, I didn't have to put any effort into finding a query where chatgpt falls on its face.
I guess that the web chat can have a search skill to remove the prose and give only links, plus maybe an excerpt of each result
Its not because chatgpt is so superior. Its just because google search is dogshit.
They work on killing the web as we knew it and I fear its kinda working.
Streaming services already adding in ads to “ad-free” tiers they’ve now named “premium”.
Quality of life on the internet has gotten shitty while Reality Classic stays mostly the same, though more expensive.
Do any large ISPs use visit data to feed into a search index?
I genuinely forget I’m not using Google until I come across articles like this
Google has been encoding the target url for years.
I don't miss Google at all.
Goodbye you shit company.
Nothing good ever comes from businesses desperately trying to protect their moats rather than making their products better so they don't need to.
In reward for making Google Search materially worse in every way except ad revenue, he failed up again and was promoted to a cushy do-nothing role.
Everything wrong with the tech industry, embodied in a single person.
and trying to protect your moat isn't automatically a bad thing. they clearly feel it's helping competition, so they're closing a hole. competition is good doesn't mean help your competitors.
- coming from someone who's been using fastmail as my personal for ~10 years because i don't want my emails to be backprop fodder
I’ve got something that will blow your mind. Google now has tracking analytics, for get this, your business’s phone number. Some “Adsense partner” convinced our web admin to install a little script which changes your phone number on your website so they track phone call enquires back to search engine leads / advertising spend.
Yeah no thanks, that was creepy as hell and had it rolled back ASAP. You’ve got to realise the power these tech companies hold over your business. Don’t show up in the search results, someone lists your business as closed in maps, a tracking phone number goes dead so they can’t call you, you might as well have shut up shop and ceased to exist.
Some of the things TV companies do will shock you too.
https://docs.clearurls.xyz/
I recommend reading more than the headline
The article also suggests that it cannot be decoded.