Which as some running a website raises a fascinating question. If Google is just going to crawl my sites and present information as an AI summary on their site, then what exactly do I gain by allowing Googlebot to crawl my sites?
pflenker 19 minutes ago [-]
A couple of years back I worked with a company which maintained specific data which was the main traffic driver on that page.
Google approached them and wanted to pay for the rights to get the data and display it on top of the search results, a feature which was fairly new back then.
This was an interesting dilemma because it was very clear that the money was way less than the loss in ad revenue due to traffic drop, but it was also clear that if we wouldn’t take the deal, a more desperate competitor would, which would result in the same traffic loss but without the extra google money.
So the company took the deal.
History repeats itself here, with the difference that instead of paying for the data, the ai crawlers simply take it for free.
iamtedd 11 minutes ago [-]
Real-world Prisoner's Dilemma.
tedd4u 1 minutes ago [-]
[delayed]
coldpie 2 minutes ago [-]
> allowing Googlebot to crawl my sites
As far as I know, you don't have a choice. They have no obligation to respect your wishes, and LLMs are allowed to scrape & republish your content.
Andrex 20 minutes ago [-]
Free speculation: I could see a future where Google populates a footer on results with the website logos of the sources. Presumably, the new web will require users to memorize websites/brands and go directly to those sites if they see a lot of their results are being provided by one source.
Websites may go back to being simply labors of love.
whazor 25 minutes ago [-]
Websites tend to be updated and considered to be the source as well.
thedelanyo 13 minutes ago [-]
> then what exactly do I gain by allowing Googlebot to crawl my sites?
Mention
swarnie 10 minutes ago [-]
Allow? Deep down, do you think you have a choice?
Mechanisms might exist to make you think you have one, the same way copywrite should prevent millions of books being gobbled up by TheZuck but ultimately do you really have a choice?
Rules and laws don't exists for you.
alt227 4 minutes ago [-]
Yes, Google advertises its crawler IP ranges and it is quite easy to keep track of this and block them. But only if you control the infrastructure that your site runs on of course.
SoftTalker 6 minutes ago [-]
The web as we know it is over.
Websites will die on the vine if LLMs intermediate all the content.
The "website" of the future will be an API optimized for LLM crawlers, serving plain-text content that no end-user will ever view directly. The SEO game will change to LLMAO.
LetsGetTechnicl 39 seconds ago [-]
That sounds like absolute hell
pclowes 40 minutes ago [-]
I understand why they are doing this. My Google search usage is easily down 50%+. I doubt I am unique here.
While there are times where I want pure search (Kagi, Old Google) I mostly use LLMs to search now and have them provide me links for source data.
When I do use LLMs as a search engine I always want it integrated into my AI workflows with access to tools and scripts etc. I never want to have a conversation with a website that is geared towards advertising me products.
mghackerlady 8 minutes ago [-]
I think LLMs are good answer engines, but terrible search engines. For example, if I just want the answer to "How do I foo this bar with a thingamajig", or "what kind of foo exists for bar", LLMs are 100% better because they'll give me an answer without trying to sell me their thing or pump me with ads. It also lets me go more in depth than the very surface level seo spam sites that appear when you do a search like this. On the other hand, search engines should be more like "<thing>s released 1908" or "<topic> and give you results talking about what you searched. If I search for C algorithm design, I don't want to learn what C is, what an algorithm is, and various other seo garbage. I want to learn about C algorithms and their design. If I search for influential books from 1908, I don't want "top 10 classics from the 1900s" or "hidden gems of 1908".
Currently, search engines are pretty bad at the second one because people try to use them as the first one
ivraatiems 39 minutes ago [-]
I'm not at all in the same boat as you; I do not and likely will never primarily rely on LLMs for information. But it's fascinating to hear that even folks who do don't find this approach useful.
BobbyTables2 29 minutes ago [-]
For me, Google search results have gotten so poor (and other engines aren’t any better), that I’d rather just ask an online LLM for what I’m looking for.
I was once very good at advanced Google search queries but they seem to no longer respect such queries - either showing irrelevant results or none at all (that should exist).
I don’t love LLMs, but they seem to not make up stuff very often these days and usually cite links to what they summarized. Sometimes the tone of the summary is slightly wrong “algorithm X was designed for Y” (when I know it wasn’t) but it’s otherwise very close to the mark.
What does amaze me, is the LLM seems to “understand” my question with very little context — I would have to give a human many more details about goals/intent.
I know damn LLMs are not capable of thought and are just a glorified search engine, but they do it well. Perhaps all my education made me little more…
I used to mock Sci-Fi movies where characters lazily dictated questions to the computer and it gave high quality answers.
We’re living in that world now.
sonofhans 1 minutes ago [-]
Kagi is better. Kagi is damn good, as much a revelation as the Google of old. Not free, though.
jolt42 7 minutes ago [-]
Yep, really advanced Google searches were never that good. LLM, yeah, it halucinates, it's never spot on but as sure as heck it knows what I'm trying to ask. It doesn't give me arborists if I say something like "list tree searches".
ignu 16 minutes ago [-]
As someone who was getting information this way most of last year, I'm pretty sure I'll never want to again.
An increasing number of studies are indicating a reliance on "AI" leads to deleterious cognitive effects. I felt this acutely myself.
I've noticed a significant boost to my recall since shunning "AI" as much as possible.
jerf 26 minutes ago [-]
LLMs-as-search-proxies have some pretty nice capabilities. For instance you can say "limit your search to scientific papers" and they'll do a much nicer job. I've also had some success recently prompting them with "I'm looking for reputable sources", e.g. recently I was looking for ways to repel deer from my apple tres. A naive internet search had vendors of shady crap jumping me. The LLMs pulled up relevant papers and university extension programs from my area.
Though I will say I get much better results from the LLMs I pay for than the free ones with Google or DuckDuckGo, which seem to be way way way more prone to just make crap up based on your search and cite web pages that, when followed, don't have the claim being made in the AI search results at all. By contrast every "source" link I've followed in the for-money AIs has 100% backed what the AI said it backed. Don't judge by the free AIs the search engines put out, those things are probably starved of resources and are nearly useless.
(Which I did not intend as a commentary on Google's plans here, but it is a data point of interest... that pressure to cut costs on the "free" services is quite directly at odds with providing quality AI services for the forseeable future.)
chrismorgan 3 minutes ago [-]
Just today, I think that I got a useful citation in DuckDuckGo’s Search Assist (AI stuff) sources for the very first time. The sources it lists have hitherto invariably been already right there in the regular results, or actually not supporting the AI output at all (the far more common case). There was also a useful regular search result in second or third position, but the one in the Search Assist citations was better, and not in the regular search results, even on the second page.
And I’ve tried Google’s once or twice and seen it used once or twice, and used ChatGPT exactly once, last week, and I was not at all impressed by any of them. Their output, for what I’ve personally seen, has been nonsense, obvious, or unverifiable.
I'm not jumping in with both feet, either, but "never" is a very big word.
ivraatiems 32 minutes ago [-]
"Primarily" is the other key there. I'll use it from time to time with sources. But it's not first-line acceptable.
Corence 31 minutes ago [-]
I think LLMs are better at finding the most helpful sources now, but that's more a testament to how much the front page of web search has lost to low value LLM content.
embedding-shape 30 minutes ago [-]
The fact that you can express "Only show me websites run by Italian companies incorporated by Greece owners born in Turkey" for example, and it'll be able to filter through a bunch of stuff, just makes searching so much easier. Fuzzy-search is also on another level with language models.
gnatolf 25 minutes ago [-]
Interestingly enough, precise search is on the way out.
ignu 14 minutes ago [-]
I'm pretty sure there'd be a double-digit drop in LLM use if Google hasn't made search worse every year for the last decade.
embedding-shape 17 minutes ago [-]
Yeah, which makes no sense, talk about shooting yourself in the foot, but this is big tech, part of the process to irrelevancy I suppose.
bdangubic 25 minutes ago [-]
this is exactly it… if google was smart they would focus on providing that experience on search vs. AI summarizing results. I want that fluid search experience, with refinement that remembers my previous ask…
VLM 3 minutes ago [-]
As a concrete example, some advertising supported topics place search as an unwanted middleman, may as well ask a LLM directly. Consider "chocolate chip cookie recipe".
Using google search, will return roughly infinite recipe sites. The sites were generated to spam AI generated recipes surrounded by advertisements. None of them are really any good because they were generated by a script and not looked at by a human until I come along and click. The standard is for all recipes to have at least 10-15 screenfulls of vertical spam wrapped by ads for recipe pages. The internet, at least using Search, is now useless for food recipes. I would have better, faster luck driving to the public library and looking in a physical cookbook; at least those recipes were probably tested at least once by humans unlike the advertising spam sites. Nobody has 45 minutes to watch 44 minutes of filler material surrounded by ads on Youtube either. If you want to cook food, the internet is near dead at this time, unfortunately.
AI search will plagiarize the "Original Nestle Toll House" recipe from the back of every bag of chocolate chips ever made. Its a good recipe and I've baked them many times over the decades.
I wish the internet were more useful, but the people in charge of it don't want it to be useful; here have some ragebait and doomscroll while watching the ads.
snailmailman 27 minutes ago [-]
LLMs are so frequently inaccurate its crazy to think of it fully replacing search.
I've been trying to use LLMs for things and it makes mistakes all the time. Just this week i had multiple instances of various LLMs basically saying "just run the software with --flag-that-fixes-your-problem" or "edit the config and add solve-your-issue=true" hallucinating non-existant options. Even if i manually link the relevant documentation pages it will still just make basic mistakes. and if im having to read the documentation myself anyway to fix the AI's mistakes, why is the AI even in the loop.
its infecting search too, because blogspam/slop articles are managing to make their way into search results by just making up untrue information, claiming software can do things it cant, or has options that don't exist.
cyanydeez 25 minutes ago [-]
My google use is down because it turned to garbage. They're likely doing this because they poisoned their own well besides the advent of LLMs.
zemo 34 minutes ago [-]
> I never want to have a conversation with a website that is geared towards advertising me products.
yeah man good thing LLMs are structurally incapable of being incentivized to sell you a product or render referral links, this is surely future-proof
NietzscheanNull 27 minutes ago [-]
Or subtly misrepresent politically inconvenient facts, or gently steer you into opinions based on a synthesis of broker data and demographic info, or quietly flag you in some database column due to exhibiting dissident-adjacent ideas or behaviors, or...
Yeah, they probably aren't doing (most of) these now, but it doesn't take much mental energy to extrapolate once you factor nearly every other tech company's ethical trajectory and the current geopolitical environment. Substituting classic search entirely with LLMs is not a savvy move.
pclowes 13 minutes ago [-]
Doesn’t classic search literally already do everything you fear LLM’s will?
pclowes 19 minutes ago [-]
My thought here is that there are many. They have proven to be commodities in most use cases.
As soon as one gets annoying, expensive, advertiser heavy etc. you just rip it out and replace it with the other one. AFAICT there is zero lock-in or moat. I often am able to switch models in one click or command. This is why all the LLM providers are desperate for a product layer/comprehensive tool set.
Sure maybe they all end up that way, but there’s plenty of reasons corporate customers will want private LLM usage that is not skewed towards advertising. I am happy to pay for that.
Also, open source models are a bulwark against another search style ad Monopoly.
SoftTalker 17 minutes ago [-]
"I never want to have a conversation with a website that is geared towards advertising me products."
Since this is how Google makes all their money, why are they killing it off? Do they think people will eventually pay for LLM search? Do they plan to stuff the results with ads, not even sharing the ad revenue with the content sources?
Sharlin 26 minutes ago [-]
About 90% of my searches are straightforward enough that an LLM wouldn't bring any added value (and all of them happen straight from the browser address bar, either to Google or Wikipedia at a more or less 50:50 ratio). And for the rest, yeah, I just use Claude or whatever directly.
zkmon 28 minutes ago [-]
> I never want to have a conversation with a website that is geared towards advertising me products.
The advertisements fed the content, which fed the AI, which in turn feeds your AI workflows. AI is still not trusted unless it's output is grounded with sources.
cortesoft 34 minutes ago [-]
That’s the thing… sure, this new search will be useful at times.
But I still want to also be able to do my normal, old school searching.
skiing_crawling 20 minutes ago [-]
I use claude/gemini as my homepage now (I have to keep switching as these companies make "updates" that periodically render their models useless). Even if I want to search for simple things, I would rather have an LLM wade through the result and extract just the information I asked for. SEO, and now mountains of slop content have made this necessary. Only a matter of time before the SEO industry in large figures out how to game LLMs too, making them equally useless.
I already saw a article recently about how to set up a business domain which can reliably show up in a search result and dump overly positive reviews into anyone's context.
snapetom 24 minutes ago [-]
We all know Google search has been broken for a long, long time. SEO trash will fill up your first page with results from trash content generation sites that repeat the same thing, usually flat out wrong. Actual meaningful results are buried deep, if Google will even let out of the "In order to show you the most relevant results" hell hole.
My experience with AI searches is that they'll still be wrong a lot of times, but it will condense/flatten the content generating trash sites and give me alternatives from these deeper results. What I'm looking for is usally in there.
imoverclocked 36 minutes ago [-]
I don't trust facts from LLMs. When I am searching for something, I usually want to find primary sources. As soon as a number is involved, I do my best to not even look at the AI output.
Even though the result is often good and combines information from multiple sources, it can also get things wrong by combining information from different eras or just plain outdated advice. AFAICT, without primary sources, the result is for entertainment purposes only.
nostrademons 35 minutes ago [-]
You can ask them to cite their sources. It's very good practice to do so, and to check those sources, because I've found that about 30-40% of the time their source doesn't support their answer at all.
nilamo 32 minutes ago [-]
If it's wrong 2 out of 5 times, why even waste your time going to it in the first place? That's a massive failure rate.
masfuerte 30 minutes ago [-]
Yes, but this is much more effort than a traditional search result that has a relevant quote from the source right there.
lunar_mycroft 19 minutes ago [-]
If I have to read the sources anyway, why not just have the model give me the links themselves? You know, like search engines already do?
jrmg 6 minutes ago [-]
Search engines don’t do that any more - they just give you a bunch of SEO spam sites, now mostly filled with plausible slop. Answers from search are _less_ reliable than answers from an LLM now.
I worry that the LLMs are just the equivalent of a ‘lagging indicator’ of web quality though - that they will also soon be overwhelmed with the sheer volume of plausible nonsense that is the web now, just like search engines are.
Model collapse everywhere.
puttycat 26 minutes ago [-]
ChatGPT is the only bot that reliably cites sources (through Web search mode).
The other bots either make up links or simply don't provide any information that is distinguishable from the LLM predictive output.
Ironically Gemini is also very bad at this, while it should have been the best at Web search.
Gemini also does something very patchy, which is to provide "links" which are in fact GET queries into classic Google search. I'm guessing they did it this way because the links generated/hallucinated by the LLM were too unreliable.
dbbk 34 minutes ago [-]
All of Google AI Mode is sourced.
25 minutes ago [-]
skywhopper 31 minutes ago [-]
Yes, and those sources often contradict the AI summary if you follow them (or if you know anything about the topic).
thfuran 30 minutes ago [-]
If it even exists.
skybrian 29 minutes ago [-]
Sometimes I use chatgpt thinking mode for searches when I expect there will be a lot of noise. "What are some in-depth reviews for <some book I've heard of>"
Have you tried explicitly asking for links to primary sources?
fidotron 40 minutes ago [-]
Objecting to this from the user end seems a bit like complaining the original Google was trying to be too magic when what you wanted was AltaVista. This has been the inevitable direction the whole time.
The real problem here is assuming this takes off what incentives will anyone have to provide the information to feed the beast?
maybewhenthesun 13 minutes ago [-]
I strongly disagree. Altavista had exactly the same function as google, but with worse results. Both linked to original sources. Early google had a very good idea with pagerank and that payed off.
An llm rephrashing / regurgitating other websites is imo different, because you loose the direct connection to the original source. Even if llms give sources they also directly give you a plausible (but unreliable) answer to your question. They are right often enough that you get lulled in to the false sense of security of not needing to read the original sites. I'd much prefer them to just give a clean list of sources like early google, but then why would you need an llm.
It's a pity that probably the main reason you'll need an llm to find anything on the web is to weed out all the llm-generated low quality garbage.
torben-friis 32 minutes ago [-]
>Objecting to this from the user end seems a bit like complaining the original Google was trying to be too magic when what you wanted was AltaVista. This has been the inevitable direction the whole time.
Did AltaVista get replaced by the owner of the site to justify a giant investment?
dawnerd 33 minutes ago [-]
Exactly, why should sites give free bandwidth to the google bots hammering them for nothing in return? Outside of retail, there's no point in allowing google to crawl if you're not getting anything in return.
microtonal 36 minutes ago [-]
Also, as a user, I want websites written by real humans. I do not want generic LLM output always has the same boring style. I like human writing, perfectly native English, broken second language English, I don't care. Human writing is unique and makes reading a pleasure.
Of course, even Google the search engine has gotten worse at surfacing interesting websites. First came the SEO spam websites, now the slop websites.
I'm glad that alternatives like Kagi exist.
ignu 2 minutes ago [-]
I used to use DuckDuckGo out of protest, despite it being inferior, but sometime in the last year (between general improvements and Google's rapid enshitification) it started outperforming Google for me.
skywhopper 29 minutes ago [-]
?? Google search results were in exactly the same format as Altavista results, only they weren’t filled with spammy nonsense.
Now, the spam is back and it’s coming from Google itself.
paxys 14 minutes ago [-]
The hardest decision a company, especially in tech, can make is to disrupt an immensely successul business of their own before their competitors can. Apple killed their biggest cash cow, the iPod, to push a smartphone. Netflix killed its entire business of DVD rentals in favor of streaming. Microsoft stopped selling software in boxes and pivoted to SaaS. Similar to all of these the business of typing words in a search box and getting 10 blue links was dead the moment ChatGPT got popular.
cdrnsf 12 minutes ago [-]
I haven't missed it since switching to Kagi.
embedding-shape 49 minutes ago [-]
Basically people who want to search, will now not be able to, they'll be forced into a UI they might have consciously avoided, otherwise they'd be using their chatbot in the first place. Seems like a strange UX decision, rather than recommending "Hey maybe you want to try our chatbot", they just force the user into a chat straight up.
billyp-rva 1 minutes ago [-]
5 years later...
People who wanted to ask a specific question now won't have that option. Instead, they'll simply be shown whatever Google things is most relevant to them at that moment. The "Chat" UI we've grown so accustomed to is on its way out.
ivraatiems 46 minutes ago [-]
That's the whole shape of AI for consumer-facing functions like this. It's not superior to the previous experience, but huge sunk costs and a misguided belief it's the "next thing" is leading companies to try to force the issue. It's the Apple removing the headphone jack of the modern Internet. A change for the worse that we'll all have to find ways to work around.
munk-a 41 minutes ago [-]
Google famously dragged on development of Glass for more than a decade stubbornly failing to admit that nobody wants to look like a cyborg the entire time only to be swept aside by Meta when they built a device that was glasses first with some recording and interactions built in.
If their leadership has an itch they'll scratch it until it's raw.
wslh 34 minutes ago [-]
Beyond the AI expenses, prompting captures more information from the consumer that keyword search. I assume they can take a lot of advantage from this and a new generation of ad engines is near the corner.
Calazon 41 minutes ago [-]
I read some of the article and skimmed the rest, and didn't see anything about old-fashioned search no longer being an option.
Is the idea that by making the new AI chat UX the default, that's how they're forcing people into it and making them not able to search? Or is there something I'm missing?
embedding-shape 15 minutes ago [-]
Second paragraph:
> Instead of returning a simple list of links, Google Search will drop users into AI-powered interactive experiences at times.
So basically you'd get redirect into a chatbot interface, rather than letting you browse search results as normal, "AI-powered interactive experience" tends to be euphemism for chatbot UIs, is my experience at least.
coldpie 46 minutes ago [-]
Makes sense to me. A chat UI has more avenues for subtle advertising & sentiment manipulation than plain links do.
TitaRusell 17 minutes ago [-]
Yeah just giving you the information/solution doesn't pay the bills. It's why supermarkets frequently change the layout of their aisles.
Never give the customers what they want give them what makes you money.
munk-a 39 minutes ago [-]
It is weird that they're putting all their eggs in one basket though. Wouldn't a more sensible business move be to launch that platform (and advertise it heavily) but maintain the old search UX to fend off competition?
Going all in like this carries a very real risk of burning users onto other platforms and the continued evolution of integrated search bars are already slicing off significant user segments.
threetonesun 26 minutes ago [-]
I assume they internally see the traffic they are losing to ChatGPT and see this as the best path forward. Or it's even more simple, and much like stacking sponsored links at the top of the results, they see that no one interacts with content below the AI response anyway.
Melatonic 16 minutes ago [-]
Kagi all the way
ivraatiems 49 minutes ago [-]
Kind of Google to create a market opening for its competitors like this. I hope Kagi, Bing, and DuckDuckGo are taking notes.
nostrademons 36 minutes ago [-]
The problem is that the web as we know it (useful, human-curated information that's put out there to help people) is also over. It's been totally overrun with AI slop. Even before AI could be used to create propaganda on a scale that we could only dream about 5 years ago, it's been declining under the weight of SEO sweatshops for a good 10 years. Meanwhile the actually decent content, the individual hobbyists who are just sharing their knowledge, have largely left under the weight of comment spam and DDoS attacks and doxxing.
So if another search engine does arise, it won't find anything useful, because the useful content on the web has been buried under slop, and largely removed. Your best bet today is a curated directory, sorta like the original Yahoo, where you allowlist the web to only real sites, download them, and make them searchable. I think this is actually Kagi's approach. But the open web as we knew and loved it is dead.
Zigurd 36 minutes ago [-]
I'm sure there's a niche for a product for search nerds. Something that leans into inverted indexes like the classic Lexis/Nexis search. But it's got to have Google-like coverage.
cortesoft 32 minutes ago [-]
Niche + Google-like coverage is not very economically viable. To store and update a search index of that size requires a lot of resources, and being niche means you don’t have a lot of resources.
Very few of the smaller search engines actually do their own indexing for exactly this reason.
torben-friis 43 minutes ago [-]
My literal first thought was "do I seriously need to use bing now?".
Supermancho 39 minutes ago [-]
Bing has been better than Google for some time. Again, it's embarrassing for them to sacrifice marketshare for paid results and an intermediate-form AI fad that will turn into the same paid result funnel.
mrweasel 26 minutes ago [-]
Bing is surprisingly not to bad. I don't use it anymore, but it's been providing better results than Google for sometime.
RyanOD 27 minutes ago [-]
I hear people cite other search engines as "better" all the time. Better how?
sphars 30 minutes ago [-]
There is a 99% chance (IMO) that Microsoft is going to go the same route as Google here
xerox13ster 24 minutes ago [-]
I've been using Startpage as my default search engine for a while now for any search where I actually need information and not sales or marketing bullshit.
When I use google, usually from my phone, I am reminded of why I don't use google on desktop.
With the announcement of this move by them, I just manually removed google as an address bar search engine option in all my browsers on desktop and mobile.
comrade1234 31 minutes ago [-]
I no longer use Google search for simple coding questions, even though it uses a bunch of Claude tokens to ask, for example, what's the null-safe operator in JavaScript vs ruby because it sends half my project with the question, I'll still just ask in my ide rather than a google search.
I caught myself yesterday starting to ask Claude in my ide what ship did grace and Rocky take back to Rocky's homeworld.
jerf 30 minutes ago [-]
Does the math math on this to be "free" for a long period of time? Ads can only pay for so much and AI can really suck down the money.
Ads have been close enough to covering costs for conventional internet search that even though I'm clearly the product and not the customer the relationship has still generally worked. If AI makes the "searching" 50 times more expensive, though, that could shift the relationship pretty badly in a direction of "if you're not paying for this you're not getting honest results". Paying may not sufficient for honesty but it may be necessary.
Honest question. But anyone who wants to answer this and who looks at Google's income/profit/revenue and is bedazzled by the size, don't forget to divide out by the number of Google's customers and ponder what that means. The per-user numbers are the much more relevant numbers and much less likely to cause Large Number Syndrome.
dweinus 18 minutes ago [-]
So to make this profitable they need ads revenue from it, right? Imagine for a moment the ways AI can manipulate responses and conversations for marketers, because I guarantee the marketers have already thought about it.
maybewhenthesun 23 minutes ago [-]
Google search has been over for a few years already.
Nearly all other search engines give better results with less annoying ads at the top. First thing I do when installing a new browser is switch the default search engine to duckduckgo. Duckduckgo's results are less good than google used to be, bu way better than google currently is.
notatoad 33 minutes ago [-]
>Google’s AI Overviews will also allow users to ask follow-up questions in AI Mode, beginning Tuesday, the company noted.
have i been A/B tested into something, or has this been live for months? this doens't seem new.
hnsr 28 minutes ago [-]
Yeah, I've also already seen this for at least ~2 months
theopsimist 24 minutes ago [-]
One good thing about the (current iteration of) AI era is it’s getting people used to paying directly for data.
Yeah, of course i’d prefer information to be totally free. But if that isn’t possible, paying directly is far superior to paying for it via ad exposure.
svieira 15 minutes ago [-]
The problem is that it amazingly easy to bias the weights and the actual size of the bias is tiny. So maintaining a per-user ad biased profile is cheap and profitable. I doubt that "paying directly" will keep out the ad men (after all, Cable TV cost money. Netflix too. Both have ads.)
thevillagechief 41 minutes ago [-]
I understand the consternation here about this change. And I've noticed recently getting frustrated because I'm looking for a search list but the UI throws me into AI mode first. But the think is I use traditional search so much less now that those annoyances are the exception. I can't say whether they are making a mistake, but they've got to have extensive data, and I'm going to bet that an overwhelming amount of people don't click through to the search results anymore for most quick queries. They probably really don't have a choice if they are going to effectively keep ChatGPT at bay. Of course, all this is terrible for the internet. That headline should have been: The Internet as you know it is over.
CrzyLngPwd 29 minutes ago [-]
I imagine that they have made this decision based on the search queries people use, and now have the compute to make better sense of them.
We'll see if it works. I use chatgpt for complex queries, and for throaway ones I use just don't log in to it.
I wouldn't use google for the same queries, since I normally use google to find specific things, not for a chatbot.
neilv 28 minutes ago [-]
Often, if you visit a few of the top PageRank-ish search hits for a query, you can find where the "AI" answer was mostly plagiarized from...
(For example, a random Redditor once said something, and the AI repeats it confidently and authoritatively, as if it is universal truth widely accepted by experts and applicable to the query.)
KevinMS 29 minutes ago [-]
Its becoming like a parasite killing its host
marcosdumay 16 minutes ago [-]
That patterns seems to be repeating in every company that invested in making an LLM. Google was the last exception.
zarzavat 42 minutes ago [-]
I haven't used Google search for years. It's almost totally irrelevant at this point and existing on pure inertia.
I'm aware that most people still use it, but it's nothing like the glory days when Google was far ahead of the pack.
RyanOD 25 minutes ago [-]
How can you say Google search is "totally irrelevant" and follow that with "I'm aware that most people still use it"?
aquir 40 minutes ago [-]
Time to pay for Kagi everyone!
loehnsberg 22 minutes ago [-]
Time for Kagi MCP to become available to subscribers!
Internet search should remain internet search. If I want to use AI, it should be an option, not a replacement of internet search.
Time to switch to old style search engines which still return the 10 blue links, with an AI option.
victorkulla 29 minutes ago [-]
Even Yandex from Russia is a better search engine. But I am yet to come across a truly powerful, fair and accurate search engine.
Melatonic 15 minutes ago [-]
Kagi !
oidar 30 minutes ago [-]
On the upside, perhaps the LLM will understand the intent of search operators now.
hootz 39 minutes ago [-]
That's why Kagi is the only subscription I don't actively think about cancelling. For the love of god, keep me away from Google and all of THAT. If Kagi goes down the same path, I'll selfhost something or just return to monkey and use link indexes and the favorites list + the native search of websites.
mrweasel 21 minutes ago [-]
The boss man got a few of us Kagi gift-subscriptions/credits earlier this year, after we've been taking about wanting to try it. Before that I used Ecosia, which I also considered pretty good, but Kagi and everything else it just night and day.
I've been pretty sceptical about Kagi, feeling that it was a bit to expensive and perhaps just relying on other companies indexes to much and I spend to much time looking at how many searches I had left. After getting the subscription I just don't want to go back, the price is perfectly reasonable for the value. Being able to just search again and not sort through junk and spam and ads and just getting the pages I want and need is amazing.
Honestly it's a slightly weird feeling to look a the results from Kagi and notice it found exactly what you where looking for.
Once my gifted credits run out, that is going to be an easy renewal for me. I do not want to go back, even if I think Ecosia is a good option.
hootz 4 minutes ago [-]
It's amazing how clear the manipulation and enshittification of Google's results are when you search the same thing with Kagi or even just another random search engine. Ecosia seems cool too, will keep an eye on it in case anything happens with Kagi.
charles_f 36 minutes ago [-]
Self hosting a web search engine is probably quite a feat
nostrademons 13 minutes ago [-]
It's actually not that hard now, once you get useful content. When I worked on Search (~2009ish), the primary index was called 4BBase, because it was the top 4 billion webpages (actually more like 5.5B during my time, but it had been around for a few years). A typical webpage is about 100K, and HTML compresses at 80-90% compression rates, so you're looking at 10-20K/page. The index would take about 50-100 TB.
Even after the recent AI run-up, disk prices are about $20/TB for a 20TB, so you can store this index on 3-5 hard disks that will cost you about $1200-2000. For self-hosted use you don't need to serve them in 50ms, so you don't need to put the whole thing in RAM like Google did, you can serve off of disk.
ElasticSearch uses basically the same data structures and gives you the same infrastructure that Google's ~late-00s search stack did, and is actually more advanced in some respects (like ad-hoc queries, debuggability, and updateability), so software isn't much of an issue.
The big part missing that can't really be replicated today is the huge web of authentic hyperlinks. The reason Google was so good at search was because many humans effectively "tagged" a given webpage with a series of short, descriptive words and phrases. When they went to search for a page, Google could mine this huge treasure trove of backlinks to identify exactly what the page was good for, even if those search terms never appeared on the page. SEO and link farms kinda killed this, as did the rise of social media walled gardens, and so the Google of 2009 basically wouldn't work today anyway. Maybe if you pulled old versions of Common Crawl or archive.org you could reconstruct it, but the relevant pages are often offline anyway today.
hootz 31 minutes ago [-]
I believe it is a thing. Saw it somewhere, like a peer to peer search engine.
adam12 28 minutes ago [-]
Google thinks they can do what Microsoft failed at.
bdangubic 24 minutes ago [-]
not a high bar to pass, google can (and did and does) a lot of stuff microsoft failed at
hyperhello 46 minutes ago [-]
Every organization eventually is taken over by the people who operate within it effectively, to the detriment of the people who operate outside and provide the actual public value. Google’s making a terrible, though understandable, mistake. They think people go to Google to see what Google wants to show them. This is like the people who run the airport imagining that travelers are popping by to see the decorations.
They are surely hearing themselves say the same things about how Google is “everything in one place” that every failed corporation parrots on their way out.
reed1234 45 minutes ago [-]
Google is trying to change that though. Ie a Mini Empire State Building in JFK
cynicalsecurity 25 minutes ago [-]
> They think people go to Google to see what Google wants to show them.
They are making the same mistake as Yahoo did. Ironic.
einrealist 33 minutes ago [-]
So good SEO will require prompt injection now?
Havoc 43 minutes ago [-]
Initially I thought AI would would crush google search, but starting to think the opposite. Think they have survived the transition.
After I got tired of perplexity's nonsense I realized the workspace account (which I have for custom email domain) came with fancy gemini pro chat.
Was a fucking ripoff for the domain thing...but domain plus premium chat clearly marked as "we won't train on your data"...the math starts mathing better again.
tdiff 29 minutes ago [-]
I think perplexity implements the same. Ive been using it as a default search for a month and actually still find myself explicitly using Google instead.
The ai generated summaries are slow, often miss the point of question and seem to be focused on user engagement, not in giving set of infos to sort out myself.
So there are two different types of queries, and when I want llm's answer, I ask chatgpt directly.
ReptileMan 37 minutes ago [-]
Google search has been dead for years.
What we need now is back to the roots - just a simple grep for the internet augmented by pagerank and eventually some sort of ai and harness to sort the rubish out. The AI companies have the data and the harnesses.
Google killed themselves when they made sure you can't search direct quotes or outside of your region. If I am going to sort trough vague crap - it is better AI to do it. And AI doesn't look at ads.
There is real opening for a company that just crawls and gives access to other companies to build on top of the collected stuff.
paulnpace 38 minutes ago [-]
I did not start using Google because the results were better.
I started using Google because the interface was far superior in the time before adblocking existed and after Flash existed.
Search results were better because they did not contain hidden paid results.
Search was measurably improved with the second generation of Wikipedia. Google did an excellent job understanding this and tended to just place the Wikipedia article at the top. Also helpful for Google was that Wikipedia's original search engine was useless, similar for YouTube whenever it came around.
Today, I use Google less than once per month. I'm not sure I've been there at all this year. Maybe at the end of last year I was using it and found nothing better than I found on other search engines.
hsuduebc2 40 minutes ago [-]
Finally google search result ridden with ads and useless results will be replaced by chatbot answers also ridden with ads, unnecessary commenatry from the bot and ads.
CooCooCaCha 41 minutes ago [-]
I think this will be one of those things that the hacker news crowd lambasts and calls a mistake but will either be neutral or seen as a positive to your average user.
munk-a 37 minutes ago [-]
I think you underestimate how many users loathe that "Generated with AI" box. But for me the bigger question is why they're going all in on this instead of a gradual rollout or a new tool offering.
CooCooCaCha 28 minutes ago [-]
I think people are sick of hearing about AI but they’ll embrace this change for the simple reason that they hate computers and want to feel like they’re taking to a human.
kakapo5672 22 minutes ago [-]
Yeh, my sense as well. I'm just out of college, and can tell you people my age use AI all the time and will probably be happy with this change. There is a diehard anti-AI group, but it seemed smaller all the time over the past couple years.
Bolwin 35 minutes ago [-]
The average user may be fine with it (though I think many will not). But given this is basically killing the open web, I don't see where Google plans to get the results to feed this AI thing in a year or so
adamiscool8 37 minutes ago [-]
Agree, but the average user trusts the AI knowledge box as expert summary, even though clicking through often reveals contrary information, so this is going to be a net negative overall for a while…
bigstrat2003 8 minutes ago [-]
The average user makes fun of how bad the Google AI generated responses are. I somehow don't think they're going to embrace a plan for that slop to be the only thing available.
whalesalad 41 minutes ago [-]
It's been over for years. I switched to Kagi during the pandemic and haven't looked back.
andrewstuart 42 minutes ago [-]
There is a lot at stake for Google - that search box has firehosed cash non stop into the company money bin for decades.
caspper69 46 minutes ago [-]
It's been over for years. Google scares companies into bidding against each other just to be seen. It's a complete farce & a racket. It's the pay to play web.
cynicalsecurity 33 minutes ago [-]
Google has become exclusively an advertising company long time ago, it's stopped being a search engine since years.
"Did you mean?" + excluded word was a pretty clear indication they stopped caring to provide any meaningful search whatsoever.
sublinear 35 minutes ago [-]
While I can certainly see this upsetting some people, I'm not sure if this is necessarily "bad".
Web 2.0 was Yahoo Pipes, public APIs, IFTTT, etc. while this new "Web 3.0" acknowledges that those capabilities would rather be gatekept behind AI instead of entirely removed.
At the very least we do get some of that functionality back without resorting to scraping anymore and it's now accessible to the layperson. I would think this would nudge the layperson to demand more and inevitably want the actual data without the training wheels or sandboxes. Is that not a "good" thing?
Is the pushback against this out of genuine concern or just ideological?
LogicFailsMe 41 minutes ago [-]
Slop as a Service (SaaS)...
tonymet 26 minutes ago [-]
Has the web been a meaningful experience since 2016? Before LLMs you might have visited 5 websites daily (besides utilities like banking / shopping /bills). Google concentrated on a handful of garbage-tier regime publishers with spammy ads. There were some holdouts like stack exchange and Wikipedia (at least attempting to produce quality content).
I think we can concede the WWW vision of distributed libertarian publishing has been dead for a long time. LLMs were just the final straw.
We ended up concentrating syndication on a few media companies like Google, Social Media companies.
Look at the profit margins of advertising companies vs producers and you’ll get an idea as to why.
expedition32 23 minutes ago [-]
The entire internet as I knew it is over. Everything trips Cloudflare and capcha's because of tech bros and their AI crusade.
But at least I've experienced the golden age. I feel bad for all the kids who will never know what once was.
ulrashida 39 minutes ago [-]
Cool. I hope this blows up in their face and is reverted in a few months. I don't need my phone book index to suddenly not be an index and force me to use a call center instead.
This was an interesting dilemma because it was very clear that the money was way less than the loss in ad revenue due to traffic drop, but it was also clear that if we wouldn’t take the deal, a more desperate competitor would, which would result in the same traffic loss but without the extra google money. So the company took the deal.
History repeats itself here, with the difference that instead of paying for the data, the ai crawlers simply take it for free.
As far as I know, you don't have a choice. They have no obligation to respect your wishes, and LLMs are allowed to scrape & republish your content.
Websites may go back to being simply labors of love.
Mention
Mechanisms might exist to make you think you have one, the same way copywrite should prevent millions of books being gobbled up by TheZuck but ultimately do you really have a choice?
Rules and laws don't exists for you.
Websites will die on the vine if LLMs intermediate all the content.
The "website" of the future will be an API optimized for LLM crawlers, serving plain-text content that no end-user will ever view directly. The SEO game will change to LLMAO.
While there are times where I want pure search (Kagi, Old Google) I mostly use LLMs to search now and have them provide me links for source data.
When I do use LLMs as a search engine I always want it integrated into my AI workflows with access to tools and scripts etc. I never want to have a conversation with a website that is geared towards advertising me products.
Currently, search engines are pretty bad at the second one because people try to use them as the first one
I was once very good at advanced Google search queries but they seem to no longer respect such queries - either showing irrelevant results or none at all (that should exist).
I don’t love LLMs, but they seem to not make up stuff very often these days and usually cite links to what they summarized. Sometimes the tone of the summary is slightly wrong “algorithm X was designed for Y” (when I know it wasn’t) but it’s otherwise very close to the mark.
What does amaze me, is the LLM seems to “understand” my question with very little context — I would have to give a human many more details about goals/intent.
I know damn LLMs are not capable of thought and are just a glorified search engine, but they do it well. Perhaps all my education made me little more…
I used to mock Sci-Fi movies where characters lazily dictated questions to the computer and it gave high quality answers.
We’re living in that world now.
An increasing number of studies are indicating a reliance on "AI" leads to deleterious cognitive effects. I felt this acutely myself.
I've noticed a significant boost to my recall since shunning "AI" as much as possible.
Though I will say I get much better results from the LLMs I pay for than the free ones with Google or DuckDuckGo, which seem to be way way way more prone to just make crap up based on your search and cite web pages that, when followed, don't have the claim being made in the AI search results at all. By contrast every "source" link I've followed in the for-money AIs has 100% backed what the AI said it backed. Don't judge by the free AIs the search engines put out, those things are probably starved of resources and are nearly useless.
(Which I did not intend as a commentary on Google's plans here, but it is a data point of interest... that pressure to cut costs on the "free" services is quite directly at odds with providing quality AI services for the forseeable future.)
And I’ve tried Google’s once or twice and seen it used once or twice, and used ChatGPT exactly once, last week, and I was not at all impressed by any of them. Their output, for what I’ve personally seen, has been nonsense, obvious, or unverifiable.
You can't do something like this with search.
Using google search, will return roughly infinite recipe sites. The sites were generated to spam AI generated recipes surrounded by advertisements. None of them are really any good because they were generated by a script and not looked at by a human until I come along and click. The standard is for all recipes to have at least 10-15 screenfulls of vertical spam wrapped by ads for recipe pages. The internet, at least using Search, is now useless for food recipes. I would have better, faster luck driving to the public library and looking in a physical cookbook; at least those recipes were probably tested at least once by humans unlike the advertising spam sites. Nobody has 45 minutes to watch 44 minutes of filler material surrounded by ads on Youtube either. If you want to cook food, the internet is near dead at this time, unfortunately.
AI search will plagiarize the "Original Nestle Toll House" recipe from the back of every bag of chocolate chips ever made. Its a good recipe and I've baked them many times over the decades.
I wish the internet were more useful, but the people in charge of it don't want it to be useful; here have some ragebait and doomscroll while watching the ads.
I've been trying to use LLMs for things and it makes mistakes all the time. Just this week i had multiple instances of various LLMs basically saying "just run the software with --flag-that-fixes-your-problem" or "edit the config and add solve-your-issue=true" hallucinating non-existant options. Even if i manually link the relevant documentation pages it will still just make basic mistakes. and if im having to read the documentation myself anyway to fix the AI's mistakes, why is the AI even in the loop.
its infecting search too, because blogspam/slop articles are managing to make their way into search results by just making up untrue information, claiming software can do things it cant, or has options that don't exist.
yeah man good thing LLMs are structurally incapable of being incentivized to sell you a product or render referral links, this is surely future-proof
Yeah, they probably aren't doing (most of) these now, but it doesn't take much mental energy to extrapolate once you factor nearly every other tech company's ethical trajectory and the current geopolitical environment. Substituting classic search entirely with LLMs is not a savvy move.
As soon as one gets annoying, expensive, advertiser heavy etc. you just rip it out and replace it with the other one. AFAICT there is zero lock-in or moat. I often am able to switch models in one click or command. This is why all the LLM providers are desperate for a product layer/comprehensive tool set.
Sure maybe they all end up that way, but there’s plenty of reasons corporate customers will want private LLM usage that is not skewed towards advertising. I am happy to pay for that.
Also, open source models are a bulwark against another search style ad Monopoly.
Since this is how Google makes all their money, why are they killing it off? Do they think people will eventually pay for LLM search? Do they plan to stuff the results with ads, not even sharing the ad revenue with the content sources?
The advertisements fed the content, which fed the AI, which in turn feeds your AI workflows. AI is still not trusted unless it's output is grounded with sources.
But I still want to also be able to do my normal, old school searching.
I already saw a article recently about how to set up a business domain which can reliably show up in a search result and dump overly positive reviews into anyone's context.
My experience with AI searches is that they'll still be wrong a lot of times, but it will condense/flatten the content generating trash sites and give me alternatives from these deeper results. What I'm looking for is usally in there.
Even though the result is often good and combines information from multiple sources, it can also get things wrong by combining information from different eras or just plain outdated advice. AFAICT, without primary sources, the result is for entertainment purposes only.
I worry that the LLMs are just the equivalent of a ‘lagging indicator’ of web quality though - that they will also soon be overwhelmed with the sheer volume of plausible nonsense that is the web now, just like search engines are.
Model collapse everywhere.
The other bots either make up links or simply don't provide any information that is distinguishable from the LLM predictive output.
Ironically Gemini is also very bad at this, while it should have been the best at Web search.
Gemini also does something very patchy, which is to provide "links" which are in fact GET queries into classic Google search. I'm guessing they did it this way because the links generated/hallucinated by the LLM were too unreliable.
Have you tried explicitly asking for links to primary sources?
The real problem here is assuming this takes off what incentives will anyone have to provide the information to feed the beast?
An llm rephrashing / regurgitating other websites is imo different, because you loose the direct connection to the original source. Even if llms give sources they also directly give you a plausible (but unreliable) answer to your question. They are right often enough that you get lulled in to the false sense of security of not needing to read the original sites. I'd much prefer them to just give a clean list of sources like early google, but then why would you need an llm.
It's a pity that probably the main reason you'll need an llm to find anything on the web is to weed out all the llm-generated low quality garbage.
Did AltaVista get replaced by the owner of the site to justify a giant investment?
Of course, even Google the search engine has gotten worse at surfacing interesting websites. First came the SEO spam websites, now the slop websites.
I'm glad that alternatives like Kagi exist.
Now, the spam is back and it’s coming from Google itself.
People who wanted to ask a specific question now won't have that option. Instead, they'll simply be shown whatever Google things is most relevant to them at that moment. The "Chat" UI we've grown so accustomed to is on its way out.
If their leadership has an itch they'll scratch it until it's raw.
Is the idea that by making the new AI chat UX the default, that's how they're forcing people into it and making them not able to search? Or is there something I'm missing?
> Instead of returning a simple list of links, Google Search will drop users into AI-powered interactive experiences at times.
So basically you'd get redirect into a chatbot interface, rather than letting you browse search results as normal, "AI-powered interactive experience" tends to be euphemism for chatbot UIs, is my experience at least.
Never give the customers what they want give them what makes you money.
Going all in like this carries a very real risk of burning users onto other platforms and the continued evolution of integrated search bars are already slicing off significant user segments.
So if another search engine does arise, it won't find anything useful, because the useful content on the web has been buried under slop, and largely removed. Your best bet today is a curated directory, sorta like the original Yahoo, where you allowlist the web to only real sites, download them, and make them searchable. I think this is actually Kagi's approach. But the open web as we knew and loved it is dead.
Very few of the smaller search engines actually do their own indexing for exactly this reason.
When I use google, usually from my phone, I am reminded of why I don't use google on desktop.
With the announcement of this move by them, I just manually removed google as an address bar search engine option in all my browsers on desktop and mobile.
I caught myself yesterday starting to ask Claude in my ide what ship did grace and Rocky take back to Rocky's homeworld.
Ads have been close enough to covering costs for conventional internet search that even though I'm clearly the product and not the customer the relationship has still generally worked. If AI makes the "searching" 50 times more expensive, though, that could shift the relationship pretty badly in a direction of "if you're not paying for this you're not getting honest results". Paying may not sufficient for honesty but it may be necessary.
Honest question. But anyone who wants to answer this and who looks at Google's income/profit/revenue and is bedazzled by the size, don't forget to divide out by the number of Google's customers and ponder what that means. The per-user numbers are the much more relevant numbers and much less likely to cause Large Number Syndrome.
Nearly all other search engines give better results with less annoying ads at the top. First thing I do when installing a new browser is switch the default search engine to duckduckgo. Duckduckgo's results are less good than google used to be, bu way better than google currently is.
have i been A/B tested into something, or has this been live for months? this doens't seem new.
We'll see if it works. I use chatgpt for complex queries, and for throaway ones I use just don't log in to it.
I wouldn't use google for the same queries, since I normally use google to find specific things, not for a chatbot.
(For example, a random Redditor once said something, and the AI repeats it confidently and authoritatively, as if it is universal truth widely accepted by experts and applicable to the query.)
I'm aware that most people still use it, but it's nothing like the glory days when Google was far ahead of the pack.
[1] https://github.com/kagisearch/kagimcp
Time to switch to old style search engines which still return the 10 blue links, with an AI option.
I've been pretty sceptical about Kagi, feeling that it was a bit to expensive and perhaps just relying on other companies indexes to much and I spend to much time looking at how many searches I had left. After getting the subscription I just don't want to go back, the price is perfectly reasonable for the value. Being able to just search again and not sort through junk and spam and ads and just getting the pages I want and need is amazing.
Honestly it's a slightly weird feeling to look a the results from Kagi and notice it found exactly what you where looking for.
Once my gifted credits run out, that is going to be an easy renewal for me. I do not want to go back, even if I think Ecosia is a good option.
Even after the recent AI run-up, disk prices are about $20/TB for a 20TB, so you can store this index on 3-5 hard disks that will cost you about $1200-2000. For self-hosted use you don't need to serve them in 50ms, so you don't need to put the whole thing in RAM like Google did, you can serve off of disk.
ElasticSearch uses basically the same data structures and gives you the same infrastructure that Google's ~late-00s search stack did, and is actually more advanced in some respects (like ad-hoc queries, debuggability, and updateability), so software isn't much of an issue.
The big part missing that can't really be replicated today is the huge web of authentic hyperlinks. The reason Google was so good at search was because many humans effectively "tagged" a given webpage with a series of short, descriptive words and phrases. When they went to search for a page, Google could mine this huge treasure trove of backlinks to identify exactly what the page was good for, even if those search terms never appeared on the page. SEO and link farms kinda killed this, as did the rise of social media walled gardens, and so the Google of 2009 basically wouldn't work today anyway. Maybe if you pulled old versions of Common Crawl or archive.org you could reconstruct it, but the relevant pages are often offline anyway today.
They are surely hearing themselves say the same things about how Google is “everything in one place” that every failed corporation parrots on their way out.
They are making the same mistake as Yahoo did. Ironic.
After I got tired of perplexity's nonsense I realized the workspace account (which I have for custom email domain) came with fancy gemini pro chat.
Was a fucking ripoff for the domain thing...but domain plus premium chat clearly marked as "we won't train on your data"...the math starts mathing better again.
The ai generated summaries are slow, often miss the point of question and seem to be focused on user engagement, not in giving set of infos to sort out myself.
So there are two different types of queries, and when I want llm's answer, I ask chatgpt directly.
What we need now is back to the roots - just a simple grep for the internet augmented by pagerank and eventually some sort of ai and harness to sort the rubish out. The AI companies have the data and the harnesses.
Google killed themselves when they made sure you can't search direct quotes or outside of your region. If I am going to sort trough vague crap - it is better AI to do it. And AI doesn't look at ads.
There is real opening for a company that just crawls and gives access to other companies to build on top of the collected stuff.
I started using Google because the interface was far superior in the time before adblocking existed and after Flash existed.
Search results were better because they did not contain hidden paid results.
Search was measurably improved with the second generation of Wikipedia. Google did an excellent job understanding this and tended to just place the Wikipedia article at the top. Also helpful for Google was that Wikipedia's original search engine was useless, similar for YouTube whenever it came around.
Today, I use Google less than once per month. I'm not sure I've been there at all this year. Maybe at the end of last year I was using it and found nothing better than I found on other search engines.
"Did you mean?" + excluded word was a pretty clear indication they stopped caring to provide any meaningful search whatsoever.
Web 2.0 was Yahoo Pipes, public APIs, IFTTT, etc. while this new "Web 3.0" acknowledges that those capabilities would rather be gatekept behind AI instead of entirely removed.
At the very least we do get some of that functionality back without resorting to scraping anymore and it's now accessible to the layperson. I would think this would nudge the layperson to demand more and inevitably want the actual data without the training wheels or sandboxes. Is that not a "good" thing?
Is the pushback against this out of genuine concern or just ideological?
I think we can concede the WWW vision of distributed libertarian publishing has been dead for a long time. LLMs were just the final straw.
We ended up concentrating syndication on a few media companies like Google, Social Media companies.
Look at the profit margins of advertising companies vs producers and you’ll get an idea as to why.
But at least I've experienced the golden age. I feel bad for all the kids who will never know what once was.