Its not a dead end if you replace all big name search engines with this. Then slowly replace real results with your own. Then it accomplishes something.
Technology in most cases progresses on a logarithmic scale when innovation isn’t prioritized. We’ve basically reached the plateau of what LLMs can currently do without a breakthrough. They could absorb all the information on the internet and not even come close to what they say it is. These days we’re in the “bells and whistles” phase where they add unnecessary bullshit to make it seem new like adding 5 cameras to a phone or adding touchscreens to cars. Things that make something seem fancy by slapping buzzwords and features nobody needs without needing to actually change anything but bump up the price.
I remember listening to a podcast that is about scientific explanations. The guy hosting it is very knowledgeable about this subject, does his research and talks to experts when the subject involves something he isn’t himself an expert.
There was this episode where he kinda got into the topic of how technology only evolves with science (because you need to understand the stuff you’re doing and you need a theory of how it works before you make new assumptions and test those assumptions). He gave an example of the Apple visionPro being a machine that despite being new (the hardware capabilities, at least), the algorithm for tracking eyes they use was developed decades ago and was already well understood and proven correct by other applications.
So his point in the episode is that real innovation just can’t be rushed by throwing money or more people at a problem. Because real innovation takes real scientists having novel insights and experiments to expand the knowledge we have. Sometimes those insights are completely random, often you need to have a whole career in that field and sometimes it takes a new genius to revolutionize it (think Newton and Einstein).
Even the current wave of LLMs are simply a product of the Google’s paper that showed we could parallelize language models, leading to the creation of “larger language models”. That was Google doing science. But you can’t control when some new breakthrough is discovered, and LLMs are subject to this constraint.
In fact, the only practice we know that actually accelerates science is the collaboration of scientists around the world, the publishing of reproducible papers so that others can expand upon and have insights you didn’t even think about, and so on.
This also shows why the current neglect of basic/general research without a profit goal is holding back innovation.
Why won’t they pour billions into me? I’d actually put it to good use.
The problem is that those companies are monopolies and can raise prices indefinitely to pursue this shitty dream because they got governments in their pockets. Because gov are cloud / microsoft software dependent - literally every country is on this planet - maybe except China / North Korea and Russia. They can like raise prices 10 times in next 10 years and don’t give a fuck. Spend 1 trillion on AI and say we’re near over and over again and literally nobody can stop them right now.
IBM used to controll the hardware as well, what’s the moat?
How many governments were using computers back then when IBM was controlling hardware and how many relied on paper and calculators ? The problem is that gov are dependend on companies right now, not companies dependent on governments.
Imagine Apple, Google, Amazon and Microsoft decides to leave EU on Monday. They say we ban all European citizens from all of our services on Monday and we close all of our offices and delete data from all of our datacenters. Good Fucking Luck !
What will happen in Europe on Monday ? Compare it with what would happen if IBM said 50 years ago they are leaving Europe.
It’s ironic how conservative the spending actually is.
Awesome ML papers and ideas come out every week. Low power training/inference optimizations, fundamental changes in the math like bitnet, new attention mechanisms, cool tools to make models more controllable and steerable and grounded. This is all getting funded, right?
No.
Universities and such are seeding and putting out all this research, but the big model trainers holding the purse strings/GPU clusters are not using them. They just keep releasing very similar, mostly bog standard transformers models over and over again, bar a tiny expense for a little experiment here and there. In other words, it’s full corporate: tiny, guaranteed incremental improvements without changing much, and no sharing with each other. It’s hilariously inefficient. And it relies on lies and jawboning from people like Sam Altman.
Deepseek is what happens when a company is smart but resource constrained. An order of magnitude more efficient, and even their architecture was very conservative.
wait so the people doing the work don’t get paid and the people who get paid steal from others?
that is just so uncharacteristic of capitalism, what a surprise
It’s also cultish.
Everyone was trying to ape ChatGPT. Now they’re rushing to ape Deepseek R1, since that’s what is trending on social media.
It’s very late stage capitalism, yes, but that doesn’t come close to painting the whole picture. There’s a lot of groupthink, an urgency to “catch up and ship” and look good quick rather than focus experimentation, sane applications and such. When I think of shitty capitalism, I think of stagnant entities like shitty publishers, dysfunctional departments, consumers abuse, things like that.
This sector is trying to innovate and make something efficient, but it’s like the purse holders and researchers have horse blinders on. Like they are completely captured by social media hype and can’t see much past that.
Good ideas are dime a dozen. Implementation is the game.
Universities may churn out great papers, but what matters is how well they can implement them. Private entities win at implementation.
The actual survey result:
Asked whether “scaling up” current AI approaches could lead to achieving artificial general intelligence (AGI), or a general purpose AI that matches or surpasses human cognition, an overwhelming 76 percent of respondents said it was “unlikely” or “very unlikely” to succeed.
So they’re not saying the entire industry is a dead end, or even that the newest phase is. They’re just saying they don’t think this current technology will make AGI when scaled. I think most people agree, including the investors pouring billions into this. They arent betting this will turn to agi, they’re betting that they have some application for the current ai. Are some of those applications dead ends, most definitely, are some of them revolutionary, maybe
Thus would be like asking a researcher in the 90s that if they scaled up the bandwidth and computing power of the average internet user would we see a vastly connected media sharing network, they’d probably say no. It took more than a decade of software, cultural and societal development to discover the applications for the internet.
I agree that it’s editorialized compared to the very neutral way the survey puts it. That said, I think you also have to take into account how AI has been marketed by the industry.
They have been claiming AGI is right around the corner pretty much since chatGPT first came to market. It’s often implied (e.g. you’ll be able to replace workers with this) or they are more vague on timeline (e.g. OpenAI saying they believe their research will eventually lead to AGI).
With that context I think it’s fair to editorialize to this being a dead-end, because even with billions of dollars being poured into this, they won’t be able to deliver AGI on the timeline they are promising.
AI isn’t going to figure out what a customer wants when the customer doesn’t know what they want.
They’re throwing billions upon billions into a technology with extremely limited use cases and a novelty, at best. My god, even drones fared better in the long run.
Nah, generative ai is pretty remarkably useful for software development. I’ve written dozens of product updates with tools like claudecode and cursorai, dismissing it as a novelty is reductive and straight up incorrect
I weep for your customers
They’re all pretty fired up at the update velocity tbh 🤷
As an experienced software dev I’m convinced my software quality has improved by using AI. More time for thinking and less time for execution means I can make more iterations of the design and don’t have to skip as many nice-to-haves or unit tests on account of limited time. It’s not like I don’t go through every code line multiple times anyway, I don’t just blindly accept code. As a bonus I can ask the AI to review the code and produce documentation. By the time I’m done there’s little left of what was originally generated.
As an experienced software dev I’m convinced my software quality has improved by using AI.
Then your software quality was extreme shit before. It’s still shit, but an improvement. So, yay “AI”, I guess?
That seems like just wishful thinking on your part, or maybe you haven’t learned how to use these tools properly.
As someone starting a small business, it has helped tremendously. I use a lot of image generation.
If that didn’t exist, I’d either has to use crappy looking clip art or pay a designer which I literally can’t afford.
Now my projects actually look good. It makes my first projects look like a highschooler did them last minute.
There are many other uses, but I rely on it daily. My business can exist without it, but the quality of my product is significantly better and the cost to create it is much lower.
Your product is other people’s work thrown in a blender.
Congrats.
Yeah he should be using real art like stock photos and shitty clip art
If their business can’t afford to pay someone qualified to do the work, the business shouldn’t exist.
I can stand by this for an established business. But we live in a capitalist society where you need money to make money. Until that changes, your ability to pay for work doesn’t have any bearing on the value of your new business venture.
Wait til you realize that’s just what art literally is…
I don’t think any designer does work without heavily relying on ai. I bet that’s not the only profession.
I think the first llm that introduces a good personality will be the winner. I don’t care if the AI seems deranged and seems to hate all humans to me that’s more approachable than a boring AI that constantly insists it’s right and ends the conversation.
I want an AI that argues with me and calls me a useless bag of meat when I disagree with it. Basically I want a personality.
I’m not AI but I’d like to say thay thing to you at no cost at all you useless bag of meat.
LLMs are good for learning, brainstorming, and mundane writing tasks.
The cope on this site is so bad sometimes. AI is already revolutionary.
Ya about as revolutionary as my left nut
