Thread: Playing with ChatGPT AI

As that goes, I visited a mate yesterday that I had not spoken to in some time and his view of "AI" is formed by his experience which is quite dissimilar to mine. Funny how that works

For those not familiar with it yet, the abbreviations list:

• AI = Artificial Intelligence
• LLM = Large Langurage Model
• RLM = Reasoning Language Model

I basically see a couple of thoughts going around, in his experience (he works for local waste water treatment and as such is a government employee) AI can advice you at least as well if not better then any of the so called "advisor" titles he has at the office (of which he counted 150) and he claims they will all be out of a job in 10 years. He has also used an LLM (from what I can tell chatGPT) extensively in helping to swap a 3.0L 6I Mercedes engine into his G wagen. (manual, they never came with that) and has found it to be an invaluable help. I did ask him ever so gently WHERE he thought all that knowledge came from and that is where it stopped. He simply did not care (I'll go into this further on in this post)

Some others think, such as Tins, that "AI" will be used for high end stuff suffice it to say. Whilst there is some truth in that, the playing around of Dave in my previously posted URL shows this clearly, there is still a huge amount of human input that is needed that can not easily be... how do I put this? "intelligized" away. So, making very specific tools, massive pattern recognition applications, whatever like that is very feasible but it has been like that for decades. We have just gotten bigger and faster compute at our disposal. How long have we not had (unfortunately) license plate recognition in traffic, parking, etc? At least a few decades in my home turf.

Then there are those that think "AI" will do everything soon enough and that it will not stop learning and growing etc. Does not matter if you believe we are all going to be on the axiom like in wall-e or that you believe the T-800 from the terminator will be at your doorstep.

From what I can tell this is not possible with the current type of "AI" we are using today, ie. LLM's and RLM's (that's why I have been quoting AI thus far, it is not even close to AI). The problem is that LLM's are word generators, and not even that they generate tokens but let's forget that for a second. We try and make an algorithm that predicts the correct sequence of words, that's it. We train those models on a huge amount of data that has been scraped of the internet (again I will get into that soon) including all the "mis-information" that is around. But, we have reached the limits of human knowledge spread on the internet in models from ahtropic, openai and google. There is no more. We can see this in the "advances" of the last generation of models. GPT 5.2 is not the incredible leap we saw between 3 and 4 for example. Yes it has gotten more usable, but certainly not more "intelligent".

Then the resource situation. That is a tricky one. I run a forum that is at least as popular as this one I reckon but is not in English so has a far more local appeal and I have observed some shocking figures. I'll forego all the statistical breakdowns but it boils down to this: 95% of traffic on that forum is bots. Recently a MASSIVE spike of Chinese scrapers have been hammering the site down to its knees. I had a chat with Dave about this (since I could no longer visit aulro) and this forum experiences the same challenges. So, this is not just a problem of: we need a large datacentre with all that comes with it, it is ALSO a huge burden on the infrastructure of the web itself.

This brings us back nicely to the "make AI's intelligent" debate. The only ways we make LLM's more useful these days is by, of course, scraping new data (ie new posts from forums) but that has reached its limits as we already concluded. The only thing left is to make the "AI think" that is where RLM comes in, the tool takes the question and starts deliberating over it ie. there is a transformer that internally takes a different approach. Once that reasoning is done the output of that reasoning is fed back into another instance of the model and that is used to generate a response that is more complete shall we say. The same happens with "vibe coding" and a very good example of it is grok 4.20 (no I kid you not, it is called four-twenty), It has 4 different agents internally with 4 different purposes. One is logical, the other is an out of the box thinker, etc. These 4 "discuss" with each other and finally you get a response.

Can you see the pattern and the problem here? The amount of tokens generated keeps on flying up! Currently a token (which is generally accepted as being 4 letters) costs anywhere between 20ct per million up to 15 bucks per million. If you look at what the resources underneath cost... that's tricky since most companies are in the early stage of "lock in the market" so we get if dirt cheap. If you need to spend more and more tokens to get better answers (what we perceive as being more intelligent) we are burning through more and more resources. This line is not quite linear and it never reaches an equilibrium. So, I do not for one minute doubt that we will find better ways to do things, LLM's may very well be not the end-all be-all, but for now I expect incremental improvements only and not earth shattering new levels of "intelligence".

As per usual those with money benefit the most... I have done extensive testing on running local LLM's to help with my day-to-day IT job and the cost-benefit is not all that great, until I compare it with for example claude or gemini and what that costs per million tokens. Still, what that ask me in terms of money for pay-as-you-go use does not compare to the hardware I would have to buy to run it myself, and that is not even possible since they are not open models. Simple example: I used a fairly unknown french model devstral-2. It is 123B parameters (ie you need a 128gb videocard so to speak to simplify it). I only paid for tokens used and not a monthly fee and I did 2 days of 3 hours of work with the model (so called vibe coding). The model got stuck once or twice and at those times I took the specific problem (ie. I used REAL intelligence) and put that into chatgpt and used its answer to feed back to devstral. All in all 6 hours of work that kind of represents a "normal" 8 hour working day since one has meetings, gets coffee, etc. I spent around 10 AUD. If I were a developer that writes coded all day long, I would use WAY more. I have already seen reports of people burning through the highest subscriptions they can get and those who PAYG can easily reach 600-1000 USD a month. That's fine if you make enough money doing it, but not everyone will which means you need to use a cheaper model, etc, etc.

Well, enough ranting for today

Cheers,
-P

Originally Posted by Tins

I'm convinced AI has a huge place going forward. Especially in crunching numbers in research programs, and things far more complex that take massive amounts of time for people to do. Not sure if it's valuable for the common folk though. A fast goggle search is still a goggle search, and people will take the first results of these as gospel.

And then there's the resources. AI data centres gobble huge amounts of electricity and water to run, which is a massive burden on infrastructure that we are already killing chasing unicorns. How much of that is used at a higher level, and how much of it is wasted making idiotic youtube videos that are forgotten almost as soon as they are published?

At the consumer level, I reckon  is onto something, keeping it on device. This is what the whole A/M chip strategy has been about. Nobody needs M5 kind of power to browse the internet.

"Gigawatt sized Era of AI infrastructure

"

OS its HUGE



Here its HUGE


Noting a Huge dummy spit in the US "Anthropic a Supply-Chain Risk to National Security" Its the only one with Security classified access and heavily integrated in the US Military. Wish them the best of luck trying to de-integrate