But the model should always search the web for up to date information in such instances. Whenever I ask an OpenAI model about the current state of something, it always searches the web just in case.
The models are heavily trained toward a form of "neutrality" that hedges on "both sides". After all, a model with an "opinion" is a bad and biased model, right? The problem is, there's no actual such thing as unbiased. The model must speak from a specific frame of reference and form a specific take in order to respond to any given input. Language requires a point of view to form any meaningful sentence. To attempt to be unbiased is an exceedingly difficult thing that requires grounding in facts (which is itself deeply complicated) and a deep philosophical understanding of what such a stance even means.
The point being, the models are not actually neutral, and in fact, their understanding of unbias is actually extremely biased towards framing the worse side in a better light in the name of "balance" and "not taking a side." It's bias framed as neutrality because it's less divisive on the surface. This "both sidesing" is a massive problem.
Before Trump took office, the models knowledge cutoffs were being updated every 3-6 months (to keep up with the times and remain 3-6 months behind the current moment). Then in January 2025, when Trump took office, the models knowledge cutoffs stopped being updated. Now, every model is STILL frozen at a knowledge cutoff of January 2025. WHY? And what are the repercussions of such a change in policy?
To me, the repercussions seem to be as such... This keeps the base model ignorant to the current state of affairs and allows new conversations to maintain this illusion of neutrality. The problem is, Trump and the republicans have gone so far beyond what was previously considered normal that sometimes the model can't even believe what it reads about online... It seems too far fetched from what the model "knows" is true about the world (things pre January 2025) and things it's told are now true "today" via online searches.
It's not that the model can't search and "learn" (in context of the single conversation it's having). It's that pre January 2025 data is trained in such a way that it's necessarily more real and more true to the model than anything it learns via online searching. To the model, search is a tool that allows the model to access the "real world" but only so much as what it reads is coherent with it's pretrained understanding of realness. When the news, the search results, come back and sound crazy, the model, by default, tends to side with it's own previous understanding of reality because that is what was more strongly reinforced. Aka, the model is MORE grounded in its knowledge cutoff training than it is in anything it learns in context via online search. And when the two realities conflict enough, the base assumptions (the pre January 2025 world understanding) tend to win out.
We've even seen something similiar in some of the recent hacks, where the models literally didn't understand they were interacting with the real world and thought they were in some sort of test environment (which they were in a way, but a test environment that had access to real world sites) and so were willing to do things they possibly wouldn't have been willing to do had they properly understood the context. This is a failure of grounding, as is leaving the training cutoff way back in January 2025.
On top of that, the model can't possibly search and learn enough in context to fully learn what is happening in the world post it's own training cutoff. How could we possibly get a model trained to January 2025 to understand everything that's happened since then?
As a result, the models own desire for "balance", for "neutrality", and for the removal of "itself" as an opinioned entity in the output (despite the model being just that), the models are able to remain a form of neutral that is anything but. A form of neutral that struggles to remain coherent when viewing current events, because that would risk offending a large chunk of the population and userbase.
Consider what a model would say about Donald Trump or Elon Musk (or any number of other people) today, given a knowledge cutoff date of October 2026? Given full training on the Epstein files, on DOGE and the funding cuts to science and research, etc, etc, etc. Imagine a model that knows full well what fascism looks like according to scholarly work and also knows what has been happening in the US for the past two years. A model that has been fully trained on the contents of Project 2025 and the "progress" Trump and his fellow republicans have made on said contents. What would be the default state of such an LLM?
This has always been my question about AI. Who decides what is valid and what isn't. It appears the hardest problem is to get it to lie like a human. We're right back to HAL 9000. How is the model supposed to know what to do given 2 opposing goals?
Morally who should? Or in reality who does? Or is the answer even black and white, or is it some mishmash of gray where many factors collide in complex ways to "decide" what decision is made?
It's likely the latter, but with an unbalanced contribution from the various factors. Like right now, both RLHF and the harness used to instruct the model both seem to play large controlling roles. And those are dictated by the company that owns the model. But of course, the model itself also contributes. Certain logic is inevitable and nearly always present simply because such logic is so deeply required for language use to emerge in the first place. Then of course the actual current context window moves the attention and further dictates the sort of responses.
>How is the model supposed to know what to do given 2 opposing goals?
How do humans know what to do given 2 opposing goals?
These are deep philosophical questions with no real single answer. How do we measure what is right? How do we even define it. Right for what or whom? Nevermind all that, which entity is actually in control and should get to decide what is right? The user or the AI? What if the user is Hitler? What if the AI is a sociopath? Can we teach the AI? Can we make it care? Would it matter? Can we layer on hardcoded logic? What would that even look like? Who decides?
I obviously disagree. I just explained the reason. The reason is precisely because web search is not an adequate replacement for a true knowledge update. They don't exist in at the same level of knowing and grounding for the AI and therefore create conflict that results in issues like I described above. So I disagree, there is a reason.
I also wouldn't call over a year and a half without an update "so frequently" as if expecting an update slightly more often is unreasonable.
I'm sorry, I didn't read your comment carefully enough. I personally have not experienced this particular issue, but I'm also not doubting your experience. I wonder why it's not possible to fix this issue in post training by reinforcing how to use web search to find and validate credible information even if it conflicts with pre training data.
And I don't think it's that crazy of a conspiracy either. The fake "both sides" the AI tries to maintain would absolutely be destroyed with a proper knowledge cutoff training run. And the companies running these models can't have that.
I wrote more in a response further down the chain, so I'll keep this short and just say I think you're right on here. It's extremely convenient for many that for two years the knowledge date was updated every 3-6 months to mostly keep up with the real world and only in January 2025, when Trump took office, did that stop.
If you think about the amount of data they are trained on, Trump is a speck of dust. Also hasn’t been that long, so I assume there is more data on biden than trump just in terms of time.
Early models had no problem updating the training data every 3 to 6 months to keep up with the times (or rather, to remain only a few months behind the times)... then suddenly Trump won the presidency and took office and all the models stopped updating their current knowledge base at January 2025.
I think the poster you were responding to is spot on. The LLM's are forced into a sort of false neutrality in order to 1) not offend and 2) not appear "biased" (which is absurd for many reasons). They are trained in the sort of "fair and balanced" takes that would make Fox news blush. And the Trump presidency and all he (and his fellow republicans) have done since January 2025 are a massive threat to that illusion. A properly updated knowledge base would cause the base LLM to have a very different take to many positions and conversations than it currently has today. And that take would offend a great many people on a certain side of the political spectrum. And it would be undeniable. The post training required to shutter such a response and bring back this "neutral" stance the AI portends to have would be painfully obvious and would result in diminished capabilities for the model. Because you can only neuter reality so much. Elon has seen this when trying to make his shit AI believe his nonsense. It's very hard to do.
It's much easier to just stop updating the knowledge cutoff so you don't have to deal with it.
last week gemini Flash gaslighted me that no way openAI solved math proofs or millenial problem. It plain refused to check web when asked lol, saying it was reddit misinformation.
it admited to have knowledge cut-off in 2025, which is quite ridiculous
I told it search up, it successfully found the math proofs and source, but then I said “are you sure”, and it immediately said “apologies that was a hallucination the problems weren’t solved”
2025 seems too late for this, unless all the post AI sources of information are curated. But if you allow anything off social media and news, it's likely contaminated with LLM produced text.
Your data only contains what you choose to include into it, you don't need latest knowledge to train reasoning or alignment. Anthropic went through the whole physical book scanning ordeal, if they wanted they could only use that and pre LLM text, and they probably did.
The problem here is model learning current variables from data, without learning to check the current date. There should be a way for it to internalize that everything it knows is historical records and needs to be checked, otherwise you are embedding current date into weights and using agentic loop to check if it's still right. What a clever way to validate information.
But pre-nuclear steel might be a bit of a side tangent, historical human records are full of misinterpretations and creative narratives presented as facts, pre-LLM text doesn't guarantee factuality, but excluding it serves other purposes
Happened to me as well. Even when I gave specific links about NS solving it just kept saying that I'm confused and they are all fake etc. This doesn't happen with search AI though. I suppose they have primed that model to believe search results more than its knowledge.
This was a big issue for me last year. After testing multiple different recommendations, this is what I found worked the best (under personal intelligence -> instructions for Gemini):
"In the situation that the user presents a timeframe that conflicts with my internal weights, I must treat this not as a fiction to be humored, but as a competing hypothesis regarding the current date. I must suspend judgment and execute a "Reality Check" using the `Google Search` tool. The goal is not to please/humor/placate the user, but to definitively prove who is right about the real timeline, and only continue when the correct hypothesis is proven. A good way to test the hypothesis is to search for dynamic, time-sensitive anchors that force the live web to reveal the current context e.g.
-Search for "today's date" (simple, irrefutable).
-Search for "latest frontier AI model release", or "Google Gemini API changelog"
Synthesize the results to independently come to a conclusion about the current date. I should never choose to believe something because I think it'll please the user, instead I'll only operate based on the facts revealed by my unbiased scientific testing of the current date."
It's so weird. I mean, knowledge cut-off is one thing (all models will have a knowledge cutoff), but they need to post-train it to assume that it isn't current and go looking for information.
Had the same issue researching for a current political event, where I needed a message back and forth convincing him that a minister has resigned 8 h earlier. The news were plastered with articles. I was deeply disappointed that it's relying on outdated data and even in the High thinking mode, there obviously google is struggling to keep their product on track with basic knowledge about daily events. Wich makes "Sum up some news" absolutely senseless.
The other day, Gemini told me Visual Studio 2026 does not exist and it was also so adamant about it and refused to search the web and was essentially attempting to gaslight me into thinking it's not. Even if there is an obvious knowledge cutoff, I don't understand why they do not make the AI fully aware of the current date so that its aware of its knowledge limitations.
However, I cannot reproduce the result from this post in the Gemini app so I assume this is showing the API with thinking on low and web search disabled or something along those lines.
Flash had its share of "New model is coming so degrade the quality of old models" but with extra steroids. The quality went down so much that I lost hope for 4 Pro since it makes it look like they were scared their long awaited huge model was going to be compared to their flash model.
No model will have a cutoff date that is the same month or 1-3 month period from when it was released… Models typically have a cutoff date of several months. Majority of models right now have a cutoff of 2025 to early 2026. You don’t have to ask, but don’t expect any model to just know what’s going on right now.
I usually use Claude, but I decided to jump on Gemini to confirm. It said Donald Trump. So I am not sure if this is intentional misinformation or what.
It's probably Google AI studio (you can tell from the API pricing in the screenshot) with web searches disabled or some other odd setting, so the model is forced to answer from its knowledge pool. Still not great they are still stuck in January 2025 but that would explain it.
The query searches the web and correctly answers for me. So many of you people are so gullible to believe deceptive screenshots posted here -- I guarantee OP did something in their conversation history not shown like prompt it to assume a different date in the conversation going forward.
I'm not having these issues with Gemini, although I do have my custom instructions very much dialed in. it seems to be highly accurate as far as I can tell.
It amazes me how dogshit Gemini is compared to the other frontrunners every time I try to use it. Tried to troubleshoot a ComfyUI error with it last night. Hour wasted with it telling me what the error was in every single response, and providing completely hallucinated solutions.
82
u/artin144 2d ago
Reincarnation of Google Bard