Gemini 4 Neon looks like an Opus 5.5 competitor

(micjm.com)

66 points | by seizethecheese an hour ago ago

19 comments

  • rootusrootus 41 minutes ago

    I guess what I really want to know is how Gemini 4 Neon w/Antigravity compares with Opus 5.5/Claude Code. I'm not really into using LLMs in a particularly advanced way, I just use CC like 99% of the time when I need to write something, and then maybe Codex to second guess it.

    But that's about as complicated as I get. I have recently tried to use Copilot because that is what work wants us to use, and even with Opus 5.5 it is noticeably inferior to CC on the same model. Wild. So clearly the harness matters.

    • scottyah 23 minutes ago

      The models can be very optimized for the harness, Clem (of Hugging Face) just recently released a Qwen3.8 model that used RL training on the harness and it got some pretty significant gains in the benchmarks.

      - SWE-bench Pro (hard 51): 39.2% → 64.7%

      - Terminal-Bench 2.1: 39.5% → 55.8%

      https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted

      • seizethecheese 7 minutes ago

        I'll call bullshit on that.

        If this were true, models would do way better in their own harness than in others, which doesn't seem to be the case (effect is minor).

        Your link has a "link to the release blogpost" which is supposed to explain the training recipe but the link is down.

    • anukin 12 minutes ago

      Antigravity in general is a poor harness. I have had better experience using Gemini 3.8 flash over api than using it over agy.

    • seizethecheese 35 minutes ago

      Totally possible for your use case, though the analyses I’ve done on harness model interactions show it’s actually not that important for benchmark performance. https://micjm.com/blog/on-model-harness-interactions/

      For what it’s worth, the linked article data (Agent Duel data) is all run on OpenCode. I’m curious if people would prefer we run each model in its native harness (at the expense of user experience because it’ll be buggy).

    • thefunnyman 33 minutes ago

      We had CC for a while and now have to use GHCP. As such I haven't used CC in like 6 months but around the time I switched I felt like GHCP CLI had pretty much caught up. Curious to hear what you feel are the gaps? Are you using the cli versions of both?

    • pritambarhate 30 minutes ago

      Also last I checked there was no clear flag to turn off training in paid plan of Antigravity. So if you care about that looks like Google doesn’t want you as their customer.

  • wmf 24 minutes ago

    Projecting something that is already out is an epistemic crime.

    • seizethecheese 12 minutes ago

      Author here, guilty as charged. To be fair, we don't have early access to any models (just getting started).

      The blog itself can be seen more as an exploration of how speed impacts coding agent battle votes, than anything specifically about Neon.

      Applying the speed impact to Gemini Carbon is particularly interesting, though, because it's the top model in our leaderboard when you exclude early votes (https://agentduel.com/leaderboard?excludeEarlyVotes=true) but mediocre when you let developers vote. Gemini literally takes 3x as long as opponents in matched battles.

  • bwat49 30 minutes ago

    I'm pretty impressed so far, this is the first google model I've used that feels good for coding

  • quinndupont an hour ago

    Fun analysis. I guess it’s not surprising to see the need for speed™ as an important factor, but who is using Fable for run of the mill coding work? It’s too pricey!

    • seizethecheese 40 minutes ago

      Yeah, check out DeepSeek performance on the leaderboard (http://agentduel.com/leaderboard). It wins ~half the time because it’s lightning fast and good enough.

    • hedgehog 32 minutes ago

      I used it to write something last night that was intricate enough that Opus would have involved a lot more rework. Subscription discount though.

  • fud101 6 minutes ago

    I'm blown away by Neon, it's as good as Argon (which was amazing) but fast and cheap (i think). Opus is trash, Fable is trash - I will just use Opus for Day Job which it handles with ease and keep Neon for my serious side projects which Claude can't handle well at all.

  • afavour an hour ago

    Looks like a good bunch of speculation in there but no bad news to be found. Just as good but cheaper? Sign me up.

    For this model, I mean. And not the Anthropic IPO.

    • seizethecheese an hour ago

      Lots of speculation, yes.

      Just as good but cheaper comes from the benchmark performance. Also faster from openrouter. The opus 5.5 parity projection is an internal model based on those assumptions from the Agent Duel team.

    • enraged_camel 19 minutes ago

      Nobody with money is gonna look at this model and change their opinion on the Anthropic IPO lol

  • lacoolj 14 minutes ago

    First, this post has a lot of words like "assume" and "projected". What are we actually reading here?

    Second, I will be trying this with agy when my Opus 5.5 limit hits. I only hope its faster than Argon which is already dog slow

  • ralusek 5 minutes ago

    This title reminds me of the doctor from Arrested Development.

    "It looks like he's dead."

    "...He's covered in blue paint, or something."