How the Claude Code team thinks about verification

(claude.dev)

37 points | by craigmart 1 hour ago

15 comments

  • OtherShrezzing 18 minutes ago
    This article feels like the result of having unlimited tokens at your disposal. It’s got some very expensive practices, where near-free practices would produce the same or better results.

    There’s a long section in the middle where the article describes per-change verification via skill files and “ask Claude to use the chrome-for-Claude tool” when checking if changes have introduced regressions.

    But there’s already tools which do this without spending tokens tokens. Playwright will achieve equal quality but deterministic results across the same verification attempts. It is write-once-use-often, so has a marginal cost near zero.

    • xnorswap 8 minutes ago
      I agree completely, there was a much better talk on this by Lauren Tan: https://www.youtube.com/watch?v=O_zcc6Xl12g

      A key pillar was introducing things like static analysis to provide cheap deterministic checks as the first goal and only then moving to other forms of verification.

    • arbll 3 minutes ago
      What makes you think they are not using Playwright ?

      For me this article is more aimed at devs that are still stuck treating code generation and verification as two separate tasks. Giving agents the right tools (including playwright) to do both at once makes things more token efficient, not less.

    • bitlad 10 minutes ago
      They are living in the future.

      I think future AI cost will be as cheap as doing an API call in classic sense.

    • simianwords 1 minute ago
      Its almost become a meme at this point: well actually you can achieve the same using X so why west tokens?!

      The reason you should use Claude and not playwright always is because the the invariant you wanna capture is at the natural language level.

      Like imagine I wanted to verify that Claude Code works and doesn't break. I can describe the invariant I want in natural language. Like: make sure that subagents don't block the original thread.

      Doing this in playwright means that you have already bought into some previously existing architecture which means your tests are now correlated to your code. Having the invariants written in natural language allows you to decouple code vs the invariant.

      I expect a lot of greybeard conservative programmers to keep opposing using AI here until eventually, not using AI will be seen as more risky.

    • pluc 3 minutes ago
      Here's how to properly use XYZ, yes it'll cose you more but we manufacture it so to us it's no issue.

      Do you not see the propaganda through the pseudo helpfulness?

      • hackyhacky 1 minute ago
        It's the same reason they give you a recipe for Chex Mix on the back of every box of Chex.
    • baxtr 1 minute ago
      Wait, are you saying a company that makes money from selling tokens generated a content piece recommending people solve their problems by using a lot of tokens?

      Scandalous.

    • SecretDreams 2 minutes ago
      You don't get a nice juicy IPO valuation by trying to save money or be efficient with your token usage!
    • catchnear4321 8 minutes ago
      It behooves the token merchant to espouse the benefits of having more tokens.

      Not that it is an explicit, conscious direction, but it very much is one.

      You’re not wrong, in that free flowing tokens are enough to convince such a group it’s simply good advice.

    • cyanydeez 12 minutes ago
      Chinese gonna beat anthropic because they gluttonous use of AI.
  • koolba 0 minutes ago
    > Nobody can read every change, so we use failures as one way to decide what needs a person.

    Whether a human can or should review every diff is a conscious project level decision. I have yet to see a project where it is not feasible. The trick is to break things into small enough pieces that you can review them. Just like one would do if you were doing such changes manually. Nobody in their right mind performs non-mechanical changes of 100s of files.

  • Narann 0 minutes ago
    The more I read articles like this, the more I realize Claude is good for process prototyping, but you need to "bake" the process with flat code/script/organization to decrease costs.

    The future of IA would actually be to large scale prototype easily, even more like Python was a prototype language.

  • girvo 52 minutes ago
    > With them, you can run dozens or hundreds in parallel.

    Im already spending my $2k usd per month in tokens with just one or two agents. How do they expect me to do that?

    • varispeed 4 minutes ago
      This is probably aimed at gullible governments to burn tokens using tax payer money.
    • thatsabadlook 37 minutes ago
      You should fire yourself and have your boss go all in on anthropic duh. If you are the boss then you either shouldn't have gone down this path, accept your code won't be verified, or shut your doors.
      • 4lb3rn 29 minutes ago
        Exactly this. I build large systems like erp, ipaas just from a Claude team with two premium licences. everything builds, verifies, deploys itself. I can track everything and implement features in a days. security and big fixing is background work. I would love to know what 2000$ worth of tokens do in your business.
  • nandanadileep29 40 minutes ago
    excellent content from the team. very concise and clear
  • runtarmac 13 minutes ago
    [flagged]
  • oceanrelay 47 minutes ago
    [flagged]
  • mohsen1 33 minutes ago
    Verification is the bottleneck and that's why we're building https://Proofline.sh

    Starting with infra evidence and soon expanding to application verification loops. This is a very exciting place to build and we're seeing great results once the verification loop is closed for the teams

    • beatrice-costa 24 minutes ago
      how is this different from a claude code verification loop?
  • dotdi 33 minutes ago
    > https://github.com/anthropics/claude-code/issues has over 16k open issues

    this u?

    • laichzeit0 27 minutes ago
      And 90k closed. Looks like it's doing a pretty good job.
      • ahsillyme 8 minutes ago
        Issues are automatically closed after being inactive for too long. It's also true that many issues that are filed there have nothing to do with Claude Code proper. To conclude, these numbers (either many open, or many closed, or any relationship between them) a-priori mean nothing either way.
      • orbital-decay 7 minutes ago
        Opening and closing GitHub issues does not necessarily mean doing some job.
      • catlifeonmars 9 minutes ago
        [delayed]
  • Ampersander 48 minutes ago
    Why is the team thinking? Their product should be doing that for them
    • Goto80 12 minutes ago
      Why is there a team? Their product should be the team. /s
  • inamorty 13 minutes ago
    Can they not afford someone with an engineering background?
    • FartyMcFarter 4 minutes ago
      They poached a lot of people from companies like Google that have solid engineering practices. I imagine it's a culture issue if anything, not a knowledge/background issue.
  • ForHackernews 28 minutes ago
    Ironic that this page loads partially styled with broken CSS:

    "The resource at “https://claude.dev/_next/static/chunks/241gnzjfqbp9f.css” preloaded with link preload was not used within a few seconds. Make sure all attributes of the preload tag are set correctly."

  • dude250711 1 hour ago
    > ...On the Claude Code team, the vast majority of our code (~90%) is written and reviewed by Claude...

    I am more interested in the remaining 10%.

  • mococa 19 minutes ago
    So a lot of slop...
  • jonenst 3 minutes ago
    [dead]