AI can be very useful thought partners, but they aren't especially great thinkers especially when their context window fills up. How do you know you've gone too far? When it says: "You're absolutely right".

Think

One of the main takeaways from the Engineering track is DO NOT OUTSOURCE THE THINKING: Context Engineering & Human-AI Collaboration where the best techniques for effectively using AI in coding tasks are starting to crystallize.

The key thing is to manage your context window. That's all your messages that you've sent, the tokens generated by the model (thinking and otherwise) and all the junk that the tool calls have returned. And remember: you are sending the entire context back every time on every message! So your token usage doubles on every message. "Thanks for all your help!"

Regardless of what the marketing says, there seems to be a decline in utility well before you get to max context size. Geoffrey Huntley somewhere says keep it under 170K. Dex Horthy says over 40% and it's entering the "dumb zone" - so what are some techniques for not making the AI dumb?

Avoiding the dumb zone

• SKILLs and other progressive discovery techniques

  • Limit the MCPs you have

  • Tell the LLM where to find the information it needs rather than immediately stuffing it into context

• Keep prompts small and concise

• Intentional Compaction -- don't wait till it gets back, compact as you go

  • Research Phase -> Compact into design docs

  • Planning Phase -> Compact into signatures, files, modifications

  • Implementation Phase -> Execute with clear specs

• Use the filesystem as extended memory, write things to markdown files and explicitly reference them later

• Subagents -> Burn a bunch of tokens on a side quest and only return the result to the main thread not the chatter; or better yet write to the file system, check out our tech-researcher agent

Bonus: Warning Signs You've Hit the Dumb Zone

• Excessive agreement ("you're absolutely right")

• Model repeats your words back

• Confident but wrong suggestions

• Forgetting earlier constraints

• Repetitive/looping behavior

Check out the complete article on thefocus.ai AI Engineer Summit Report.

What's your experience for keeping the agents smart?