Local AI developers report GPU speed gains as agent latency remains stubborn

Consumer GPUs become a tuning laboratory

@sudoingX described a project that grew from a single llama.cpp setting into 78 measured configurations contributed by 60 people. According to the post, enabling multi-token prediction raised an RTX 3090 from about 41 tokens per second to the mid-60s, while a contributor reached 84.4 tokens per second on agent tool calls by increasing draft depth. The author also reported 191.9 tokens per second on two RTX 5090s in tensor parallel. These figures concern different configurations and workloads, rather than a single interchangeable performance ranking. Source

The more useful finding may be the project's account of configuration trade-offs: the author said optimal draft depth varied by hardware and workload, a confidence gate helped slower cards but hurt faster ones, and dual-GPU setups needed their split mode addressed first. In a separate post, the same developer reported running a 27-billion-parameter system with vision and a 256,000-token context on a 16GB RTX 5060 Ti, using roughly 15GB. Computer-use tests and several simultaneous agents were described as the next step, not completed results. Reported setup

Another performance claim, shared by Ivan Fioravanti from @WescheNex1q, put TensorFold at three times the speed of vLLM on the same DGX Spark using three prompts with thinking disabled. The visible post does not supply enough detail to turn that small comparison into a general recommendation. Comparison

Faster browsing does not guarantee a faster agent

@MilksandMatcha repeated a dinner-reservation experiment after reportedly receiving news of browser-harness improvements. The reported completion time fell from 7 minutes 40 seconds to 7 minutes 16 seconds, against 37 seconds for the author and 22 seconds for a Cerebras comparison. The author cautioned that the agent's minute-level prompt timing introduced roughly 30 seconds of uncertainty, making the two agent totals effectively similar. Browser-use time had fallen substantially, the post said, but extra reasoning and orchestration consumed the savings. Experiment

Elsewhere, a Hex-Rays announcement reshared by @badlogicgames introduced an official IDA MCP Server, describing it as free, open source and compatible with any large language model. The announcement says an agent writes IDAPython; its remaining performance claim is truncated, so no efficiency conclusion follows from the visible text. Announcement

On-device experiments move beyond chat

Fioravanti reported an updated interactive diary experiment running locally on an iPad Pro M5 with iPadOS 27. His description combines MLX-based inference, voice through mlx-audio-swift and Apple Vision for handwriting, with a demonstration planned for the following week. In another hands-on experiment, @KingBootoshi said an AI system designed a fidget toy, tested it in MuJoCo and then produced a 3D print. These are accounts of individual projects, not evidence of general reliability. Diary · Fidget toy

The economics of owning that compute also drew debate. @kunchenguid argued for buying hardware rather than renting a VPS, comparing a $2,500 Mac mini used heavily for app development with an estimated $300 monthly rental for similar specifications. On those quoted prices, rental exceeds the purchase price within nine months. That is the author's cost comparison, not a verified like-for-like hosting assessment. Argument

Infrastructure releases and company announcements

Beyond local inference, a post from @confusedqubit reshared by @jezell announced that Loophole Labs was joining LiveKit; the visible excerpt gives no transaction terms. A separate release announcement from @rough__sea said celld v0.6.0 had arrived, that the project was now considered beta, and that changes were mostly bug fixes. Company announcement · Release

A TiDB X post from @siddontang described an Infrequent Access approach that keeps SSTs in object storage, prepares metadata locally and fetches segments on demand. Separately, a Latent Space podcast headline reshared by @MilksandMatcha presented Stripe's acquisition of OpenRouter as its subject. The supplied excerpt contains no deal terms or supporting announcement, so that acquisition remains a claim in the podcast headline here. Storage design · Podcast headline