IBM-backed llm-d framework adds controls for running AI models in-house

IBM-backed llm-d framework adds controls for running AI models in-house

IBM Corp. and several other contributors to the open-source llm-d project are releasing version 1.0 of their distributed artificial intelligence inference framework, adding operational controls intended to help enterprises run large models reliably on their own infrastructure.

The release, announced ahead of the PyTorch Conference North America in San Jose today and tomorrow, focuses on managing competing workloads, maintaining service during upgrades and improving hardware efficiency as AI agents generate…