2 points | by westurner an hour ago
1 comments
meshy-dev/overmind: Cut PyTorch model loading time from 15s to 0.2s with zero-copy shared memory caching.
> Fast Model loading: First load caches to shared memory; subsequent loads are ~5x faster
> Process-agnostic - Cache persists across process restarts via a background server
> Non-intrusive - Just add one line of code; no changes to model loading logic
meshy-dev/overmind: Cut PyTorch model loading time from 15s to 0.2s with zero-copy shared memory caching.
> Fast Model loading: First load caches to shared memory; subsequent loads are ~5x faster
> Process-agnostic - Cache persists across process restarts via a background server
> Non-intrusive - Just add one line of code; no changes to model loading logic