I am curious if it reuses the LLM across all tabs, hard to imagine most machines... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

bearjaws 4 months ago | parent | context | favorite | on: Google Chrome silently installs a 4 GB AI model on...

I am curious if it reuses the LLM across all tabs, hard to imagine most machines can boot up 1-2 of any 4gb model unless its a more powerful system.

nsvd2 4 months ago [–]

I think it obviously will, what would be the benefit to spinning up more than one copy?

wtallis 4 months ago | [–]

It should only need to load one copy of the weights, but each tab/site will need a separate context and KV cache.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact