How much of a hit would you take on quality if you moved the processing local? h... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		captainregex 3 months ago \| parent \| context \| favorite \| on: Production RAG: what I learned from processing 5M+... How much of a hit would you take on quality if you moved the processing local? have you experimented with it? don’t think llamaindex has local sadly

tifa2up 3 months ago [–]

Quite a decent hit. Local models don't perform very well in long contexts. We're planning to support a local-only offline set-up for people to host w/o additional dependencies

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact