I wonder if FauxPilot's models (Salesforce Codegen family) can be quantized and ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		phantom32 on March 22, 2023 \| parent \| context \| favorite \| on: FauxPilot – An open-source GitHub Copilot server I wonder if FauxPilot's models (Salesforce Codegen family) can be quantized and run on the CPU. I was able to run the 350M model on my machine but it wasn't able to compete with Copilot in any way. Salesforce claims their model is competitive with OpenAI Codex their github description[1]. Maybe their largest 16B model is, but I haven't been able to try it. [1] https://github.com/salesforce/CodeGen

ayushkaushal on March 22, 2023 [–]

We will add quantized CodeGen for fast inference on CPUs up on cformers (https://github.com/NolanoOrg/cformers/) by later today.

meghan_rain on March 22, 2023 | | [–]

> by later today

Wow, that's the timeframe things are moving at right now, we better get used to it!

syntaxing on March 22, 2023 | | | [–]

Whoa is there a PR or wiki about this

underlines on March 22, 2023 | | [–]

4bit GPTQ maybe?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact