How to run Underdog Saluki 27B in llama.cpp: A practical guide on testing 2-bit Qwen3.8-27B with tool calling
This tutorial compares running Underdog Saluki 27B (the 7.9 GB 2-bit quantized Qwen3.8-27B version) against ISTA-DASLab's GSQ-RCO quantized version in llama.cpp, comparing GGUF headers tensor by tensor and performing byte-level comparisons. Both models are tested for memory usage and speed on the same GPU, using the BFCL official checker to evaluate 120 tasks with identical seeds under both strict and relaxed parsing criteria.
Return to original text
A short introduction is retained here; the full text from the source website is not reposted. For specific wording and subsequent updates, please refer to the original article.
Open AIHOT original