Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B

Summary

Semalith v1.4 is a DeBERTa-v3-base security classifier with 184M parameters that claims to achieve top-tier prompt injection detection while using 44 times fewer parameters compared to Llama-Guard-3-8B.

Why it matters

The model addresses the demand for lightweight, multi-faceted security classifiers in financial services and agency applications by delivering a zero false positive rate on benign prompts. Its efficiency could enable the broader deployment of security safeguards in resource-constrained environments.

Highlights

• It covers 22 categories, including benign prompts, nine subtypes of prompt injection, general harms, and eleven BFSI tags.
• It outperforms Llama-Guard-3-8B in all prompt injection benchmarks (7/7) and 11 out of a total of 18 comparisons.
• It achieves an FPR of 0.000 on 208 benign agency prompts, whereas Llama-Guard-3-8B records an FPR of 0.063.

Critical take

The article notes that Llama-Guard-3 still leads in general harm benchmarks, suggesting that Semalith's superiority is limited to prompt injection and BFSI contexts.