https://research.google/blog/making-llms-more-accurate-by-using-all-of-their-layers/

This paper put out a year or so ago, and referenced by today's blog post, shows a method for decoding using the weighted average of every layer's logits. It improves factuality over DoLa (which itself improves over just standard sampling?) by anywhere from 2-16%with only a 4% hit to speed! I'm surprised I haven't seen this here since it seems like it shouldn't be too bad to implement into something like VLLM or llama.cpp, and it seems to work for many different models.