For those of you who are new to Raspbian, Raspbian is an unofficial port of Debian wheezy specifically compiled for Raspberry Pi. There’s lots of info on Raspbian here. Although vanilla Debian is great, only Raspbian is compiled and built specifically for the Raspberry Pi. There are many specific reasons to use Raspbian.
Raspberry Pi’s ARMv6 CPU does have a hardware Floating Point Unit (FPU). The issue with standard Debian is that it only support the FPU on ARMv7-A generation CPUs and higher. Raspbian provides support for the Raspberry Pi’s ARMv6 CPU’s FPU.
In the context of a PBX which has audio codec processing needs, Raspbian is a clear winner over vanilla Debian because Raspbian has support for the Raspberry Pi’s FPU. When transcoding audio codes, having the FPU available to help is critical.