Why is the billy goat so fascinating? It is a nasty beast, it smells bad, it attacks you, you can´t eat his meat, a symbol of maliciousness and all evil.
But without it you can not breed yur lifestock.
So I think, it stands among other things for the necesssary evil, that we have to approve in order to have a good life, like so many things...and off course for fertility ;)