There's quite a bit of background about William Sianis and the Billy Goat in this article.
I knew about the Cubs game portion of this story, but hadn't heard about what happened with the goat and Sianis in the eleven years before the Cubs incident. It's pretty interesting if you're into Chicago history.