Posts and Pages are both content types and while they have a lot of similarities there are also a few differences specific to each content type. To put it simply, the differences are not unlike different kinds of publications: Pages are like Books, Posts are like Magazines.
OK, let’s dig a little deeper.
Pages generally hold content that is fairly static; once you’ve published it, while you may wish to be able to change it, it isn’t something that you’re going to have to edit regularly. It isn’t time sensitive and it remains in its place and usually will have a spot in your menu.
Pages are like a book on the bookshelf, their content is useful over the long term. They’re unlikely to change much, or very often, and they’re good to have easily accessible for reference.
Pages can also be built hierarchically in your site structure and you can apply different kinds of layout templates to them according to the templates available in your theme (and yes, this is kind of where the metaphor ends).
Posts on the other hand are time sensitive, they may be categorized, they will also be included in your RSS feed (people may subscribe to them). They will be displayed on your blog page, or front page, in reverse chronological order and they’re less likely to be needed to be included in your menu because of their more disposable nature.
See? Just like a magazine… time sensitive, usually stacked in order with the newest on the top, categorized by date and content type, and generally more interesting in the short term than the long.
If it’s easier to have a visual representation of how that looks, here’s a graphic view in which you can see where there is cross over between the content types and where there are differences.