GIF is the older of the two formats, it can have a maximum of 256 colors out of a large palette. It compresses images and works most efficiently when the image contains large regions of the same color.
JPEG is a "lossy" form of image compression which is designed to store "real world" images (meaning those with continuous variations in tone). It stores 24 bits of color (16.7 million different possible colors) and can reduce files to a tenth the size of a similar GIF (which saves network bandwidth!).