The Mages' Guild is easily identified by the plants growing all over its walls.
The Mages' Guild is a very scholarly place, with a large library of books to read. But hey, we all know why most people join up - to tap into powerful magicka. And for a beginner, that means buying spells. So the person everyone talks to (at least at first) is Gondore Mooring.
Certain spells are almost absolute necessities:
You'll want to get a Heal spell right away. You'll sometimes need to cast heal spells during battle to avoid getting killed, especially early on in your career when you won't be able to withstand very much damage.
Another way to deal with damage is to get a shield spell. They tend to take a lot more magicka to cast than a heal spell, but sometimes your enemy is strong enough to kill you with just one or two blows - leaving you with insufficient time to cast a heal spell. So casting the shield spell before going into battle may be your only chance of defeating your enemy.
Of course, one of the best ways to survive a battle is to avoid it. One way to do that is to sneak past your enemy. An invisibility spell is essential if you're going to try to do that successfully.
You should also get a levitation spell as early as possible. You definitely don't want to fall into a pit or something in a dungeon and not be able to get out!
Well, I did say that the Mages' Guild was a scholarly place, and if you want to be allowed access to the really good stuff, you're going to have to be a good student. The teacher, Uthore Moorsmith, may not look like much, but he can teach you several languages, and more importantly, enlighten you in the six schools of magicka.
Another thing you'll need to do if you want to get in good with the Guild is to run some errands for them. Theodyval Kinging will have you rounding up fresh ingredients for magic potions, killing off run-away magicka experiments, and guarding the Guild and its members against its many foes.
If you're lucky, you'll find some magical items early on during the course of one of your quests. It can be mighty dangerous playing with unknown magicka, so you'll want to pay Agrane Copperston a visit. He'll identify your items for you, for a price of course. You'll probably obtain an identify spell of your own at some point, but even then it's usually easier to just leave it to a professional.
After you advance through a few of the ranks, you'll be granted access to the spell maker. You can design new spells that precisely match your adventuring style and your character's abilities. Just give Elausa Wickston a visit, and bring your gold.
Now casting spells is invaluable, but it is also limited by your own inherent magicka and your ability to boost it with your intelligence. You can often find yourself surrounded by Ancient Liches with not even enough spell points for a decent shield spell, much less a soul trap spell. Magic items carry their own magicka, and some are enchanted by binding them with the souls of powerful creatures. And of course you have to have soul gems to hold the spirits of the dead if you're going to trap their souls. To buy some of these goodies, just talk to Alabistair Hawkhart.
It will take you a while to gain access to the magic item maker, but it is certainly well worth the wait. You'll probably get quickly tired of the few under-powered, short-lived items that you got from that last guy, and you'll be ready to make your own. Just go to Bedyn Kingham and bring lots of stuff to enchant.
There are a couple of other services at the Mages' Guild of which Gaimmienne knows, the Daedra Summoner, Andoryan Moorfield, and the Teleporter, Theodywyr Hawkston. However, the Guild in all of its wisdom has not yet seen fit to grant her access to these. Anyways, who needs a Daedra Summoner? Gaimmienne finds enough Daedra already, and she likes to just split them down the middle with her wakazashi and take their gold - they're usually packing quite a bit.