I want you to explore your site's files a bit using either the Site Explorer or FileZilla. Here's what you might find:

wp-content This is the folder that all of your customizable elements are held. Theme files, Image Uploads, Plugins and more are all stored in here. If you want to delete an annoying plugin, simply navigate to this folder, go into the Plugins folder and delete the offending plugin's folder.

You can also investigate how Wordpress stores your image uploads.

If you want to mess around with your theme, the files are stored in wp-content >> themes.

File Permissions

There is an easier way to edit your themes, and that involves editing file permissions. I'm not going to double up on content here, rather point you to welshy's excellent tutorial on this subject.

These are primarily the reasons I use FTP. As I mentioned in the introduction, you may never need to use FTP. But if you do, know it's very simple. And when in doubt, ask a question in the comments!

Thanks,

Mark