You can not delete the default uncategorized category, but you can rename it.
Posts>Categories>Uncategorized
Edit the Name box