Like individual clients, you can opt to delete an entire Company in lieu of merely deactivating it.

Deleting a Company permanently deletes all records associated with it.  You can also opt to delete all clients from that Company as well.

Just like with clients, it's GENERALLY better to deactivate a Company than delete it.