Once you are in the subaccount, click on Settings (bottom left)

Click on Phone Numbers

Click on Phone Numbers

Click on the trash icon to remove the number

Click on highlight

Once you confirm by typing Delete below, Click on Confirm Delete

Click on Confirm Delete

If you want to keep/reset the twilio number webhooks, click on NO here so the number will still exist in your Twilio sub-account.

Otherwise, confirm by typing Release below and click on Confirm Release.

Once you click on NO, 

Click on Add to add back the Twilio number to the account, this helps to reset the Twilio number's webhook

Learn more about Restoring a previously released phone number