You can delete a mini-budget by delete the expense associated with it. Read the 'My Expenses' help section to find out how to delete an expense.