If you do care about your students, what do you not like about your job? Do you feel it is not worthwhile? Do you not make as much money as you want? Do you feel obligated to use your degree? Do you have a passion for finance and feel that you aren't really doing what you want?
Figure out what you really want to do. Then feel good about every step you take toward that goal.