Depends on so many things.
In the large scale of things most people do not have a "second serve" that they can take a full swing on. For your second serve to be "good" you have to be able to take a full and confident swing. The biggest misconception about second serves is that you swing softer so it goes in more. A properly executed second serve uses a full swing.
In doubles you have to take into account how each of the receivers are taking your ball and whether or not your partner at net is being effective. If your partner at net is getting killed you may want to serve that person harder than if he/she is just doing generic returns and poaching the return is common.
In singles you have to consider how many of your first points youre winning compared to your second. If you're losing more first serves than you are second serves you're serving too hard and making too many errors.