At your current scale, just adding a slave for redundancy and hot take over should be enough?

There's also the aurora serverless way which may be interesting since your workload is only a few hours a day. Scale up automatically during the peak and wind down the rest of the time.