How soon can we see improvements when we start using Scrum?

Managers , take it easy. In many organizations there is an urgency to measure, look at numbers as they move to scrum within the first two sprints. As a general guideline, do not take too much notice of the statistics for four or five sprints.

Teams take a while to form and norm. They also make a lot of mistakes and tend to work a lot harder in the beginning.  Remember the Agile priciple

Simplicity–the art of maximizing the amount
of work not done–is essential

Teams take a while to master this. Once they have this and resoved team forming, norming, that would be a good time to start comparing them with others.