Russian forces have began a gradual withdrawal from the Ukrainian border, a Ukrainian defence ministry official told AFP, without specifying the number of troops involved.