Olzha provides operating leases for freight wagons to corporate clients in Kazakhstan, as well as for transportation of liquefied petroleum gas and oil products.
EBRD will provide $9m to Olzha, which is an addition to a loan worth $24.8m that was approved in 2013.
To meet market demand, this will be used by the company to purchase around 100 freight wagons and a locomotive.
EBRD Russia and Central Asia Infrastructure director Ekaterina Miroshnik said: "By supporting further development of the company, the EBRD will strengthen the expansion of the private sector supply of operating lease services and promote the application of modern business practices."
The railways in Kazakhstan play a key role in the national transport sector, accounting for around 60% of all freight movement.
EBRD and the municipal government of Almaty had signed a memorandum of understanding for the development of a 14km light rail line in Kazakhstan's second largest city in November 2012.
To date, EBRD has invested around $6.5bn in the country's economy, with more than half of the bank's projects supporting the private sector.